gpuOwL-specific reference material

[kriesel]

There's a separate post in the Google Colab thread for setting that up.

Ewmayer has recently created a comprehensive guide, at https://mersenneforum.org/showthread.php?t=25601

Earlier:
Prime95 suggested for setting up on a local Linux system, https://mersenneforum.org/showpost.p...5&postcount=76
and "also do a sudo apt install libncurses5" https://www.mersenneforum.org/showpo...postcount=2242
Ernst's needed packages list https://mersenneforum.org/showpost.p...postcount=2243


Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

GPU ram occupancy by Gpuowl v6.11-340 was observed using GPU-Z v2.33 on Windows 10 during performance of PRP iterations, for several scattered exponents. Idle gpu ram occupancy was entered for exponent 0. V6.11-364 was spot tested and observed essentially identical for the same exponent. At 100Mdigit, LL gpu ram usage was observed to be about 80% that of PRP. Note, no PRP proof computations have yet been observed for gpu ram usage. System ram usage for the process was observed to be less than gpu ram usage.
The regression fit was somewhat better plotted against FFT length than against exponent, not surprisingly.


Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

The script requires Python 3. Programming environments are not standard parts of a default Windows install, so Python, perl, C etc may be absent. Python may have been already installed as part of a MS Visual Studio install, or separately.

Check whether your target system already has Python, and which version.
On Windows 7:
Start, Control Panel, Programs, Uninstall a Program, scroll down the list of installed programs to see if a version of Python is installed.
Or on Windows 10:
Settings, Apps, scroll down the list of installed programs to see if a version of Python is installed.
If no Python 3.x is installed, follow the installation directions at https://docs.python.org/3/using/windows.html
Test that it can be easily invoked from a command prompt.
At a command prompt: python
If python is included in the path environment variable, it will respond with something like

Python 3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>>

(Ctl-Z Enter to exit interactive python.)
If it does not recognize python, it might just be missing from the path environment string.
On Windows 7:
Fix the path in a permanent way so that Python can be run in any future command prompt:
Start, Control Panel, System, Advanced System Settings, Environment Variables...
In the System variables portion, if it's not present, append something like the following to add the location of the Python folder on your system to the path
;C:\Program Files\Python36
(Scroll down to Variable "Path", click the line, click "Edit...", scroll to and click on the right end of the existing path, paste in the necessary additional text, click OK.
(This was necessary in my case because Python was not present in the path, and the cmd line setx /M did not work because the path was already beyond its 255-char limit, thanks to a lot of NVIDIA related content, and ordinary set can handle the long path length but is temporary, as is the command prompt box in which it is set.)

Make a hi.py file to test the Python install, containing

print ('Hello!')

and try it:

>python hi.py
Hello!

>

If it worked, continue. If not, check that path modification.

If Gpuowl's primenet.py and upload.py are not present, copy them in. For test purposes I put mine in a gpuowl working directory.
If you have previously reported results in the results.txt file, now is a good time to move or remove them. (Otherwise they will generate "already reported" errors.)

Try running primenet.py with no parameters:

>python primenet.py

If the PYTHONPATH environment variable is not set properly or at all, it will produce something like

Traceback (most recent call last):
  File "primenet.py", line 9, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Create new system environment variable PYTHONPATH, per a hint from Mihai, since I found it did not exist.
On Windows 7,
Start, Control Panel, System and Security, System, Advanced system settings, Environment Variables..., scroll the system variables list. If there's no PYTHONPATH there, click New..., enter the Variable name PYTHONPATH, and enter a string corresponding to the location of requests.py as the Variable value.

By experimentation, the value needed seems to be in my case,

C:\Program Files\Python36\Lib\site-packages\pip\_vendor

There should already be a proof subfolder in the working directory. Otherwise it will generate an error from the script.
When the script runs it will create subfolders __pycache__ and uploaded.
It requires at least a username. Following is the beginning of a successful run.

>python primenet.py -u primenet-uid

Work type: 150
Will fetch ahead 2 tasks. Check every 3600 sec.
User: primenet-uid
Watched dirs:  ./
Primenet password: primenet-pwd

It will wait and loop until terminated.

For reference, from gpuowl's readme.md, primenet.py's case-sensitive options are

## Primenet.py Arguments
-h, --help            show this help message and exit\
-u USERNAME           Primenet user name\
-p PASSWORD           Primenet password\
-t TIMEOUT            Seconds to sleep between updates\
--dirs DIR \[DIR ...\]  GpuOwl directories to scan\
--tasks NTASKS        Number of tasks to fetch ahead\
-w \{PRP,PM1,LL_DC,PRP_DC,PRP_WORLD_RECORD,PRP_100M\}   GIMPS work type

Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Gpuowl now supports generation of PRP proof files. These require uploading to the server for verification. Preda has provided primenet.py and upload.py for performing the upload. Installing Python on every system to use them may be unappealing or impractical for some. Here are some alternatives. Note, upload the JSON results records first, then the corresponding proof files later.

1. Python interpreter, primenet.py, upload.py on each system. (Might not be allowed by corporate policy, disk space, available user time, etc.) This is I think the preferred method supported and created by Preda, author of Gpuowl.
2. Python interpreter, primenet.py, upload.py on one system. Manually or by small script copy proof files over from other systems, to a file share on the Python-equipped system in which primenet.py operates.
3. Upload via a prime95 or mprime install or multiple installs. As George described it (mostly) at https://www.mersenneforum.org/showpo...&postcount=220

   1) Manually submit your results.json.txt first, before giving prime95 or mprime a chance at the proof file.
   2) move the proofs to a folder where prime95 v30.2 (or newer, or mprime) is running.
   3) wait for the proofs to get uploaded (I archive my proofs using advanced resource limits dialog box).

   Note that applied to prime95; for gpuowl submit results.txt contents in step 1. Note that if there's an MD5 mismatch on the proof file, it may just loop on retries ~65 minutes apart until you manually intervene, logged in results.txt, in prime95 V30.3b3, resembling the following.

   [Fri Sep  4 07:51:19 2020]
   Unexpected error during 58000013-8.proof upload: {"error_status":401,"error_message":"Unauthorized","error_description":""}

   A nonnull error description would be helpful there; something like "MD5 mismatch detected", or whatever the case may be. I think this is a server-side consideration.
4. For V1 proofs produced by gpuowl V6.11-380 to v7.1-x, but not V2 proofs produced by gpuowl V7.2 or later: Use the standalone uploader.exe program Prime95 previously provided for 64-bit Windows at https://www.mersenneforum.org/showpo...&postcount=154. (No longer available on dropbox from that link. See attachment on this post below. Uploader reposted with George's permission.) Usage is
   

   uploader user_id proof_filename[ chunk_size[  upload_rate_limit]]

   with chunk_size expressed in MB and upload_rate_limit expressed in Mbps apparently.
   A file share and tiny batch script can expedite this.

   for %%a in ( *.proof ) do uploader your-primenet-uid %%a

   Or for a bit of logging of the uploads,

   for %%a in (*.proof) do powershell ".\uploader.exe your-primenet-uid %%a | tee uplog.txt -Append"

   To include time stamps in the logging, and console output:

   prompt $d $t $p$g 
   for %%a in (*.proof) do powershell "echo start upload %%a %date% %time% >> uplog.txt;.\uploader.exe kriesel %%a | tee uplog.txt -Append; echo end upload %%a %date% %time% >>uplog.txt"

   Retry in case of upload failure is manual; remove or rename successfully uploaded proof files or exclude them with the filespec. Delete the local proof copies after successful upload, if at all. (I'm currently using uploader.exe on a file share for occasional manual upload of gpuowl-produced proof files, and deleting local proof files after successful upload. Earlier I was waiting to delete until after a successful verification was manually confirmed.) Note, while prime95 and mprime's upload capability is being maintained, the standalone uploader may not be, and error handling may be somewhat less effective. If I recall correctly, this was created as a stopgap, before prime95 included proof file upload capability. Mprime/prime95 upload is preferred over the standalone uploader. The standalone uploader may require more than one user initiated try, as in the following.

   Thu 08/27/2020 10:59:53.11 E:\PRP-PROOF-CACHE>for %a in (*.proof) do uploader kriesel %a
   
   Thu 08/27/2020 10:59:53.12 E:\PRP-PROOF-CACHE>uploader kriesel 160456789-8.proof
   MD5 of 160456789-8.proof is 5f8f1c693b35765a465072735a042649
   Proof file exponent is 160456789
   Filesize of 160456789-8.proof is 180513949
   No entries in need list: {"error_status":401,"error_message":"Unauthorized","error_description":""}
   
   Thu 08/27/2020 10:59:55.76 E:\PRP-PROOF-CACHE>up
   
   Thu 08/27/2020 11:01:23.41 E:\PRP-PROOF-CACHE>for %a in (*.proof) do uploader kriesel %a
   
   Thu 08/27/2020 11:01:23.42 E:\PRP-PROOF-CACHE>uploader kriesel 160456789-8.proof
   MD5 of 160456789-8.proof is 5f8f1c693b35765a465072735a042649
   Proof file exponent is 160456789
   Filesize of 160456789-8.proof is 180513949
   Success!
   
   Thu 08/27/2020 11:01:46.65 E:\PRP-PROOF-CACHE>

   Occasionally an upload will report an error, but a retry will report it has already been uploaded.

   E:\PRP-PROOF-CACHE>powershell ".\uploader.exe kriesel 39000037-8.proof | tee uplog.txt -Append"
   MD5 of 39000037-8.proof is ed9f4f30202027b27a6bbed5caa84691
   Proof file exponent is 39000037
   Filesize of 39000037-8.proof is 43875102
   CURL library error: Operation timed out after 180012 milliseconds with 0 bytes received
   
   E:\PRP-PROOF-CACHE>powershell ".\uploader.exe kriesel 41000051-8.proof | tee uplog.txt -Append"
   MD5 of 41000051-8.proof is 640129af8806cfb84b84084bb84bf284
   Proof file exponent is 41000051
   Filesize of 41000051-8.proof is 46125120
   CURL library error: Operation timed out after 180011 milliseconds with 0 bytes received
   E:\PRP-PROOF-CACHE>up
   
   E:\PRP-PROOF-CACHE>for %a in (*.proof) do powershell ".\uploader.exe kriesel %a | tee uplog.txt -Append"
   
   E:\PRP-PROOF-CACHE>powershell ".\uploader.exe kriesel 39000037-8.proof | tee uplog.txt -Append"
   MD5 of 39000037-8.proof is ed9f4f30202027b27a6bbed5caa84691
   Proof file exponent is 39000037
   Filesize of 39000037-8.proof is 43875102
   Server response missing URLToUse: {"error_status":409,"error_message":"Conflict","error_description":"Proof already uploaded"}
   
   E:\PRP-PROOF-CACHE>powershell ".\uploader.exe kriesel 41000051-8.proof | tee uplog.txt -Append"
   MD5 of 41000051-8.proof is 640129af8806cfb84b84084bb84bf284
   Proof file exponent is 41000051
   Filesize of 41000051-8.proof is 46125120
   CURL library error: Operation timed out after 180005 milliseconds with 0 bytes received
   E:\PRP-PROOF-CACHE>up
   
   E:\PRP-PROOF-CACHE>for %a in (*.proof) do powershell ".\uploader.exe kriesel %a | tee uplog.txt -Append"
   
   E:\PRP-PROOF-CACHE>powershell ".\uploader.exe kriesel 41000051-8.proof | tee uplog.txt -Append"
   MD5 of 41000051-8.proof is 640129af8806cfb84b84084bb84bf284
   Proof file exponent is 41000051
   Filesize of 41000051-8.proof is 46125120
   Server response missing URLToUse: {"error_status":409,"error_message":"Conflict","error_description":"Proof already uploaded"}
   E:\PRP-PROOF-CACHE>

   In my experience these do go on to get certs run, successfully, so seem not to be an issue. Note that if there's an MD5 mismatch, it may look sort of like this:

   MD5 of 58000013-8.proof is fed107d6833f4c8e1f67a450e15272bc
   Proof file exponent is 58000013
   Filesize of 58000013-8.proof is 65250075
   Server response missing URLToUse: {"error_status":401,"error_message":"Unauthorized","error_description":""}

   A nonnull error description would be helpful there; something like "MD5 mismatch detected", or whatever the case may be. I think this was a server-side security consideration. It has been changed to give a list of possible reasons, which the end user can check. At some point the standalone uploader may become no longer compatible with gpuowl or future proof file versions in general. That may be V7.2 or proof version 2. So far, I've seen these combinations (and more):
   

   gpuowl V# ok? examples
   V6.11-364 yes 66000013
   V7.0-35   yes 150714349, 149929849
   V7.0-40   yes 123456841
   V7.1-1    ? stay tuned
   V7.1-11   yes 153021377
   V7.2-13   no  120132049 "Error getting version number from proof header"

5. Roll your own uploader script using curl or perl or whatever. For example, https://www.mersenneforum.org/showpo...&postcount=271
6. Translation of the python scripts to a preferred language, such as perl. This is unappealing, as it would create a recurring maintenance workload whenever Preda revises his python scripts. Also knowing both the preferred language and knowing or learning python.
7. Compilation of the gpuowl primenet.py, upload.py into a standalone executable. (I'm exploring this. If it works out, the primenet executable could be included with future Windows gpuowl build posts.) This would approximate #1 above, without the need to install a Python interpreter on every system running gpuowl.

Possibly some client management applications will add proof uploading capability at some point, also. (Mlucas primenet.py seems likely)
The total number of different uploaders ought be limited to a manageable set for the server maintainers to support on that end.


Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

There was a comment in the http://mersenneforum.org/showthread.php?t=22204 thread several months ago that 99% of gpuowl users ran it on Linux. This led me to think about what the real mix of gpuowl usage versus OS was, and where to find indications. I came up with several, which vary in mix indication, but seem to indicate about equal usage of Windows and Linux. These were examined in July, and I'm finally getting around to posting it now.
This post is not about bashing any participant, OS, etc. but about examining what usage distribution is by all practical means. All the following are limited by small or moderate statistical sample size. Within that limitation, they are consistent with gpuowl usage rates roughly comparable between Linux and Windows.

1) I asked the commenter for the basis of the statement. There was no response. I now believe it was hyperbole.

2) I started a poll. The results indicated 48% Linux, 57% Windows. https://www.mersenneforum.org/showpo...34&postcount=5

3) I reviewed the gpuowl thread for what users who posted there indicated about their own use. That showed 44% Windows-only, 37% Linux only, 10% both, 0% other, 10% undetermined (rounded to the nearest percent), or 54% Windows, 47% Linux after factoring in users of both (including Google colab users like myself as Linux usage).

4) I reviewed all the posts of Windows or Linux builds, as of mid June, and tabulated then how many views (downloads) were indicated as having occurred for each. There were many Windows builds posted, but only one for Linux.
Average number of downloads per build post for Windows was ~31; for Linux, 16, a ratio of nearly 2:1. (Maximum download count per build was 121 then, in post 1877). The build view count if accurate is a lower bound on users, since others may build their own, and because kracker and I and perhaps others post builds, not download our own builds. There does seem to be a decline over time in the number of views (downloads) per Windows build posted. This could be reflective of a shift toward Linux or off Windows, or simply that downloads are cumulative.
The scatter is quite substantial. One theory for it is that a moderator has modified some of the counts. Another is the long gap in builds prior to that post 1877 and another that had 90 views led to an unusually high number of updates using the build. Another is a bot or spider regularly retrieving it. Another is that some builds offer new features of significant value that will motivate a large number of downloads. In considering the nearly 2:1 ratio, note that Linux users are much more likely to do their own builds, for various reasons, including the large number of Linux variants, and the prevalence of zero cost build environments in Linux.

5) I asked James Heinrich (administrator of mersenne.ca and the GIMPS software download mirror) if he analyzed server logs for the download.mersenne.ca software mirror site. He indicated no, but sent me (anonymized) summary log data for May 2020 and oked sharing analysis of it.
For gpuowl, this indicated downloads for Windows, but not Linux. The counts likely are contaminated somewhat by spiders and bots activity. This analysis seeems to indicate substantial Windows usage on all GIMPS gpu applications, as well as on prime95.

6) James Heinrich also provided anonymized summary log data for early June. From June 1 to June 15, 2020, there were 8 downloads of gpuowl for Linux, and 9 of gpuowl for Windows. Again, rough comparability and small sample size.
The overall download rate of all apps was higher for Windows (40), than for Linux (22)

7) I asked Mihai Preda (gpuowl author) about any available statistics from the github repository for gpuowl. None were available. They would not reflect OS usage very well, since every time msys2 is used to create a Windows build, it is probably mistaken for Linux. And that one access results in dozens of downloads from builds posted to the forum.

8) Somewhat arbitrarily averaging as follows, the various measures above, combining 5 and 6 because they are the same measure for two different time periods, gives an aggregate of all samples available:

Average of 2, 3, 4, (5+6)/2
      Linux Windows
2       48    57
3       47    54
4       34    66
5 /2     0    50
6 /2    18    32
average 37    65

None of the above indicates how many gpus are running gpuowl per user or total or versus OS, or any difference in prevalence of fast versus slow gpus. I build and post once and deploy the build to many gpus on Windows. Other users are running multiple-gpu systems on Linux (including George and Ernst and Mihai).

Little of the above indicates the impact of cloud computing. Paid cloud computing price structures heavily favor Linux. Free Google Colaboratory usage is Linux-only, including my own regular attempts to do P-1 or TF there. In fact that is why I list myself as using both Windows and Linux in the tabulation of users in the gpuowl thread.

From the available sparse data we can conclude that the Linux and Windows contingents were roughly comparable. All the available measures indicated higher Windows adoption than Linux adoption. Mihai Preda made a wise choice to support both from the very first week of the original gpuowl announcement thread.

There are reports of speed advantages using Linux with the ROCm driver, and a few features require that driver. It's likely to motivate some migration from Windows to Linux/ROCm over time. Unfortunately while WSL2 is bringing some gpu support, it's CUDA and DirectML, not OpenCL which gpuowl depends on. Maybe later.


Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

At least the following, to varying degrees:

The gpu model has a great deal of effect. Radeon VII or RX6x00 preferred for performance/cost ratio. See https://www.mersenne.ca/cudalucas.php

OS overhead in general: headless or at least low graphical activity on the gpu(s), few services, Linux

Gpuowl version; a fairly recent one, v7.2-21 or v6.11-364 to 380

gpuowl performance is best with -use ASM where compatible.
-use ASM I think requires OpenCL support and gpu driver support.
The Windows Adrenalin driver does not support ASM. https://mersenneforum.org/showpost.p...postcount=1288
The Linux amdgpupro driver (compiler?) does not support ASM. https://mersenneforum.org/showpost.p...postcount=1292
The ROCm driver supports ASM as implemented by Preda in gpuowl.
ROCm driver is only supported /available on Linux. https://github.com/RadeonOpenCompute/ROCm
ROCm is not supported on Windows and does not appear to be a priority https://github.com/RadeonOpenCompute/ROCm/issues/390
"Failed ASM implies Windows. AMD's Windows OpenCL support for the Radeon VII is awful." Prime95, https://mersenneforum.org/showpost.p...&postcount=282

There's considerable variation in throughput versus fft length https://mersenneforum.org/showpost.p...postcount=2272

There's noticeable variation in performance versus driver version.
For ROCM version,
for better: https://mersenneforum.org/showpost.p...postcount=2043
or worse: https://mersenneforum.org/showpost.p...postcount=2316
https://mersenneforum.org/showpost.p...postcount=1089
Similar variation or more occurs in Windows driver version performance. I've seen up to 5+% drops for Windows Adrenalin driver "upgrades", or slight improvement.

Other variables affecting performance include ambient temperature, fan curves, and power curve settings; some users reduce power and throughput to improve performance/cost ratio. Throughput is less than linear with power.
Linux by Preda: https://mersenneforum.org/showpost.p...postcount=1623
Windows 10 by Kriesel: https://mersenneforum.org/showpost.p...postcount=1625

Also running two instances per gpu, as in https://mersenneforum.org/showpost.p...postcount=1937

Testing alternatives for same or similar fft lengths, and using the fastest that is sufficiently reliable

Overclocking especially gpu ram, but only up to the point where it's still reliable. Explore those limits running PRP/GEC which will very reliably and quickly detect errors. Back off from detection of errors, down to where error rate is less than 1/week, and a little bit lower for stability in LL DC or P-1. There are reports of reliable memory clock rates up to 20% above nominal on some Radeon VIIs.


Top of this reference thread: https://www.mersenneforum.org/showthread.php?t=23391
Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1