prime95-specific reference material

[kriesel]

This thread is here for comparison to the GPU-based applications. Please use the reference discussion thread https://www.mersenneforum.org/showthread.php?t=23383 to make comments or suggestions.

Mprime and prime95 are Intel-compatible-processor-specific. Older processor models will have limited if any support. For ARM and other not-Intel-compatible CPUs, see mlucas.

Stable version:
There is a version of v30.8 (build 17), available at the location for what's considered stable, https://www.mersenne.org/download/. V30.3 and later are PRP-proof capable versions, which greatly reduce the effort of verification of a primality test, but will require considerably more disk space to accomplish that. See the readme.txt and other documentation included in the compressed distribution file, for more info on that.

Older versions:
There are also older versions 30.7b9, v29.8b7 and older, for legacy operating systems, available at https://www.mersenne.org/download/.

Newer versions:
Generally George leaves the stable version on the download page while newer versions are available and in active development or testing. There is typically a thread for that activity, including occasional announcements of / links to new builds.

Coming attractions:
There is V30.10 now at b4 with ECM improvements in development.
Pre-Beta version of prime95 v30.11b1 https://mersenneforum.org/showpost.p...&postcount=195
Pre-Beta version of prime95 v30.12b1 https://mersenneforum.org/showpost.p...&postcount=235

Custom builds and source code:
Custom builds are not recommended. George provides prepackaged prebuilt prime95/mprime/etc images for numerous OSes.
For the truly ambitious or foolhardy, there is the possibility of downloading the source code and attempting to modify and compile and run their own builds. Note however that the security module source code is not included in the available source code. I'm unsure how prime95 results lacking an accurate security field in their results will be handled by the server.
The current production version source code is accessible by https://www.mersenne.org/download/, and numerous other versions' source code via https://download.mersenne.ca/gimps and https://download.mersenne.ca/gimps_a...rchived_source
Reading source code is one of many ways to learn about the field. Prime95 is a very ambitious program, supporting numerous CPU instruction sets, and computation types, for multiple number types, in a single monolithic build per target OS. Its source code is massive, at over 600 files in over 25 folders totaling 335 MiB for v30.8b15. The beginning number theory programmer might be better served by starting source examination with a different program that tackles less of a cornucopia of cases and has orders of magnitude less code size & complexity.
See Mlucas & Mfactor (21 MB), mfakto (3.5 MB), mfaktc (2.3 MB), CUDALucas (700 KB), gpuowl (700 KB), clLucas (400 KB), CUDAPm1 (260 KB), gpulucas (93 KB), pzilla (38 KB), Factor5 (22KB), lucas (20KB).
Build processes for the prime95 family of programs would necessarily depend on the target OS type and are not documented anywhere to my knowledge, other than in the prime95 source code package itself, e.g., for the winnt service version, see winnt\compile64 or compile, for the Windows GUI version see prime95\compile64 or compile, etc.

Setup:
Setup instructions are included at https://www.mersenne.org/download/. Follow with https://www.mersenne.org/gettingstarted/
One thing to avoid is installing into "Program Files" or other restricted directories. Permissions problems will follow with such errant installs.

Making a separate working directory for prime95 under the user's home directory is the way to go.

I strongly recommend benchmarking over the range of fft lengths expected to be used, analyzing the results in a spreadsheet, and configuring for best throughput that is consistent with latencies shorter than applicable expiration periods. Configure worker windows for your preferred work type, and make sure that trial factoring is not it; GPUs are far more effective at that.

It is normal for the PrimeNet server to issue a new prime95 installation only LL DC, until each prime95/mprime worker has completed 4 LL DC successfully. After a new installation accumulates a history of reliability, the PrimeNet server will allow additional work types.

Do not do TF in mprime/prime95 if it can possibly be avoided. Modern GPUs are much more effective at it.
If performing P-1, P+1, or ECM factoring, check the setting for allowed ram use in stage 2 and give it as much as practical. Default settings will result in only running stage 1 in most cases, which is inefficient and less effective. Allowing too much ram will result in excessive paging slowing both prime95 and other applications.

Note that with AVX512 hardware, mprime/prime95 can process exponents up to 1169M, somewhat exceeding the mersenne.org server's maximum 1000M. If PrimeNet API connected, it will try and fail to report such high exponents that are not tracked in the mersenne.org database. Mersenne.ca has been enhanced to accept such P-1 results in prime95 JSON format here. Such large exponents should be avoided on most hardware and by most users, since there is little need for P-1 on them currently, as primality testing would take too long, and odds per unit of compute time of finding a Mersenne prime at >1G exponent are between 2 to 3 orders of magnitude lower than at ~1/10 that exponent size. Odds are best at the first-test wavefront.

Concepts:
(including portions originally posted as part of https://mersenneforum.org/showpost.p...05&postcount=5)
Mprime/prime95 is a mature, richly featured, and complex application supporting full use of many cores of a multicore system from a single running instance. This is unlike most GIMPS applications. Other applications may require multiple instances to fully utilize the CPU cores of a system, or one instance or more per GPU.

Mprime/prime95 is often the first GIMPS application a new user will learn to use. Be careful not to project to other applications, expectations they will act just like prime95/mprime.

These things have different names because they are different concepts:

• running instance (process)
• worker
• thread
• core
• assignment
• folder (directory)
• configuration
  • default
  • optimal

One mprime instance can have (and often does have) multiple workers running simultaneously in one process, each using multiple cores via multiple threads, and having separate queues of multiple assignments, with all work in progress for the various assignments saved in the same prime95/mprime folder. Typically prime95 will install by default with 4 cores per worker, and as many workers as it would take to fully utilize all physical cores on a system.
(For example, 4 core HT i5-1035G1, 1 worker; dual-8-core x2HT Xeon, 4 workers; 68-core x4HT Xeon Phi 7250, 17 workers.)
It's quite easy to accidentally launch multiple mprime instances in the same folder on the same assignments, but that wastes time and may make a mess. (Detached processes in Linux make this particularly easy.) When in doubt, check "top" process listing.
Unless a system is known to have NUMA, one instance is generally enough, even for dozens of cores.
If you find you have multiple mprime processes, I suggest you stop all the mprime processes, restart one, and see what you get for system loading and top listing. Then adjust its configuration either by using the GUI for prime95, the menus for mprime, or while all are stopped or exited, editing directly (and carefully!) its text based configuration files, prime.txt and local.txt (which will in v30.10 become a single file).
If for better performance, running multiple instances on a NUMA system such as multiple-Xeon or Xeon Phi, use separate install folders and configuration files for the separate instances.

For remaining questions see the program's extensive included documentation. Reviewing the software forum thread corresponding to the particular version may also be helpful. For example, for v30.8, https://mersenneforum.org/showthread.php?t=27366

Additional posts in this reference thread:
PRP run time scaling for low p https://www.mersenneforum.org/showpo...78&postcount=2
P-1 run time scaling https://www.mersenneforum.org/showpo...92&postcount=3
Effect of number of workers https://www.mersenneforum.org/showpo...18&postcount=4
Effect of number of workers (continued) https://www.mersenneforum.org/showpo...19&postcount=5
Effect of frequent interim residue output https://www.mersenneforum.org/showpo...44&postcount=6
Prime95 documentation https://www.mersenneforum.org/showpo...03&postcount=7
Prime95 exponent limits https://www.mersenneforum.org/showpo...74&postcount=8
PRP proof capable versions https://www.mersenneforum.org/showpo...35&postcount=9
Performing version upgrades https://www.mersenneforum.org/showpo...2&postcount=10
Effect of number of workers continued 2 https://www.mersenneforum.org/showpo...4&postcount=11
Use as hardware reliability test https://www.mersenneforum.org/showpo...7&postcount=12
Memory use during PRP https://www.mersenneforum.org/showpo...2&postcount=13
Interpreting the error-counts 32-bit word https://www.mersenneforum.org/showpo...0&postcount=14
P-1 performance https://www.mersenneforum.org/showpo...0&postcount=15


See also the Concepts in GIMPS Trial Factoring post at https://www.mersenneforum.org/showpo...23&postcount=6


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Run time is fitted as approximately proportional to p^2.094, for 86243 <= p <= 2976221. LL run time is expected to scale very similarly. For comparison a theoretical fft convolution based primality tester scales as p² log p log log p, which over the mersenne.org interval fits as p^2.117. Overhead at low exponents lowers the power on a fit.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

A small number of widely spaced exponents were run to observe the run time scaling.

For prime95 v29.4b8 x64 run on a Windows 7 x64 system with dual e5-2670 chips, 4 cores (half a chip package) per worker, 32,000 MB allowance per worker, run time was approximately proportional to exponent p^2.33 up to 595M (27 days), a somewhat higher power than observed for P-1 on gpus (~2.1).

Another prime95 v29.4b8 x64 run on an FMA equipped i7-7500U Windows 10 X64 system seemed to be taking inordinately long to perform P-1, at p=101M, on 7,200 MB memory allowed, one core. It had been running for two weeks to perform stage 1 and reach 90% in stage 2. It appeared to be paging to disk excessively. The same system can complete an 83M primality test per core in about 2.5 weeks. It was allowed to complete that P-1 and then reset to 4096M memory allowed, after it was found to still page excessively at 6144M. This is a system with 8GB ram currently. In all cases it was running 1 core per worker; the other worker was running an 83M LL. It projected P-1 run times ranging from 4.4 days for 201M to 43 days for 605M, 67 days for 701M. However, attempting 605M resulted in "Cannot initialize FFT code, errcode=1002".
The fit to observed run time is p^2.087 (with five data points).

Another run, a mix of prime95 V29.7b1, v29.8b3, and v29.8b6, on an FMA equipped i7-8750H Windows 10 X64 system was able to run 801M (at 8GB allocated of its 16GB installed ram, 37 days run time), and 901M (at 12GB allocated, 57 days run time) and is expected to be capable of up to 920.8M. The offset in the estimated days runtime is believed to be due to whether mfakto is running on the Intel igp or not. It seems to be using somewhat lower bounds than GPU72 figures for exponents above p~400M.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Similar to the number of threads choices in gpu applications, on multicore systems, the effect of number of cores per worker in prime95 is unpredictable, and so there is provision for benchmarking.

Number of workers could be chosen to optimize performance. But which measure of performance? Aggregate throughput maximized, latency of one assignment minimized, number of joules used for a 100GhzD primality test, aggregate throughput given a constraint of latency low enough to avoid assignment expiration, something else? For which single fft length, or for the current and next several?

For minimum latency, as for confirming a newly discovered Mersenne prime, Madpoo has run experiments on a dual-14-core system. He reported the fastest primality test time around 20 cores out of the 28 available; any more than 6 on the lesser use package, and the increased package to package data transfers slow the progress.

For picking number of cores/worker per cpu type, that's a reasonable compromise for maximum aggregate throughput, so I can set it and forget it for months or years on each system, I ran the built in prime95 benchmarking over wide fft ranges for a variety of cores/worker, on a variety of cpu types. Then the timings were tabulated in spreadsheets and graphed.

If going after the maximum performance per fft length, consider that some work types restart from the beginning when the number of workers is changed. Read the readme.txt and other files, back up before changing number of workers, plan ahead, etc.

Some patterns emerge. Worker counts that would straddle the divide between processor packages if divided evenly typically do not provide as much throughput. A 12-core 2-package system with 3 workers with equal cores/worker would have at least one worker with cores in each package (4 2 + 2 4). George indicates recent versions of prime95 prevent the straddle by assigning unequal numbers of cores to the workers.
For larger core counts there can be quite a few choices to evaluate. What's fastest for one fft length may not be for others. A compromise that averages a small percentage penalty is usually available. Plotting the various combinations with trend lines seems a useful visualization method for selecting one configuration to run with for a long time.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Working around the 5-attachment limit per post, see below, and https://www.mersenneforum.org/showpo...4&postcount=11.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Timing runs on LL DC on the same 51M exponent and old 32-bit hardware with prime95 29.4b7 yield conflicting information on the cost of a Res64 output as a multiple of an ordinary iteration. The res64 cost is estimated as 7/8 to 4 times an iteration. Note that because of numbering skew between prime95 and other conventions, prime95 outputs res64 at 3 successive iterations, with cost ~3.1 to 12 times an iteration. The lower value is based on prime95-provided timings per iteration, the higher value on prime95-provided time stamp of 1 second resolution of the res64 output line.

An initial attempt to make a similar measurement on an i7-8750H with UHD630 igp in prime95 v29.4b8 x64 yielded negative per-res64 cost in two tries. I speculate this was an interaction with mfakto running at the same time on the same chip package power budget. Performance monitor indicates the cpu utilization drops considerably when frequent interim residue output is enabled.

A retest, with the UHD630 mfakto instance halted, yielded timings that indicate a cost per PRP3 res64 interim output on the i7-8750H system of 2.7 seconds, equivalent to 263. iterations, on an 83M primality test. One of the 6 cores stays very busy while the rest are only used at a low duty cycle when outputting an interim residue every 10 iterations. This cut throughput from 96.6 iter/sec to 3.54 iter/sec, a rather severe 96.3% reduction. The estimated effect on run time for the exponent when producing interim residues for the primenet server at 5,000,000 iteration intervals is about 45 seconds, 52ppm of run time. The retest was brief, taking 48 seconds for iterations with interim residues, and 114 seconds without, so accuracy is no better than a percent or two. Note also the cpu clock was not held constant during the test. In this case the agreement between time stamp based rates and program-computed ms/iter was very good, ~1/4%.

Another test, on a dual-xeon-e5-2690 system, v29.6b6 x64 on Win10, 4 cores/worker, 83.9M PRP tests, gave ~305 iterations/interim residue64, 3.45 sec/interim residue, or around 61ppm for the default 5,000,000 iteration interval. The preceding figure ignores the initial 500K-iteration interim residue, which raises the impact a bit to 65ppm for ~84M exponents, and somewhat more for DC exponents.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Most GIMPS applications include a readme file. Prime95 has very comprehensive documentation included in the zip package, in multiple files.

• License.txt for the license terms
• Readme.txt for the new user and periodic reference
• Whatsnew.txt particularly useful when upgrading
• Stress.txt relating to stress testing and reliability testing
• Undoc.txt documentation of the perhaps less frequently used options

Read them early and often. Like for the other applications, reading the documentation again after additional experience with the program is useful.
Also, for those who would like a deeper understanding, there is the source code and its gwnum folder's tutorial.txt.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Prime95 and its sibling mprime contain many code paths specific to processor types and exponent magnitudes. What range of exponents is supported varies by processor type. I think what has been implemented was determined by a combination of processor throughput versus exponent size and decisions by George on which to spend his programming time.

There are several ways to determine what these limits are.
George has made statements about them in email or on the forum.
https://mersenneforum.org/showpost.p...&postcount=219

The whatsnew.txt describes numerous changes in what was supported.

The source code is available for examination.

Trying runs on differing hardware and OS may obscure the situation, because it could be that it's an old operating system version, not the processor type, that prevents running some versions of code.

For a given CPU instruction set, refinements from one version prime95/mprime to another will sometimes raise the exponent limit per fft length and overall. So the attachments may be viewed as rough guidelines.

Note, while benchmarking goes on AVX512 go all the way up to 64M fft, ~1169M exponent, up to v30.7b8 or 9 the LL limits for AVX512 are about the same as for AVX2, ~922M. Also, P-1 and PRP appear in testing prime95 v30.7b9 to be limited to prime exponents below 2³⁰, 1,073,741,789 max, which makes the 64M fft length unnecessary and limits use of the 60M fft length also. This was observed on an AVX512 i5-1035G1 system. V30.8b14 or perhaps earlier added support up to 1169M for P-1, LL, and PRP.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

UPDATE:
As of ~2022-09-28, latest available promoted to general use is v30.8b17. Get it here or here.

Some earlier versions did not include the proof file upload capability. PRP proof generation was introduced in v30.1; automatic uploading of proofs in v30.2.
The standalone command-line uploader, which works for gpuowl as well as prime95, is described briefly at https://www.mersenneforum.org/showpo...&postcount=154
but the direct download from dropbox for Windows x64 is no longer available. It can be found as an attachment at https://www.mersenneforum.org/showpo...0&postcount=26
NOTE: it is not being maintained, and preferred usage for prime95/mprime output is upload through a current version of prime95 or mprime.

Usage is

uploader user_id proof_filename[ chunk_size[ upload_rate_limit]]

with chunk_size expressed in MB and upload_rate_limit expressed in Mbps apparently.

(Note, for gpuowl, there are more choices; https://www.mersenneforum.org/showpo...0&postcount=26, some of which might conceivably apply to prime95/mprime too, at least for the most adventurous. But I encourage users to stick with prime95 & mprime's built in PrimeNet API & supported features whenever practical.)


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

The most efficient method will depend on whether it's a single install or a fleet of them to be upgraded. Each leaves your results files, worktodo, log files, work in progress files, and configuration files in place and undisturbed. (But you should be doing regular system backups anyway.)

Single install:
Stop and exit the prime95 program to allow prime95 program files to be overwritten.
Download the zip file. Unzip it.
If necessary, move the new files into your working directory. Select replace if prompted. Restart the program in the working directory.

Multiple systems, with USB drive:
Download the zip file. Put it onto the USB drive. Unzip it there.
On each system:
Insert the USB stick.
Stop and exit the prime95 program to allow prime95 program files to be overwritten.
Copy the new version's files from the USB stick to the working directory, overwriting the old.
Start the program in the working directory.
"Eject" the USB drive. Its file explorer window will close.
Remove the USB stick.

Multiple systems, with network drive:
Download the zip file. Put it onto the network drive. Unzip it there.
On each system:
In file explorer, navigate to the update version prime95 folder on the network drive.
Stop and exit the prime95 program to allow prime95 program files to be overwritten.
Copy the new version's files from the network folder to the working directory, overwriting the old.
Start the program in the working directory.
Close the file explorer window for the update version folder.

It's possible to streamline the above somewhat with a bit of batch script.

Strictly speaking, it is not necessary to copy and overwrite files that have not changed from the previous version, but it does little harm.
Unneeded copying can be efficiently avoided by date sorting both source and destination folders, and only copying what's newer than the corresponding destination file.

For more detail, quoted with some editing, from S485122 at https://mersenneforum.org/showpost.p...61&postcount=4
prime.txt contains the GIMPS user data,
local.txt contains the machine data,
worktodo.txt contains the current work (assigned or not),
at some times a file named prime.spl which contains the results not yet transmitted to the server might be present,
the work files pnnnnnnn mnnnnnnnn etc and their backup copies .bu, bu2, etc...
None of these files are in the prime95.zip archive and will thus not be overwritten. They are essential for continuity.
There are other user files that are not in the archive either, but they are less critical (results.txt, results.json.txt, prime.log, gwnum.txt, ...)
In other words, keep all other files in the folder, since they contain your user and machine data and preferences, your work in progress and results. The only files overwritten will be the program and version dependent files.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

[kriesel]

Additional processor types:
FMA3 capable 6-core i7-8750H (no code running on the IGP at the time)
Xeon Phi 7250 (68 cores in one socket) see also https://www.mersenneforum.org/showthread.php?t=25767


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1