PFGW v3.1.0/Prime95 v25.11

[mdettweiler]

Not long ago, version 25.11 of the gwnum math library, utilized by LLR, PRP, PFGW, and Prime95, was released. This new release includes optimizations for non-base-2 numbers that will significantly benefit this project!

PFGW v3.1.0 and Prime95 v25.11, which both utilize the new math library, can now do PRP tests significantly faster than either LLR or Phrot. LLR will probably be updated sometime in the near future, though in the meantime, Prime95 and PFGW (which both produce residuals compatible with LLR and Phrot's) are the way to go for testing non-power-of-2 numbers.

Note that version 3.1.0 of PFGW has a bug in which the first character of the residue is produced incorrectly. This is because PFGW produces 62-bit residues, not 64-bit like LLR, Phrot, and Prime95. Yet, for compatibility with those programs, PFGW outputs a 16-character (64-bit) residue nonetheless. This results in the first character being incorrect, which can cause problems when the residues are compared with true 64-bit ones.

PRPnet's built-in doublechecking functionality is aware of this bug, and ignores the first character when comparing residues. However, since we don't use PRPnet's built-in doublechecking functionality here at CRUS (instead doing second-pass tests separately from first-pass tests when they are needed), this won't help us.

Mark Rodenkirch (rogue), who is leading the PFGW development effort, said that the next release of PFGW will have true 64-bit residues. Until that is released, though, we should try to avoid using PFGW for testing on CRUS, since it may impact possible future doublechecking efforts. If you must use it, please be sure to note this when posting your results (just like you would if, say, you forgot to run Phrot with the -b=3 option enabled to produce LLR-compatible residues).

In the meantime, though, Prime95 (mprime on Linux) handles PRP tests quite nicely and with the same speed, yet without the residue bug. I've been testing it recently on some Riesel base 94 numbers, and the speed increase is enormous--approximately 3 times faster for most bases! Thus, I highly recommend that CRUS members start using it for non-power-of-2 PRP tests (at least until LLR is also updated to accomodate the new math library).

Prime95 v25.11 can be downloaded from this thread:
http://www.mersenneforum.org/showthread.php?t=12155
PFGW 3.1.0 can be downloaded from this thread:
http://www.mersenneforum.org/showthread.php?t=12134

One disadvantage of Prime95, though (at least for the purposes of this project), is that it doesn't accept NewPGen- or ABC-formatted input files. Sieved candidates must be formatted in Prime95's proprietary worktodo.txt format. This cannot be done directly with srfile, though srfile can produce worktodo.txt format entries for performing P-1 testing through Prime95. The P-1 lines can be modified somewhat easily with a text editor to be used for PRP testing as follows:

-Run srfile on a sieve file of any srfile-supported format, with the following command:
srfile -p 1 sieve-file.txt >> worktodo.txt
srfile will output a file worktodo.txt with entries like this (example from the Riesel base 3 mini-drive):
Pfactor=17589196,3,78001,-1,37,1
All of the lines in the file should have the ",37,1" section at the end. (Note: the 37 may be something different depending on the sieve depth, since it represents the approximate sieve depth in power of 2. The above example is for 2^37. Use the number that you see in the file for the rest of these steps.)
Now, use a text editor (such as Notepad or Wordpad) to search/replace for the string "Pfactor" and replace it with "PRP". After you've done that, do the same for the string ",37,1" and replace it with "" (i.e. a blank string). Now your lines should look like this:
PRP=17589196,3,78001,-1
Now your worktodo.txt file is all set to be run through Prime95.

To set up and use Prime95, do the following:
-Download the appropriate version of Prime95 (32-bit or 64-bit) from the thread linked above.
-Extract the downloaded .zip file to a new folder.
-Run Prime95.exe.
-It will ask you whether you want to join GIMPS, or if you want to use Prime95 just for stress testing. Choose "just stress testing"; that will tell Prime95 not to talk to the PrimeNet server or ask you to enter a GIMPS user ID, which is what you want for CRUS PRP testing.
-It will then ask you to input stress test parameters. Click Cancel.
-Prime95 is a lot like LLR, except a bit more complex: it has various additional features, such as multi-threading. (LLR is based on an earlier version of Prime95, v24.14, which is even closer to LLR in appearance and behavior.)
-In Options>Preferences, you can adjust many of the same options as you can with LLR. The only ones you might want to adjust are "iterations between screen output" and "minutes between writing save files". Those options are just the same as they are in LLR.
-Once you're satisfied with everything you've set up, make sure worktodo.txt is in place in your Prime95 folder with the work you want to run.
-Now click Test>Continue to begin work. (Just like with LLR.) Prime95 will output your test status on the second subwindow ("Worker #1"). (The top subwindow is mainly for PrimeNet communication info, which is not applicable for non-GIMPS uses of Prime95.)
-Results will be outputted to results.txt.

Note: since the copy of Prime95 you just set up is configured for non-GIMPS manual testing, you may want to merge all of the subwindows into one, since the upper one is primarily for PrimeNet communications info, which does not apply to this use of the program. To merge all the subwindows into one, click Window, and choose "Merge Main and Comm and Worker". You may also want to maximize the one remaining subwindow so it fills the entire Prime95 window.

If you've got any further questions on how to use Prime95, just post here and I'll answer as soon as I get the chance.

Max

[Mini-Geek]

Originally Posted by mdettweiler

Pfactor=17589196,3,78001,-1,37,1
All of the lines in the file should have the ",37,1" section at the end. (Note: the 37 may be something different depending on the sieve depth, since it represents the approximate sieve depth in power of 2. The above example is for 2^37. Use the number that you see in the file for the rest of these steps.)
Now, use a text editor (such as Notepad or Wordpad) to search/replace for the string "Pfactor" and replace it with "PRP". After you've done that, do the same for the string ",37,1" and replace it with "" (i.e. a blank string). Now your lines should look like this:
PRP=17589196,3,78001,-1

Do you assume to save two tests (first- and second-pass) if a factor could be found? If you replace the ",37,1" with ",37,2", (the format is bits_already_factored,tests_saved_if_factor_is_found) then Prime95 will calculate if running some P-1 first on each candidate would save time in the long run. (for your example it chose not to run P-1, so in most cases it probably won't run P-1, but might as well let it check, right? Since it does it on a per-exponent basis, it can account for differences more easily than the large sieves where the candidates were first sieved)

Originally Posted by mdettweiler

(LLR is based on an earlier version of Prime95, v25.14, which is even closer to LLR in appearance and behavior.)

24.14

Originally Posted by mdettweiler

-Prime95 is a lot like LLR, except a bit more complex: it has various additional features, such as multi-threading. (LLR is based on an earlier version of Prime95, v25.14, which is even closer to LLR in appearance and behavior.) The multi-threading mode is disabled in "stress testing" mode; you can still use it for PRP tests, though you'll need to do a bit more tweaking. I won't go into that here; if anyone is interested, post here and I'll provide instructions.

Something that could be useful for some people is using separate worker threads that each use one core. Under Test > Worker Windows, set the top box to how many different workers you want run, then put "[Worker #1]" and so on at even spacings in the worktodo.txt file.

Originally Posted by mdettweiler

(The top subwindow is mainly for PrimeNet communication info, which is not applicable for non-GIMPS uses of Prime95.)

If you check the second item under the Window menu, the Main and Comm windows will merge with the first worker, so you don't see them.

[mdettweiler]

Originally Posted by Mini-Geek

Do you assume to save two tests (first- and second-pass) if a factor could be found? If you replace the ",37,1" with ",37,2", (the format is bits_already_factored,tests_saved_if_factor_is_found) then Prime95 will calculate if running some P-1 first on each candidate would save time in the long run. (for your example it chose not to run P-1, so in most cases it probably won't run P-1, but might as well let it check, right? Since it does it on a per-exponent basis, it can account for differences more easily than the large sieves where the candidates were first sieved)

Actually, in my instructions above the number of tests saved is arbitrary, since it's just going to be removed later when you use a text editor to convert the Pfactor= lines to PRP= lines. I guess it wouldn't hurt to leave a ",37,2" on to do P-1 factoring before testing; though, in many cases, we've sieved well enough anyway that P-1 is not worth the extra time it takes.

24.14

Ah, good catch. Thanks, I'll fix it.

Something that could be useful for some people is using separate worker threads that each use one core. Under Test > Worker Windows, set the top box to how many different workers you want run, then put "[Worker #1]" and so on at even spacings in the worktodo.txt file.

Yes, that could be handy; however, unfortunately the multiple worker threads option is not available in "stress test mode". A wee bit of tweaking in prime.txt is needed:
StressTester=0
UsePrimenet=0
UsePrimenet=0 should already be set if you originally set up Prime95 in stress testing mode. What this combination of options does is set Prime95 to "GIMPS mode" (so that multithreading is available), but it won't grab work from the PrimeNet server so the effect is essentially the same as stress testing mode. This is how my client is set up. I didn't use this configuration in my instructions, though, since some people might find the whole idea of editing the prime.txt file confusing.

If you check the second item under the Window menu, the Main and Comm windows will merge with the first worker, so you don't see them.

Ah, that works too. I was aware of that option, but I hadn't thought of using it for my non-GIMPS copy of Prime95...thanks, I'll do that.

[Mini-Geek]

Originally Posted by mdettweiler

Actually, in my instructions above the number of tests saved is arbitrary, since it's just going to be removed later when you use a text editor to convert the Pfactor= lines to PRP= lines. I guess it wouldn't hurt to leave a ",37,2" on to do P-1 factoring before testing; though, in many cases, we've sieved well enough anyway that P-1 is not worth the extra time it takes.

Yeah, I figured that, but it wouldn't hurt to leave it in with the correct values instead of just taking it out. Then again, depending on how the bit level is rounded and how Prime95 calculates saved time, and especially the time taken for P-1 on smaller numbers, it just might hurt! I really don't know, but I'd guess that generally it won't hurt.

Originally Posted by mdettweiler

Yes, that could be handy; however, unfortunately the multiple worker threads option is not available in "stress test mode". A wee bit of tweaking in prime.txt is needed:
StressTester=0
UsePrimenet=0
UsePrimenet=0 should already be set if you originally set up Prime95 in stress testing mode. What this combination of options does is set Prime95 to "GIMPS mode" (so that multithreading is available), but it won't grab work from the PrimeNet server so the effect is essentially the same as stress testing mode. This is how my client is set up. I didn't use this configuration in my instructions, though, since some people might find the whole idea of editing the prime.txt file confusing.

Tweaking is not necessary, merely pressing Test > Continue (or restarting Prime95) once after you've picked "Just Stress Testing" and have a proper worktodo.txt (including the right number of sections).
In more words: If you set up the worktodo.txt file appropriately before running Prime95, then run it, choose Just Stress Testing, click Cancel, and press Test > Continue... then it will set
StressTester=1
UsePrimenet=0
and your tests will automatically resume when you start Prime95.
Maybe it's different in mprime, which you're probably using since you use Linux.

Originally Posted by mdettweiler

Ah, that works too. I was aware of that option, but I hadn't thought of using it for my non-GIMPS copy of Prime95...thanks, I'll do that.

Ok, do you think it could fit well in the first post so others can see that too?

[mdettweiler]

Originally Posted by Mini-Geek

Yeah, I figured that, but it wouldn't hurt to leave it in with the correct values instead of just taking it out. Then again, depending on how the bit level is rounded and how Prime95 calculates saved time, and especially the time taken for P-1 on smaller numbers, it just might hurt! I really don't know, but I'd guess that generally it won't hurt.

Tweaking is not necessary, merely pressing Test > Continue (or restarting Prime95) once after you've picked "Just Stress Testing" and have a proper worktodo.txt (including the right number of sections).
In more words: If you set up the worktodo.txt file appropriately before running Prime95, then run it, choose Just Stress Testing, click Cancel, and press Test > Continue... then it will set
StressTester=1
UsePrimenet=0
and your tests will automatically resume when you start Prime95.
Maybe it's different in mprime, which you're probably using since you use Linux.

Actually, I've been using Windows as my primary OS of late (it's a dual-boot system), so yes, I am dealing with the same program. I realize that no tweaking is necessary to simply get Prime95 testing, but from what I can tell, the # of threads section of the "Worker Windows" screen is grayed out while in stress testing mode.

Ok, do you think it could fit well in the first post so others can see that too?

All right, I'll do that.

[Mini-Geek]

Originally Posted by mdettweiler

Actually, I've been using Windows as my primary OS of late (it's a dual-boot system), so yes, I am dealing with the same program. I realize that no tweaking is necessary to simply get Prime95 testing, but from what I can tell, the # of threads section of the "Worker Windows" screen is grayed out while in stress testing mode.

It's not grayed out for me. The only things that are grayed out are things that ought to be, like the box to choose the type of work to get from PrimeNet, or the "threads for this worker" option on my dual core CPU with the worker windows set to 2 already. Perhaps this is what you're seeing. Set the number of worker windows to 1 and the option for how many threads that worker should use will un-gray.
To see it, run Prime95 from an empty folder. It will open like it's the first time, and you can see for yourself.

[mdettweiler]

Originally Posted by Mini-Geek

It's not grayed out for me. The only things that are grayed out are things that ought to be, like the box to choose the type of work to get from PrimeNet, or the "threads for this worker" option on my dual core CPU with the worker windows set to 2 already. Perhaps this is what you're seeing. Set the number of worker windows to 1 and the option for how many threads that worker should use will un-gray.
To see it, run Prime95 from an empty folder. It will open like it's the first time, and you can see for yourself.

Ah, I see it now. I'd done the same thing before (ran Prime95 from an empty folder) when I was writing the instructions, but I'd forgotten that Prime95 is "smart" enough to not let you use more threads than you have cores, and that that was the cause of the graying-out.

I'll remove the tidbit from the first post about multithreading not being available in stress testing mode, since that was most definitely proven wrong.

[Cruelty]

Just a quick benchmark using LLR:

4421*10^228022-1 is not prime.  RES64: 56DAA17A59780CCB.  OLD64: 048FE46F0C68265F  Time : 3631.383 sec.

The same test using Prime95 takes ~15 minutes 75% speed improvement? How is that possible???
BTW: the only thing I miss in Prime95 is the actual time the test took to complete.

[mdettweiler]

Originally Posted by Cruelty

Just a quick benchmark using LLR:

4421*10^228022-1 is not prime.  RES64: 56DAA17A59780CCB.  OLD64: 048FE46F0C68265F  Time : 3631.383 sec.

The same test using Prime95 takes ~15 minutes 75% speed improvement? How is that possible???
BTW: the only thing I miss in Prime95 is the actual time the test took to complete.

Yes, that's consistent with the results everyone else is getting. Agreed, other than outputting the total time it took to run the test, Prime95 is pretty complete. (I guess George must have figured a "total time" figure would be largely useless in a program that tests 10 million digit numbers as its primary task. Most of those are essentially guaranteed to be stopped and restarted at some point during the run.)

PFGW does output its test times, so even if its residues are slightly off, it's still quite useful for getting benchmark data since its times should be the same as Prime95's.

[MrOzzy]

This is indeed a very nice improvement and makes proving some of these conjectures a bit more realistic. It also means I probly have sieved some of my work quite a bit. Some questions about prime95 though:

Should I worry about this bug:

Some non-base-2 numbers get roundoff errors because of poor FFT length selection. Examples include: 211*210^2047-1, 211*210^2687-1, 211*210^7679-1. Fixed in 25.12.

Is there a way to assign one big batch of work to all your workers without doing some copy-paste work and estimate the time needed for each batch?

I also saw prime95 sometimes uses an all-complex FFT length for some n and uses a zero padded FFT length for some other n pretty randomly. Should I worry about that?

The result file seems to be pretty messy (all workers results are put in the same file, time codes and check in between and so on). Is there an easy way to clean this file and sort all pairs by n?

[Prime95]

Originally Posted by MrOzzy

Should I worry about this bug:

Some non-base-2 numbers get roundoff errors because of poor FFT length selection. Examples include: 211*210^2047-1, 211*210^2687-1, 211*210^7679-1. Fixed in 25.12.

No. It will be rare and you'll get an error message letting you know the number was not tested properly.

Is there a way to assign one big batch of work to all your workers without doing some copy-paste work and estimate the time needed for each batch?

No.

I also saw prime95 sometimes uses an all-complex FFT length for some n and uses a zero padded FFT length for some other n pretty randomly. Should I worry about that?

No.

The result file seems to be pretty messy (all workers results are put in the same file, time codes and check in between and so on). Is there an easy way to clean this file and sort all pairs by n?

No.