Double-checking with PFGW
 

I have been doing some testing with PFGW 3.3.6 and have discovered that a number of candidates need to be retested due to the carry propagation issue found in gwnum that was fixed in PFGW 3.3.6. It is possible that some primes have been missed. I'm opening this topic to discuss how we want to proceed.

For all of the bases I have reserved I have run the remaining k up to n = 1000 (my original script limit) to see if I missed any primes. Fortunately I did not. I then ran all tested candidates for n > 1000 against PFGW 3.3.5 and 3.3.6 using the -F switch. Around 8% of those candidates had a different FFT size. I updated the PRPNet database to force all of them to be retested. That should finish in the morning. At this time about 70% of those tests have completed. All residues match between 3.3.5 and 3.3.6, which means that none of those 70% were tripped up by the carry propagation issue.

Based upon testing I did with [URL="http://www.mersenneforum.org/showthread.php?t=13797"]k*b^b+/-1 numbers[/URL], I did see numbers with invalid residues. IIRC it was about 3% of what remained after sieving, which is significant, especially since a number of primes were missed. The problem was more apparent as k increased. I didn't see any problems until k > 1000, but that was a small sample set. I am running an extended test overnight (2 < k < 2002, 3 < b < 1003, 402 < n < 10402) which will encompass over 1,000,000 tests. I should have some statistical results by the morning. In other words, I should be able to state, with a certain amount of confidence, the ranges of k, b, and n that should be retested.

I will state now that anyone using a pre-3.3.6 release should upgrade before continuing with any testing. According to George, base 2 should not be affected, but the higher the base, the more likely it is affected by the issue. Note that 3.3.6 will chose a larger FFT size for some numbers even if that is not required to complete a successful test. George is working on a permanent fix, but until I have one, 3.3.6 is the way to go for this project.

This sounds very problematic. I'm open to any advice on how to proceed. I and I'm sure many others have done a large amount of work with PFGW 3.3.4, which I thought to be an excellent virtually bugfree version.

A couple of questions:

What is the "carry propagation" issue that you are referring to? If it's real technical, you don't need to provide a lengthy explanation. Just a quick synopsis will do.

Why test only k*b^b+/-1 ? What is significant about the base and exponent being the same? Wouldn't the bug exist across n-values that are not equal to the base?

All of us believed that gwnum 25.14 was bug-free. George was as surprised as anyone else that this was discovered.

George describes the issue [URL="http://www.mersenneforum.org/showpost.php?p=228471&postcount=65"]here[/URL] far better than I could.

The issue was discovered by kar_bon and 3.14159 when testing numbers of that form. b = n has no significance. It is just a coincidence.

[quote]
PFGW Version 3.3.5.20101007.Win_Stable [GWNUM 25.14]

Special modular reduction using all-complex FFT length 768 on 9276*626^627+1
9276*626^627+1 is composite: RES64: [3E2E5E4CAE04201F] (0.0701s+0.0001s)
[/quote]

[quote]
PFGW Version 3.3.6.20101007.Win_Stable [GWNUM 25.14]

Special modular reduction using zero-padded FFT length 896 on 9276*626^627+1
9276*626^627+1 is composite: RES64: [DD2C7BD5BCC04FAE] (0.0864s+0.0001s)
[/quote]

Gary: more info is in [URL="http://www.mersenneforum.org/showthread.php?t=12214&page=3"]this thread[/URL] especially post #65. k*b^b+/-1 searching was how the errors were initially noticed, but the problem apparently is not restricted to n=b.

Rogue: This is not very clear on the software section thread. Are N+1/N-1 tests as done by LLR 3.8.0 or 3.8.1 affected by the same issue?

I know you're not responsible for LLR but since this is a gwnum bug (used by both programs) and your technical understanding is far superior than mine any guess on LLR? Or any test cases to actually compare?

I've run a quick PFGW 3.3.5 vs 3.3.6 -F switch on about 25% of my total CRUS work (~80,000 tests, n>25K, several k's >1000). No FFT size mismatches. Maybe the problem is more prominent on smaller n and FFT sizes.

[QUOTE=gd_barnes;229269]Why test only k*b^b+/-1 ?[/QUOTE]

That's just where the bug happened to be discovered. The problem shows up more readily with large bases, so this was a natural place to find it.

I hope statistical information is collected in terms of number of forms that need to be retested at a given base, and number of those that are wrong.

[QUOTE=vmod;229280]This is not very clear on the software section thread. Are N+1/N-1 tests as done by LLR 3.8.0 or 3.8.1 affected by the same issue?

I know you're not responsible for LLR but since this is a gwnum bug (used by both programs) and your technical understanding is far superior than mine any guess on LLR? Or any test cases to actually compare?

I've run a quick PFGW 3.3.5 vs 3.3.6 -F switch on about 25% of my total CRUS work (~80,000 tests, n>25K, several k's >1000). No FFT size mismatches. Maybe the problem is more prominent on smaller n and FFT sizes.[/QUOTE]

LLR 3.8.x is also affected. I don't know if Jean is aware of the issue as he will need to re-release LLR.

According to George, smaller FFTs are affected.

Here are some statistics on what I ran overnight:

- about 6.7% of the numbers tested used a different FFT size
- the smallest k affected was 22 for 22*283^602+1
- the smallest b affected was 43 for 662*43^9802+1. For higher k, n as low as 1202 was affected for base 43
- the smallest n affected was 402 for 42*263^402+1
- I did not re-run any PRP tests to determine if residue did not match
- my other overnight test showed no residue differences for the bases I currently have reserved

Note that I skipped every 10 k, every 10 b, and every 100 n so these are ballpark ranges.

[QUOTE=gd_barnes;229269]This sounds very problematic. I'm open to any advice on how to proceed. I and I'm sure many others have done a large amount of work with PFGW 3.3.4, which I thought to be an excellent virtually bugfree version.[/QUOTE]

I'm in that boat with you. I have spent a disproportionate amount of time on CRUS.

I certainly don't want to restart this project from scratch. Unless anyone thinks that it is imperative that we capture the lowest n for each k, I would avoid retesting any k for which a prime has been found. In other words, only the k without a prime should be retested.

This is what I suggest:

1) For 1 < n < 1000, use an ABC2 file to retest each remaining k.
2) For 1000 < n < 25000, sieve the remaining k to a shallow depth (no more than 1e10), then run the output through PFGW 3.3.5 and 3.3.6 with the -F switch. Use diff plus and editor to pull out the candidates where 3.3.5 and 3.3.6 used different FFT sizes. Run those candidates through PFGW 3.3.6. This will be anywhere from 3% to 10% of the output from the sieve.
3) For n > 25000, residues should be posted in one of the threads. Grab the tested candidates, run through PFGW 3.3.5 and 3.3.6 to find out which use different FFT sizes. Use diff plus and editor to pull out the candidates where 3.3.5 and 3.3.6 used different FFT sizes. Run those candidates through PFGW 3.3.6. This should be a fairly low percentage (most likely under 3%).

I know that this seems to be a HUGE amount of work, but I don't think it should be too bad except for S63.

For this project, I would no longer accept any results from PFGW 3.3.4 and earlier. I suggest that you start a "double-check" sticky thread with a list of conjectures and k that need to be verified along with the steps I enumerated. I think that the thread should request that participants spend some time verifying completed work before grabbing new reservations. Participants can quickly reserve and verify those k, thus removing them from the list. It should be possible to verify each k up to n = 25000 in less than an hour.

I know that it looks like there are about 275,000 k remaining, but note that about 237,000 are S63, which is grossly inaccurate. About 70,000 have only been tested to n = 1000. About 45,000 have been tested to n = 10,000 and the rest have been eliminated due to a found prime. I suspect that many of the lower bases (below base 35) probably require no retesting, but that does need to be verified. Outside of S63, that means that there are under 1,000 k that will need to be retested to any extent.

The question is, how do we get users up to PFGW 3.3.6 right now for any work they are currently doing so that when the submit their results, that we can trust the accuracy of them?

Rogue, does this have anything to do with the constant switch between "All-complex fft lengths" and "zero padded fft lengths"? Also does it always constitute a problem, if the k*b^n+c test has been tested using an All-complex fft length, wich is usually 40 % faster? There has been especially many all complex fft length tests under the s58 conjecture.

Now I must admit, that I'm very very close to cancelling all my reservations and discard all work as "bogus". There is also no way that I can double check any of my S58, S60 and S70 results, since it will mean that I will first complete these 2 reservations + 1 re-reservation, some time in 2015. So do you have any stats on how many primes I can expect to have missed, below my current test depth of n=65K for S58?

I'll exam further explanations and come to a final conclusion, on weather or not to cancel my CRUS work, as new information sees the light of the day...

I'm not mad, to say it for once, but I for one, am not going to spend several CPU years on producing mallock work and doublechecking previous work. I will however see if I can find version 3.3.6 somewhere and upgrade immediately.

Regards

KEP

[quote=KEP;229303]Rogue, does this have anything to do with the constant switch between "All-complex fft lengths" and "zero padded fft lengths"? Also does it always constitute a problem, if the k*b^n+c test has been tested using an All-complex fft length, wich is usually 40 % faster? There has been especially many all complex fft length tests under the s58 conjecture.

Now I must admit, that I'm very very close to cancelling all my reservations and discard all work as "bogus". There is also no way that I can double check any of my S58, S60 and S70 results, since it will mean that I will first complete these 2 reservations + 1 re-reservation, some time in 2015. So do you have any stats on how many primes I can expect to have missed, below my current test depth of n=65K for S58?

I'll exam further explanations and come to a final conclusion, on weather or not to cancel my CRUS work, as new information sees the light of the day...

I'm not mad, to say it for once, but I for one, am not going to spend several CPU years on producing mallock work and doublechecking previous work. I will however see if I can find version 3.3.6 somewhere and upgrade immediately.

Regards

KEP[/quote]
Just a very small amount needs to be doublechecked not the whole thing. Rogue said ~3%.

I would suggest we should put this doublecheck work into a prpnet server to finish it quickly together. I am willing to test some candidates if someone posts a file.
Is 3.2.x affected?

[QUOTE=henryzz;229306]Is 3.2.x affected?[/QUOTE]

I think yes.

I've upgraded all of my computers to version 3.3.6. My quad's current batch of work will be done in a week or so; after that I'll be able to put it to work doing re-testing where needed. Note that I have limited access to my quad and all work must be fed to it via PRPnet, though that shouldn't be too big a deal since most of the work will be above n=1000 and thus somewhat reasonably PRPnet-able.

[QUOTE=KEP;229303]Rogue, does this have anything to do with the constant switch between "All-complex fft lengths" and "zero padded fft lengths"? Also does it always constitute a problem, if the k*b^n+c test has been tested using an All-complex fft length, wich is usually 40 % faster? There has been especially many all complex fft length tests under the s58 conjecture.

Now I must admit, that I'm very very close to cancelling all my reservations and discard all work as "bogus". There is also no way that I can double check any of my S58, S60 and S70 results, since it will mean that I will first complete these 2 reservations + 1 re-reservation, some time in 2015. So do you have any stats on how many primes I can expect to have missed, below my current test depth of n=65K for S58?

I'll exam further explanations and come to a final conclusion, on weather or not to cancel my CRUS work, as new information sees the light of the day...

I'm not mad, to say it for once, but I for one, am not going to spend several CPU years on producing mallock work and doublechecking previous work. I will however see if I can find version 3.3.6 somewhere and upgrade immediately.[/QUOTE]

I cannot answer your first set of questions. If George is following this thread, maybe he can chime in.

I don't consider all work to be bogus. You only need to look at the k for which no prime has been found. I would estimate that somewhere between 3% and 10% of PRP tests will use a different FFT size when run with 3.3.6 vs. 3.3.5 and earlier. Even though a different FFT size will be chosen, most of the residues will match between the two versions of PFGW.

If you want me to, I can double-check whatever work you did before you upgraded to PFGW 3.3.6. All you need to do is e-mail me the remaining k and the limits (upper bound of n) to which you tested with older versions. If you have sieve files or a list of candidates tested, that would save me some time. With such low bases I think that double-checking should take no more than 1 week.

I have no idea how to compute how many primes might have been missed.

To get a feel for what's involved in checking a particular base for errors, I did a "dry run" on Riesel base 6, which only has 2 k's remaining and thus is a rather easy one on which to do this. I've outlined the procedure here so others can base their efforts on it.

[B]Stage 1: re-test up to n=1K[/B]
For a tiny base like this, this part went really fast. I first made a file called r6.abc2 (you can name it whatever you want, just don't put spaces in it since that will make it a little hard to plug into PFGW later) with the following contents:
[code]ABC2 $a*6^$b-1 // {number_primes,$a,1}
a: in { 1597 36772 }
b: from 1 to 1000[/code]
I then ran it through PFGW 3.3.6 with the command line:
[I]pfgw -f30 r6.abc2[/I]
It finished within seconds; no primes were found. If there had been any (indicating something missed the first time around), they would have been output to pfgw.log. In that case, I would need to verify the PRPs as usual with the -tp switch and post them in the forum so Gary can remove the appropriate k's from the web pages.

[B]Stage 2: sieve n=1K-25K and check for FFT differences[/B]
Because the base is so low, I didn't go up to the full p=10G that Mark recommended. However, for most bases, you will want to sieve that high. I used the command line:
[I]srsieve -n 1000 -N 25000 -P 1e10 -G 1597*6^n-1 36772*6^n-1[/I]
For multiple k's, you can string them up on the command line as I did, or if there's a ton of them, make a sequences file--a basic text file containing one sequence per line in the same format I used--and give its name to srsieve in place of the sequences. (Note that in this case, I manually stopped the sieve with Ctrl-C a little past p=1G.)

I now had a sieve file, t17_b6.prp. (For other bases, the b* part will change accordingly and it will be t16 for Sierp., t17 for Riesel.) Now I ran that through both PFGW 3.3.5 and 3.3.6 with the -F switch; in both cases the command line was:
[I]pfgw -F -l t17_b6.prp[/I]
Each produces a file pfgw.out with each line describing the FFT it would use on a particular number. After running this through v3.3.5, I renamed pfgw.out to pfgw335.out and re-ran with 3.3.6.

Now I ran a diff on the two files to see where they differed. I have Cygwin installed, so I can use this command locally; Linux users can do similarly. If you're on Windows without Cygwin, you can get diff.exe and grep.exe from [URL]http://unxutils.sourceforge.net/[/URL] which behaves exactly like the Unix one I used. I used the command:
[I]diff pfgw335.out pfgw.out | grep ">" >> diff.txt[/I]
Now diff.txt contains just the lines representing numbers that need to be retested. In this case, I got an empty file, i.e. no numbers were bad. All clear in the 1K-25K range--next! :smile:

[B]Stage 3: check residuals on file for 25K+[/B]
First, email Gary and ask him for all the n=25K+ residuals he has on file for the base you're doing. Obviously, you won't get a reply [I]immediately[/I], so for the purpose of this example I pulled a random results file out of the R6 drive thread and used it for the comparison.

The file that I grabbed was named pfgw.out to begin with, so to avoid confusion with the output files I'll be getting later, I renamed that to orig.out.

I then coded up a quick Perl script (attached here) which can be used to strip the residues from the results file, leaving just the number itself. In this case, I used the following command line:
[I]perl results-to-input.pl orig.out orig.in[/I]
which made a file orig.in containing just the numbers from orig.out.

Now, I repeate the process from stage 2 for checking which numbers PFGW v3.3.6 uses a different FFT size. That is, run both versions with the -F option, and use diff to compare them. The format output by my Perl script, namely that of one candidate per line in raw format, is accepted directly by PFGW, so you can use it just like the sieve file in stage 2.

In this case, as before, diff.txt was empty, indicating that this range is all clear. If any numbers had needed to be retested, it would have shown something like this (just an example, edited manually for the purposes of demonstration; neither version of PFGW actually wanted to use the 32K FFT on this number):
[code]
> Special modular reduction using FFT length 40K on 36772*6^150007-1
[/code]
Thus, you need to add just the number "36772*6^150007-1" to a new file on a line by itself. Repeat for any more such numbers found in diff.txt. I don't yet have a script for doing this but if we find that some bases produce enough potentially bad tests to make it a pain to do this by hand, I'll see what I can come up with. Edit: done. The script is called fft-to-input.pl and the syntax is just like results-to-input.pl.

That's all there is to it. Oh, and BTW: in the course of this "dry run", I [B]re-checked R6 up to n=25K[/B]. No bad results were found (as expected due to the low b value).

Max :smile:

[QUOTE=mdettweiler;229317]
In this case, as before, diff.txt was empty, indicating that this range is all clear. If any numbers had needed to be retested, it would have shown something like this (just an example, edited manually for the purposes of demonstration; neither version of PFGW actually wanted to use the 32K FFT on this number):
[code]1c1
< Special modular reduction using FFT length 32K on 36772*6^150007-1
---
> Special modular reduction using FFT length 40K on 36772*6^150007-1
[/code]
Thus, you need to add just the number "36772*6^150007-1" to a new file on a line by itself. Repeat for any more such numbers found in diff.txt. I don't yet have a script for doing this but if we find that some bases produce enough potentially bad tests to make it a pain to do this by hand, I'll see what I can come up with.[/QUOTE]

You can use a text editor (like TextPad) with the output from diff and strip the characters before the number on each line, then feed that through PFGW, with one candidate on each line. A perl script could probably do that even easier.

[quote=rogue;229318]You can use a text editor (like TextPad) with the output from diff and strip the characters before the number on each line, then feed that through PFGW, with one candidate on each line. A perl script could probably do that even easier.[/quote]
Yes, that would be pretty simple. What I was thinking might be a little trickier, though, is to ensure that the script only outputs each candidate to the file once--not twice as diff does it.

Is there a flag for diff that can tell it to only output one instance of each differing line?

Bases to be checked
 

Here are the Riesel bases to be checked:

3, 5, 6, 7, 10, 15, 19, 22, 23, 24, 25, 26, 27, 28, 30, 31, 35, 36, 37, 39, 40, 42, 45, 46, 48, 49, 51, 52, 53, 55, 58, 60, 61, 67, 70, 72, 75, 78, 79, 80, 82, 87, 88, 91, 93, 94, 95, 96, 97, 100, 102, 103, 105, 107, 108, 109, 111, 112, 115, 117, 121, 123, 133, 135, 136, 138, 143, 147, 148, 152, 157, 158, 159, 160, 161, 162, 163, 168, 170, 172, 173, 177, 181, 182, 186, 187, 191, 192, 198, 199, 200, 211, 212, 213, 214, 217, 218, 221, 231, 233, 234, 235, 236, 241, 243, 249, 253, 258, 268, 272, 273, 275, 277, 281, 287, 288, 290, 298, 304, 308, 315, 318, 319, 320, 321, 326, 327, 328, 332, 333, 334, 337, 338, 343, 347, 353, 354, 361, 362, 363, 367, 368, 373, 380, 383, 392, 394, 398, 401, 402, 406, 409, 410, 412, 416, 417, 422, 423, 424, 425, 433, 439, 444, 446, 447, 452, 458, 463, 468, 469, 470, 471, 480, 488, 492, 493, 496, 497, 499, 500, 504, 505, 510, 514, 516, 520, 523, 526, 530, 533, 534, 543, 548, 549, 550, 551, 552, 563, 564, 566, 572, 573, 579, 580, 581, 582, 588, 589, 590, 601, 602, 603, 610, 611, 615, 617, 619, 620, 622, 625, 633, 634, 635, 636, 638, 639, 641, 650, 653, 654, 662, 665, 666, 667, 668, 677, 678, 679, 684, 686, 688, 694, 695, 696, 702, 703, 709, 710, 716, 720, 724, 727, 729, 730, 731, 736, 737, 741, 743, 744, 746, 747, 752, 753, 754, 759, 761, 773, 774, 782, 783, 784, 785, 789, 790, 798, 800, 803, 812, 814, 815, 821, 828, 832, 833, 836, 841, 845, 850, 859, 864, 866, 867, 868, 872, 875, 879, 883, 887, 888, 889, 892, 894, 900, 904, 905, 908, 916, 920, 926, 928, 931, 932, 941, 945, 951, 954, 962, 964, 967, 968, 980, 984, 987, 988, 994, 995, 998, 999, 1001, 1003, 1009, 1010, 1017, 1019, 1021, 1025, 1029


Here are the Sierpinski bases to be checked:

3, 5, 6, 7, 9, 10, 12, 15, 17, 19, 22, 24, 25, 26, 27, 28, 30, 31, 33, 35, 37, 39, 40, 42, 43, 45, 46, 48, 49, 51, 52, 53, 55, 58, 60, 61, 63, 67, 68, 70, 72, 73, 75, 78, 80, 81, 82, 86, 87, 88, 91, 93, 95, 97, 100, 101, 102, 103, 105, 107, 108, 111, 112, 115, 117, 118, 122, 123, 133, 135, 136, 138, 140, 143, 145, 147, 148, 151, 155, 157, 161, 162, 163, 165, 168, 171, 173, 174, 177, 178, 182, 183, 185, 187, 189, 191, 192, 193, 198, 199, 200, 204, 207, 208, 211, 212, 213, 214, 217, 218, 220, 227, 228, 230, 236, 248, 249, 252, 255, 257, 259, 263, 264, 273, 288, 289, 290, 294, 298, 304, 308, 311, 315, 317, 318, 319, 320, 326, 328, 332, 334, 335, 338, 340, 341, 342, 343, 353, 354, 365, 368, 373, 379, 380, 383, 386, 392, 394, 395, 397, 401, 402, 405, 406, 409, 410, 412, 414, 416, 417, 422, 425, 426, 428, 436, 444, 450, 458, 461, 463, 467, 468, 470, 472, 476, 480, 482, 492, 496, 497, 500, 501, 510, 516, 518, 520, 529, 530, 533, 542, 550, 567, 573, 574, 577, 578, 579, 582, 588, 593, 596, 605, 617, 620, 622, 626, 638, 643, 647, 649, 653, 676, 683, 688, 695, 696, 701, 702, 703, 706, 707, 712, 724, 731, 736, 737, 740, 743, 758, 761, 766, 767, 773, 774, 778, 780, 781, 784, 785, 788, 789, 790, 797, 798, 800, 802, 803, 805, 811, 814, 816, 821, 825, 828, 832, 833, 834, 835, 836, 844, 845, 850, 853, 859, 866, 867, 872, 875, 878, 879, 883, 887, 889, 893, 905, 908, 914, 917, 920, 928, 930, 932, 934, 935, 939, 945, 947, 949, 953, 961, 964, 968, 980, 983, 985, 987, 993, 995, 998, 999, 1001, 1002, 1004, 1006, 1010, 1013, 1016, 1026, 1029

Bases 2, 4, 8, etc. are excluded since George implied that base 2 was not affected.

I removed bases that I have double-checked already on the Sierpinski side.

If someone can sticky this thread and move this list to the first post, that would be great.

[QUOTE=mdettweiler;229319]Is there a flag for diff that can tell it to only output one instance of each differing line?[/QUOTE]

I'm not aware of anything, but you can use grep to only include lines with '<' or '>', for example:

diff pfgw_3.3.5.out pfgw_3.3.6.out | grep ">"

[quote=rogue;229321]I'm not aware of anything, but you can use grep to only include lines with '<' or '>', for example:

diff pfgw_3.3.5.out pfgw_3.3.6.out | grep ">"[/quote]
Ah, perfect--that should do the trick quite nicely. I'll change the above directions accordingly.

[QUOTE=rogue;229320]Quote containing multiple of bases, that I have for the concern of space, removed from the quoted text :)[/QUOTE]

I can see that all my reservations after the crash of my Quad, is affected. However, thanks for your offer on DC Rogue. I have written a lot to you in a PM, and as mentioned in the PM, I'm going to continue as if nothing has happened, after an upgrade to version 3.3.6! However, I'll do some brief DC of S383 as I come home in a short while, simply to see if a k has actually been missed for n<=1K. After that I might do a DC to n=25K, even though the Primepattern seems credibel, it is better to be safe than sorry.

Regards

Kenneth

[quote=rogue;229289]3) For n > 25000, ... This should be a fairly low percentage (most likely under 3%).[/quote]

That's where almost all my reservations belong. I've now completely checked FFT sizes 3.3.5 vs 3.3.6 for all my reservations past and current.

n>25K, 16 bases (42<b<930), ~75 k's (2<k<12,000), lowest FFT size 12K, ~120,000 tests. 0 tests with differences in FFT sizes.

Also 10K<n<30K, Sierp 35, ~800 k's (46<k<215,000), lowest FFT size 4K, ~410,000 tests. 0 tests with differences in FFT sizes.

Most of my testing was done using LLR's N-1/N+1. I recall a post by J.[COLOR=Black]Penné[/COLOR] saying LLR lets FFT choice be handled by gwnum. If the same is true for PFGW then both programs should do the same test with the same FFT length (?) (if someone knows this to be wrong say so) and so all results I have submitted so far should be ok (i.e. done on the same FFT size that 3.3.6 would have chosen).

Was the 3% an estimate based on differences at much lower n levels? Based on my numbers it looks like (for n>25K) it will be much lower than that. We should know when others report some more stats, but if double-checking will be necessary for only small n's then the overall work should be reasonable.

[QUOTE=vmod;229335]That's where almost all my reservations belong. I've now completely checked FFT sizes 3.3.5 vs 3.3.6 for all my reservations past and current.

n>25K, 16 bases (42<b<930), ~75 k's (2<k<12,000), lowest FFT size 12K, ~120,000 tests. 0 tests with differences in FFT sizes.

Also 10K<n<30K, Sierp 35, ~800 k's (46<k<215,000), lowest FFT size 4K, ~410,000 tests. 0 tests with differences in FFT sizes.

Most of my testing was done using LLR's N-1/N+1. I recall a post by J.[COLOR=Black]Penné[/COLOR] saying LLR lets FFT choice be handled by gwnum. If the same is true for PFGW then both programs should do the same test with the same FFT length (?) (if someone knows this to be wrong say so) and so all results I have submitted so far should be ok (i.e. done on the same FFT size that 3.3.6 would have chosen).

Was the 3% an estimate based on differences at much lower n levels? Based on my numbers it looks like (for n>25K) it will be much lower than that. We should know when others report some more stats, but if double-checking will be necessary for only small n's then the overall work should be reasonable.[/QUOTE]

I'm somewhat surprised to learn that none of your tests would use different FFT lengths. Higher n have fewer problems than lower n. Could you please list the bases referenced above (where you say you have 16 bases)? Someone still needs to look at n < 25000 for those k. It should less than a day to verify S35 for n<10000 for the remaining k.

LLR and PFGW let gwnum choose the FFT size. PFGW can tell gwnum to choose a larger size using the -a switch or internally if it detects a roundoff error.

And of course if you haven't done so (and it sounds like you have), make sure all of your clients are running PFGW 3.3.6.

I have completed a double-check of R1029. After sieving I discovered that 18% of the candidates for n < 25,000 had to be retested. That is twice as high as I have seen for any other base. It took me about 5 hours (split across 4 CPUs) to complete, about 2.5 hours per k. Lower bases should take far less time per k.

[quote=rogue;229338]Could you please list the bases referenced above (where you say you have 16 bases)? Someone still needs to look at n < 25000 for those k. It should less than a day to verify S35 for n<10000 for the remaining k.[/quote]
Not all were at n=25K some started higher (also three were subsequently proven, I just included those to sample as many tests possible in order to estimate the magnitude of the problem at high n ranges). I compared FFT sizes for:
S55, S100, S102, S189, R272, R275, S869, S914, S917, S919, S930 (25K-)
S133 (40K-)
R42, R133 (50K-)
R110 (70K-)

I can take care of checking remaining k's for n lower than the ranges above (I'm sure you have enough of your own work to double-check). Most of mine are single/few-k bases anyway, so it should be done fast. I'll do S35 (n<10K) too.

I'll post some stats on the lower n's when done. I guess more important than the number of FFT size differences will be the number of actual erroneous results (possible residue mismatches or false-composites :shock: -if any).

[quote=rogue;229338]And of course if you haven't done so (and it sounds like you have), make sure all of your clients are running PFGW 3.3.6.[/quote]
They are :tu:.

[QUOTE=vmod;229335]Was the 3% an estimate based on differences at much lower n levels?[/QUOTE]

Yes.

I run one of my bases for n<25K. No errors as well.

Suspicious that something is not right I checked a couple of the examples given in post #6 on this thread. I have reason to believe that something is amiss with version 3.3.5.
All win versions downloaded from here: [URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/[/URL]

[code]PFGW Version 3.3.4.20100405.Win_Stable [GWNUM 25.14]
Output logging to file pfgw.out
Special modular reduction using all-complex FFT length 512 on 22*283^602+1
22*283^602+1 is composite: RES64: [8F162415CC796787] (0.0317s+0.0001s)
Done.

PFGW Version 3.3.5.20100908.Win_Stable [GWNUM 25.14]
Output logging to file pfgw.out
Special modular reduction using zero-padded FFT length 768 on 22*283^602+1
22*283^602+1 is composite: RES64: [8F162415CC796787] (0.0461s+0.0001s)
Done.

PFGW Version 3.3.6.20100908.Win_Stable [GWNUM 25.14]
Output logging to file pfgw.out
Special modular reduction using zero-padded FFT length 768 on 22*283^602+1
22*283^602+1 is composite: RES64: [8F162415CC796787] (0.0467s+0.0002s)
Done.[/code]3.3.5 > Special modular reduction using zero-padded FFT length 768 on 22*283^602+1
3.3.6 > Special modular reduction using zero-padded FFT length 768 on 22*283^602+1

Also:
[code]
PFGW Version 3.3.4.20100405.Win_Stable [GWNUM 25.14]
Output logging to file pfgw.out
Special modular reduction using all-complex FFT length 6K on 662*43^9802+1
662*43^9802+1 is composite: RES64: [E060CD9C886E2362] (4.6589s+0.0005s)
Done.

PFGW Version 3.3.5.20100908.Win_Stable [GWNUM 25.14]
Output logging to file pfgw.out
Special modular reduction using zero-padded FFT length 6K on 662*43^9802+1
662*43^9802+1 is composite: RES64: [E060CD9C886E2362] (4.2856s+0.0005s)
Done.

PFGW Version 3.3.6.20100908.Win_Stable [GWNUM 25.14]

Output logging to file pfgw.out
Special modular reduction using zero-padded FFT length 6K on 662*43^9802+1
662*43^9802+1 is composite: RES64: [E060CD9C886E2362] (4.3264s+0.0006s)
Done.[/code]3.3.5 > Special modular reduction using zero-padded FFT length 6K on 662*43^9802+1
3.3.6 > Special modular reduction using zero-padded FFT length 6K on 662*43^9802+1

So Rogue can you please verify that the win 3.3.5 uploaded is indeed what it should be (looks like a copy of 3.3.6 build rather than a modified 3.3.4).

I'm stopping any double-checking until this is cleared up. If anyone else is double-checking test the pfgw versions you use against the examples on post [URL="http://www.mersenneforum.org/showpost.php?p=229283&postcount=6"]#6[/URL] before continuing.

At one point they were correct, but I had the wrong build date, so I rebuilt them.

Update: I have reposted a Windows 3.3.5 build here, [url]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/[/url]. I'm sorry for those of you who need to run it again with the -F switch. I have been using the correct build, but when I had to re-release due to the wrong date, I linked with the wrong version of gwnum.

I have 2 Windows machines and 11 Linux machines and the 2 Windows machines are not running CRUS right now. I need a PFGW 3.3.6 Linux version upgrade ASAP please.

KEP and co. Please don't sweat this and change or cancel your reservations. Your work is effectively guaranteed to be 97% OK. If we missed some primes here and there, so be it. And of that 3% that is bad, only a small percentage of those will be primes. Just because a residual is bad does not mean that a prime was missed; only that it is possible that one was missed. This is not NPLB where we try to find ALL primes in large swaths of k and n ranges. Although CRUS largely prefers that the lowest prime be found for each k, it is not a requirement. We will take this in baby steps and slowly doublecheck only k's that are remaining. We don't need to make a beeline to doublecheck all k's remaining to their current search depth. That would be horrible espcially for the numerous k's that have been tested to n=100K or more.

Let's everyone think of this as a situation where 2-3% of us had bad machines and so there are random bad residuals out there for random bases but we don't know which. We wouldn't freak out and have to immediately doublecheck everything. I do random doublechecks on many bases up to n=2500 or n=5000 and for some up to n=25K and I have many sieve files that I have kept. I can from time to time do that some more.

My opinion: Let's not even make a formal effort out of this. Feel free to doublecheck a base here and there as you see fit; especially ones that you happened to have worked on in the past. No pressure. When done with the doublecheck, please report that effort here and we will keep a formal posting of all bases and k's that have been doublechecked. Eventually we'll probably get down to some difficult ones that need to be doublechecked such as S63 or R51, the latter of which I haven't even listed the k's remaining yet.

At some point, I may even come up with a notation on the pages that bases have been doublechecked but would prefer to keep such a listing in an updatable posting for the time being.


Gary

[quote=rogue;229372]At one point they were correct, but I had the wrong build date, so I rebuilt them.

Update: I have reposted a Windows 3.3.5 build here, [URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/[/URL]. I'm sorry for those of you who need to run it again with the -F switch. I have been using the correct build, but when I had to re-release due to the wrong date, I linked with the wrong version of gwnum.[/quote]

Why are we messing with versions 3.3.5 and 3.3.6? Which one actually works? If both, why are we still updating a version that is not the newest?

No offense but I hate sourceforge. Please provide a direct link. Thanks. When I clicked on version 3.3.6, it took me to [URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/pfgw_win_3.3.6_20100907.zip?view=log[/URL]. OK, that looks good. Now I click on the "SCM Repositories - openpfgw" link at the top of the page and I get [URL]http://sourceforge.net/projects/openpfgw/[/URL]. On this page, it shows a "Download now!" link that is version 3.3.4! ?????????????

I'm all ears as to what I did wrong here.

Now do you understand why I dislike sourceforge? This kind of run-around is time consuming and irritating.

[quote=gd_barnes;229525]Why are we messing with versions 3.3.5 and 3.3.6? Which one actually works? If both, why are we still updating a version that is not the newest?

No offense but I hate sourceforge. Please provide a direct link. Thanks. When I clicked on version 3.3.6, it took me to [URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/pfgw_win_3.3.6_20100907.zip?view=log[/URL]. OK, that looks good. Now I click on the "SCM Repositories - openpfgw" link at the top of the page and I get [URL]http://sourceforge.net/projects/openpfgw/[/URL]. On this page, it shows a "Download now!" link that is version 3.3.4! ?????????????

I'm all ears as to what I did wrong here.

Now do you understand why I dislike sourceforge? This kind of run-around is time consuming and irritating.[/quote]
Here's a direct link (to the Windows 3.3.6 version):
[URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/pfgw_win_3.3.6_20100907.zip?revision=300[/URL]

For future reference, I've attached a screenshot with arrows drawn on pointing to where you need to click to get the same result as a direct link. (I say arrows plural because there are two links in two different places on the page that do the exact same thing. Yes, it's weird. :smile:)

[QUOTE=gd_barnes;229525]Why are we messing with versions 3.3.5 and 3.3.6? Which one actually works? If both, why are we still updating a version that is not the newest?

No offense but I hate sourceforge. Please provide a direct link. Thanks. When I clicked on version 3.3.6, it took me to [URL]http://openpfgw.svn.sourceforge.net/viewvc/openpfgw/pfgw_win_3.3.6_20100907.zip?view=log[/URL]. OK, that looks good. Now I click on the "SCM Repositories - openpfgw" link at the top of the page and I get [URL]http://sourceforge.net/projects/openpfgw/[/URL]. On this page, it shows a "Download now!" link that is version 3.3.4! ?????????????

I'm all ears as to what I did wrong here.

Now do you understand why I dislike sourceforge? This kind of run-around is time consuming and irritating.[/QUOTE]

If you haven't been following the thread, the change made to 3.3.5 was to add support for the -F switch. It was not supposed to have the latest gwnum library linked with it. I accidentally linked it with the patched gwnum, so what I posted originally for 3.3.5 was the same as 3.3.6. I provided a 3.3.5 because users want to know what PRP work they have done needs to be redone and that was the best way in which to do it.

I tried to fix the files at this link, [URL]http://sourceforge.net/projects/openpfgw/[/URL], but have been unable to log into sourceforge to update it due to some issue on their end. I'll try again today.

Whether or not you like sourceforge is none of my concern. I needed to put the PFGW source into a safe place where multiple people could work on it simultaneously (if desired). The 1.x version was put into a CVS repository somewhere, but there is no reference to where that repository is and the previous owner of PFGW development, Jim Fougeron, is MIA.

[quote]
My opinion: Let's not even make a formal effort out of this. Feel free to doublecheck a base here and there as you see fit; especially ones that you happened to have worked on in the past. No pressure. When done with the doublecheck, please report that effort here and we will keep a formal posting of all bases and k's that have been doublechecked. Eventually we'll probably get down to some difficult ones that need to be doublechecked such as S63 or R51, the latter of which I haven't even listed the k's remaining yet.[/quote]

I am quite OK with this. It will save me a lot of time as I would probably be doing most of the double-checking work. If you decide in the future that double-checking is important, then IMO the higher bases need to be attacked first because a higher percentage of those tests will be using a different FFT size.

[quote]
I have 2 Windows machines and 11 Linux machines and the 2 Windows machines are not running CRUS right now. I need a PFGW 3.3.6 Linux version upgrade ASAP please.[/quote]

I am working with Steven Harvey to get a Linux build. He responded to my inquiry yesterday, so hopefully the Linux build will be available later today.

[QUOTE=KEP;229331]I can see that all my reservations after the crash of my Quad, is affected. However, thanks for your offer on DC Rogue. I have written a lot to you in a PM, and as mentioned in the PM, I'm going to continue as if nothing has happened, after an upgrade to version 3.3.6! However, I'll do some brief DC of S383 as I come home in a short while, simply to see if a k has actually been missed for n<=1K. After that I might do a DC to n=25K, even though the Primepattern seems credibel, it is better to be safe than sorry.[/QUOTE]

I completed the double-check for n > 25,000 for S58 and S70. Of the over 128,000 tests done with 3.3.4, I only had about 400 to redo, which is only about .3%.

Mark,

Thanks for the info. and sorry I hadn't read all of the posts in the thread. I concluded what Max had showed about Sourceforge before he posted it but it took me 5-10 mins. to do so, which was frustrating. I'm not the only one who has been confused by Sourceforge. It came up with 2 others in the PFGW thread, who also asked for direct links because they couldn't find the program that they were looking for. The pages are designed for developers not users so some of the terminology means nothing to us.

Instead of having that huge link at the top of the subsequent page which automatically diverts your attention from the very teeny links in the middle of the page that need to be chosen, the links that need to be chosen should be either highlighted, increased greatly in size, or placed at the top of the page. I realize that there is nothing you can do about it so I'd simply like to ask for direct links in the future.

It wasn't your fault the GWNUM libraries turned out to be bad. You shouldn't have to doublecheck everything. But we'd certainly welcome your help. I'll do some here and there and hopefully others will chip in at times.

I am now reading back through the 10s of posts in this thread to get more up to speed on this issue.


Gary

I deleted a couple of my posts and a response and edited another after reading back through many of the posts in this thread. I should have done that to begin with. Most of my questions have been answered. It's nice to know that we only need to doublecheck only tests with fftlen differences. That seems like a very manageable task.

[QUOTE=gd_barnes;229623]I deleted a couple of my posts and a response and edited another after reading back through many of the posts in this thread. I should have done that to begin with. Most of my questions have been answered. It's nice to know that we only need to doublecheck only tests with fftlen differences. That seems like a very manageable task.[/QUOTE]

Let me know how you want to proceed. The task is large, but manageable.

I have been able to update the files here, [url]http://sourceforge.net/projects/openpfgw/[/url], so there is less to navigate through. The website should put the platform specific version for the OS into the link of the big green button. I didn't link to specific files/versions because users who want to double-check will want both 3.3.5 and 3.3.6, but if someone want to put links to each version into a post, they are welcome to do so.

I had over 87000 tests complete with PFGW on R928. Only 19 used different FFT lengths with PFGW 3.3.6. I was expecting to re-run thousands of tests. This will only set me back about 30 minutes. I still need to look again at 1000 < n < 15000 though.

[quote=rogue;229666]Let me know how you want to proceed. The task is large, but manageable.

I have been able to update the files here, [URL]http://sourceforge.net/projects/openpfgw/[/URL], so there is less to navigate through. The website should put the platform specific version for the OS into the link of the big green button. I didn't link to specific files/versions because users who want to double-check will want both 3.3.5 and 3.3.6, but if someone want to put links to each version into a post, they are welcome to do so.[/quote]

Let's work backwards from base 1029 since the greatest chance of error is in the larger bases but let's skip bases that have > 25 k's remaining for now. We'll knock out the low-lying fruit first. My preference is to have entirely accurate 1k remaining and proven/1k/2k/3k remaining lists. In other words, better is to have a base that currently shows 500 k's remaining that really should only have 498 k's remaining than have a base that shows 1 or 2 k's remaining that should actually be proven, since the latter is the point of the project.

This is only a suggestion. If anyone wants to doublecheck any base that he personally likes regardless of its size, k's remaining, or whomever originally worked on it, we certainly won't argue. Anyone feel like doublechecking base 3? :smile:

[QUOTE=gd_barnes;229673]Let's work backwards from base 1029 since the greatest chance of error is in the larger bases but let's skip bases that have > 25 k's remaining for now. We'll knock out the low-lying fruit first. My preference is to have entirely accurate 1k remaining and proven/1k/2k/3k remaining lists. In other words, better is to have a base that currently shows 500 k's remaining that really should only have 498 k's remaining than have a base that shows 1 or 2 k's remaining that should actually be proven, since the latter is the point of the project.[/QUOTE]

I agree that we should start on the larger bases. The problem will be managing the list of k's that need double-checking. Do we need a sticky thread for that?

Once I complete my current reservations on small conjectures, I should be able to rip though dozens of bases fairly quickly.

Gary, since we only need to check for k that are remaining, base 3 shouldn't be too hard. Just use the list of remaining k to produce and ABC2 file (as shown earlier by Max). It might need be done in multiple iterations, but it should be really fast. I put over 700 k on a single line, so even if one put 100 values on a single line, it wouldn't take too many iterations with an ABC2 file.

I recommend that double-checking only sieve up to about 1e9. 1e10 is too deep considering the percentage of k that will remain.

I suspect more time might be spent on some conjectures where n is really high. A few high n can take a lot more time than some bases.

Finally, we need to ensure that users are using 3.3.6 so that no work going forward needs to be double-checked.

R928 has been double-checked. I only had to redo less than 600 total tests between 1000 < n < 20000. That was from over 600,000 tests.

I've had some more time today to check some of my past/current bases with the proper 3.3.5. It's difficult to estimate the amount of double-checking effort beforehand. Similar bases can behave differently.

For reference the percentages of tests that had different FFT lengths and need a double-check were:
S35: (n<10K) 1.6%, (10K<n<31K) 0.5%
S55: (n<25K) 3.3%, (25K<n<50K) 3.1%, (50K<n<115K) 0%
S100: (n<25K) 0.7%, (25K<n<100K) 4.2%
S102: (n<25K) 2.1%, (25K<n<100K) 1.2%
R275: (n<150K) 0%
S914: (n<25K) 0.1%, (25K<n<100K) 0.2%
S930: (n<25K) 9.4%, (25K<n<100K) 14.1%

I've re-run the tests with 3.3.6 for S35, S55, S100, S102 and S914. All residues matched. These bases along with R275 should be considered completely double-checked up to their current testing limits.

I will take on double-checking the rest of the bases I've worked on (R42, R133, S133, S189, R272, S917, S930). I'll do 1 or 2 bases per day (depending on the number of tests) and give an update next week when done.

So, as I understand post #14 correctly, I should recheck my reserved bases s17 and s19 till 25k first, to see, if there is any difference, right?
If so, what now? Do I have to retest the complete base? s17 has been tested really really far.. a recheck would need many months :(

What with s63? I think I have to recheck it too, right?

[quote=Xentar;229989]So, as I understand post #14 correctly, I should recheck my reserved bases s17 and s19 till 25k first, to see, if there is any difference, right?
If so, what now? Do I have to retest the complete base? s17 has been tested really really far.. a recheck would need many months :(

What with s63? I think I have to recheck it too, right?[/quote]
It is just the tests that the new version uses a different fft length for that need doublechecking. This shouldn't be that many. See post #14 for instructions to find which need testing. The lower the base the less. AFAIK no wrong residues have been found b!=n

[QUOTE=henryzz;229991]It is just the tests that the new version uses a different fft length for that need doublechecking. This shouldn't be that many. See post #14 for instructions to find which need testing. The lower the base the less. AFAIK no wrong residues have been found b!=n[/QUOTE]
Ok, I re-read the post.
So, when using the -F parameter, it just writes the FFT size to a file, but doesn't do a PRP test.

And now I just have to re-test those, where this FFT size differs.

Thank you. :smile:

[QUOTE=Xentar;229992]And now I just have to re-test those, where this FFT size differs.[/QUOTE]

And only if a prime hasn't been found. I don't expect S63 to be too bad, but one can only guess to the number of retests needed.

I have found missing primes for Steven Harvey's Generalized Woodall project by looking for retests. The interesting thing is that the list of new primes I have found were missed by an older version of PFGW (possibly 1.x) and would have been found with PFGW 3.3.4 had it been used.

I'm curious: Have any new primes been found with version 3.3.6 that were previously missed by version 3.3.4?

Xentar, to add a little more to what Mark said, we are only asking that people doublecheck the k's where there is no prime, i.e. k's remaining. We are not requiring that the lowest prime be found for each k so it is unnecessary to doublecheck k's with primes. That should make the doublecheck of your S17 effort not too bad, especially since the remaining k is very low weight.

[QUOTE=gd_barnes;230008]I'm curious: Have any new primes been found with version 3.3.6 that were previously missed by version 3.3.4?[/QUOTE]

The only ones I have found to date were ones that I found when testing k*b^b+/-1 numbers. Other primes that I found were missed by older versions of PFGW. PFGW 3.3.4 would have found them. There were some issues with small FFTs that George fixed in gwnum 25.14, which PFGW first used in v3.3.3. I have not gone back to 3.3.3 to see if they were missed there.

I have discovered other k*b^n+/-c numbers where the residues changed between 3.3.4 and 3.3.6 due to the FFT size change, but none of those were prime. I don't want anyone to think that the problem is limited to k*b^b+/-1 numbers only.

I think it is possible that other primes were missed by 3.3.3 and earlier that would have been found by 3.3.4. I would not have believed that unless I found the undiscovered primes. Eventually it would probably behoove this project (and others) to retest all k*b^n+/-c for remaining k and n < 25,000 with PFGW 3.3.6. I will not start such an effort though because it is too massive. Now if I had 1000 cores at my disposal, then I might change my mind.

Riesel base 36
 

I've double checked the Riesel 36s that I earlier took to n = 100,000.
8363*36^n-1
42227*36^n-1
56093*36^n-1
96497*36^n-1
108439*36^n-1
112997*36^n-1
115883*36^n-1
116329*36^n-1

There were some different FFTs, only with k = 8363. But no new prime...

Willem.

Gary, since you are a moderator, could you update this post, [url]http://www.mersenneforum.org/showpost.php?p=229320&postcount=17[/url] to reflect what hasn't been checked for re-testing? I hope to take out as many as I can this weekend.

Regardin S383, I did a full doublecheck to n=24K, for all 48 k's remaining. There were no difference in the residuals printed by PFGW version 3.3.4 and the residuals printed by PFGW version 3.3.6! I had about 9300 tests for n>24K for S383, I did an -F comparisation, and none was found to be using a different FFT length when running PFGW 3.3.6 compared to the prior versions. So for the next couple of weeks I'm focusing on getting S383 to n=50K, before resuming S58 to n=125K!

And again to be clear, I'm not cancelling any work, however if a prime is missed at n<=1K, it just makes no sence to test that k into the millions before it finally makes a prime, that would really be a waste of resources. Well thats just my two cents. Mark thanks for your DC effort on S58 and S70. Continuing happily with the PFGW 3.3.6 version.

Take care everyone

KEP

I am going to look for retests for all Riesel bases >= 800. I have already looked at n < 1000 for the remaining k for those bases. I'll provide the results and stats when it has completed. I'm sieving at the moment and thus don't know how many retests are required.

[QUOTE=rogue;230110]I am going to look for retests for all Riesel bases >= 800. I have already looked at n < 1000 for the remaining k for those bases. I'll provide the results and stats when it has completed. I'm sieving at the moment and thus don't know how many retests are required.[/QUOTE]

I have completed sieving and have some stats. Overall I am checking 58 bases. I need to retest about 5.75% of the tests, 19,062 of 331,392. For some bases I over-sieved and for others I under-sieved. On average I sieved to about 1e10, which is a little deeper than I actually needed to sieve.

The base with the most tests was 999, with 58199 tests after sieving, but that base only yielded 21 retests. I had one base, 931, with 0 retests and a few other bases with less than 10. The base with the highest percentage of retests was 812, with 1046 of 2667 (39.22%) needing to be retested. Base 815 has the most retests (1596 of 5294, 30.15%). I expect base 1019 to take the longest to retest as there are 207 retests for n > 114,000.

I'm also taking all Sierpinski bases >= 800. I've double-checked to n=500 and am in the process of sieving. Fortunately the nastiest bases, 928 and 999 have well < 1% retesting needed, so I won't sieve very deeply for them.

Mark, I'm curious: Can you post primes that you found with PFGW 3.3.6 that were missed with 3.3.4? I haven't seen any yet. I think you said that none have been found except in the k*b^b-/+1 search and that thread was too long for me to dig through. The only issue that I have with that statement is that I ran the Sierp side of that search for k<=10K and b<=1K with both versions of PFGW and posted the primes for Karsten to add to his page. It was a complete doublecheck (not just the ones with fftlen differences). No differences in primes were found. (I didn't output residues.)

Were the missing prime(s) found on the Riesel side of the kbb1 search or for k>10K or for b>1K ? I'm very curious as to where these missing primes will pop up.

3 more things:

1. Without a Linux version, my testing is continuing with 3.3.4.
2. We'll need to add R51 to n=3K as well as new bases that I've been running to n=25K that were done since the time that you made your list to the bases that need to be checked.
3. For lack of wanting to administer another effort here, I'm dragging my feet on keeping track of what's been done and is left to do. I'll likely get more motivated when a missing prime is found.

[QUOTE=gd_barnes;230732]Were the missing prime(s) found on the Riesel side of the kbb1 search or for k>10K or for b>1K ?[/QUOTE]

The candidates during testing k*b^b+1 were:
[code]
1000000006:P:1:619:1
9238 619
1000000006:P:1:626:1
9276 626
9876 626
1000000006:P:1:627:1
7504 627
8004 627
9056 627
9256 627
1000000006:P:1:635:1
9386 635
1000000006:P:1:650:1
8619 650
9732 650
[/code]

What I've done was, comparing the results with LLR 3.8.1, pfgw 3.3.4 and pfgw 3.3.6 where I found errors with those pairs before.

Results:
- LLR 3.8.1: none of the candidates were found PRP/prime (ERROR: ROUND OFF for 9876*626^626+1)
- pfgw 3.3.4: same here, all composite
- pfgw 3.3.6: 9238*619^619+1 is 3-PRP, others composite

The Riesel-side I've not checked yet.

[QUOTE=kar_bon;230737]The candidates during testing k*b^b+1 were:
[code]
1000000006:P:1:619:1
9238 619
1000000006:P:1:626:1
9276 626
9876 626
1000000006:P:1:627:1
7504 627
8004 627
9056 627
9256 627
1000000006:P:1:635:1
9386 635
1000000006:P:1:650:1
8619 650
9732 650
[/code]

What I've done was, comparing the results with LLR 3.8.1, pfgw 3.3.4 and pfgw 3.3.6 where I found errors with those pairs before.

Results:
- LLR 3.8.1: none of the candidates were found PRP/prime (ERROR: ROUND OFF for 9876*626^626+1)
- pfgw 3.3.4: same here, all composite
- pfgw 3.3.6: 9238*619^619+1 is 3-PRP, others composite

The Riesel-side I've not checked yet.[/QUOTE]

That is very different than what I got. Here it is:

PFGW 3.3.4:
9238*619^619+1 is composite: RES64: [AF5E2DAE777932BE] (0.1777s+0.0002s)
9276*626^626+1 is 3-PRP! (0.3916s+0.0004s)
9876*626^626+1 is composite: RES64: [B377446921E05347] (0.1782s+0.0030s)
7504*627^627+1 is composite: RES64: [F46DD144C5625090] (0.1808s+0.0003s)
8004*627^627+1 is composite: RES64: [44A8BD3280FB61E2] (0.1807s+0.0003s)
9056*627^627+1 is composite: RES64: [161BFF99F7AF07DC] (0.1780s+0.0003s)
9256*627^627+1 is composite: RES64: [58B08246F49CEAEC] (0.1778s+0.0003s)
9386*635^635+1 is 3-PRP! (0.1803s+0.0003s)
8619*650^650+1 is 3-PRP! (0.3837s+0.0023s)
9732*650^650+1 is 3-PRP! (0.1905s+0.0022s)

PFGW 3.3.6:
9238*619^619+1 is 3-PRP! (0.2151s+0.0002s)
9276*626^626+1 is 3-PRP! (0.2170s+0.0028s)
9876*626^626+1 is 3-PRP! (0.2176s+0.0020s)
7504*627^627+1 is 3-PRP! (0.2182s+0.0027s)
8004*627^627+1 is 3-PRP! (0.2194s+0.0026s)
9056*627^627+1 is 3-PRP! (0.2186s+0.0023s)
9256*627^627+1 is 3-PRP! (0.2253s+0.0020s)
9386*635^635+1 is 3-PRP! (0.2221s+0.0034s)
8619*650^650+1 is 3-PRP! (0.2340s+0.0029s)
9732*650^650+1 is 3-PRP! (0.2327s+0.0020s)


When applying the -t switch to both PFGW 3.3.4 and 3.3.6, all were found prime, as they should be.

You must be using different versions than 3.3.4 and 3.3.6 in your tests.

So we have 6 primes that were not identified as PRP by PFGW 3.3.4 but were identified as PRP by PFGW 3.3.6.

[QUOTE=gd_barnes;230740]That is very different than what I got.[/QUOTE]

Sorry, mixed up the parameters. I forgot '-t' in pfgw here but done when I found those differences in pfgw V3.3.4, V3.3.6 and LLR.

[QUOTE=gd_barnes;230740]
So we have 6 primes that were not identified as PRP by PFGW 3.3.4 but were identified as PRP by PFGW 3.3.6.
[/QUOTE]

And none prime with LLR 3.8.1.

[QUOTE=gd_barnes;230732]1. Without a Linux version, my testing is continuing with 3.3.4.[/QUOTE]

There is a linux pfgw 3.3.6 uploaded at SourceForge
[URL]http://sourceforge.net/projects/openpfgw/files/pfgw_linux_3.3.6_20100913.zip/download[/URL]

Very good. Thanks Vmod.

[QUOTE=gd_barnes;230732]1. Without a Linux version, my testing is continuing with 3.3.4.
2. We'll need to add R51 to n=3K as well as new bases that I've been running to n=25K that were done since the time that you made your list to the bases that need to be checked.
3. For lack of wanting to administer another effort here, I'm dragging my feet on keeping track of what's been done and is left to do. I'll likely get more motivated when a missing prime is found.[/QUOTE]

I posted [URL="http://www.mersenneforum.org/showpost.php?p=229663&postcount=88"]this[/URL] eight days ago. You must have missed it.

I have yet to find any primes for CRUS that were missed by PFGW 3.3.4. I found one that was missed by a version of PFGW prior to 3.3.4 (mentioned [URL="http://www.mersenneforum.org/showpost.php?p=229276&postcount=799"]here[/URL]), but would not have been missed by 3.3.4. I also mentioned [URL="http://www.mersenneforum.org/showpost.php?p=230003&postcount=44"]here[/URL] about Generalized Woodalls that I found when retested with PFGW 3.3.6. These would have been found by PFGW 3.3.4. They were tested a long time ago, most likely by PFGW 1.x.

Of course if I find any missed by CRUS, I will post it here.

Sierpinski retest stats
 

I have completed sieving for retesting of Sierpinski bases >= 800 and have some stats. Overall I am checking 64 bases. I need to retest about 0.91% of the tests, 7408 of 816,786. Using some of the Riesel data for stats I limited my sieving on the heaviest bases (928 and 999, which accounted for more than 600,000 of the above number), I was able to determine that the number of retests for those bases would be very low, so I sieved to about 1e9 for them, which saved me a lot of time. Most of the others were sieved to 1e10, which, as it turns out, was too high for them as well. I probably could have sieved to about 5e9 and would have been okay for most of the bases.

The base with the most tests was 928, with 510,105 tests after sieving, but that base only yielded 30 retests. I had three bases with 0 retests and a a number of other bases with less than 10. The base with the highest percentage of retests was 842, with 499 of 1257 (31.84%) needing to be retested. A couple of other bases had over 15%. I expect base 828 to take the longest to retest as there a number of n > 25,000 to retest. Base 920 has the most retests at 559.

An interesting thing to note is that R814 has a retest rate around 23%, but S814 is only about .25%. This implies to me that the value of k has a significant impact on FFT size selection.

I am about 60% through the resting of the Riesel side for bases >= 800. No new primes have been discovered. I hope to be finished with both sides for bases >= 800 next week.

It seems that it [URL="http://mersenneforum.org/showthread.php?p=231308#post231308"]could be fun[/URL] to build the experimental PFGW version with libgw 26.2 - many new granular FFT sizes, possibly faster in many conditions etc...

[QUOTE]It seems that it [URL="http://mersenneforum.org/showthread.php?p=231308#post231308"]could be fun[/URL] to build the experimental PFGW version with libgw 26.2 - many new granular FFT sizes, possibly faster in many conditions etc... [/QUOTE]LLR 3.8.2 WOW Maybe 10% faster.

Edit: 10% faster as advertised

[QUOTE=Batalov;231315]It seems that it [URL="http://mersenneforum.org/showthread.php?p=231308#post231308"]could be fun[/URL] to build the experimental PFGW version with libgw 26.2 - many new granular FFT sizes, possibly faster in many conditions etc...[/QUOTE]

Already in progress. In fact I am working on a 64-bit build, which is even faster than the 32-bit build of v26.2.

I've now double-checked S133, S148, S189, S917, S930 and R133, R148, R272 up to their current testing limit. All residues matched.

Only remaining from the bases I've worked on is R42. It's ok up to n=1000 and needs a double-check on ~18% of the tests (evenly distributed up to n=100K). I'll leave R42 double-check for the future or for someone else if willing. It's only recently I finished with >40,000 R42 tests so it would be boring to take it on right now.

I have double-checked to n=40,000 for b>=800, both Sierpinski and Riesel. I have about 3500 tests to go. I have discovered no new primes.

I also double-checked remaining k to n=1,000 for all b>=100, both Sierpinski and Riesel. I discovered two mistakes on the webpages, but no new primes. I have notified Gary of the mistakes and they have been fixed.

I also double-checked remaining k on S63 to n=1,000. No new primes.

I have decided to take on the challenge of retesting the other bases >= 100. This is where being a programmer saves one a lot of time.

I wrote a program to parse the Riesel/Sierpinski pages to pull out the remaining k and search limits for each base. That program then created an ABC2 file for each conjecture and then ran PFGW to test up to n=1000. After that test was done, I modified that program to run srsieve up to 1e9 for each conjecture. I then merged the output into a single ABC file using the $a*$b^$c$d format. That took about two days, finishing last night. Post sieving I have 3.2 million and 3.4 million candidates to check out. Instead of using PFGW 3.3.5/3.3.6 to determine which numbers needed to be retested, I modified gwnum so that I could tell it to use the 3.3.5 method or 3.3.6 method to get the FFT size. I think wrote another program that read the ABC file and links with the modified gwnum to get both FFT sizes. If they differ, then that line gets kicked out to another file. That file will then be dumped into PRPNet.

I know that some people have double-checked bases between 100 and 800, so I will need to remove those tests from my final files before dumping them into PRPNet. I will know some time this weekend how many retests will be needed.

Now one should ask: Does Mark ever sleep? lol

Nice work Mark. :smile:

Although I won't start testing until next week, I do have some stats. On the Riesel side I have 81,761 retests. For Sierpinski, I have 53246. I have three conjectures that account for about 13,000 of the rests (about 10%). S200 is probably the worst as I have 2229 tests, all for 50,000 < n < 500,000.

I completed the double-check for all b >= 800. No primes to report. I have about 120,000 tests left for b >= 100. I will forego tests for S200, n > 25000. There are around 1300 tests in that range to be redone. None of the other bases have a retest for n > 120,000. I have no idea how long it will take, but I can say that I only have about 900 k/b/c combinations left, which is less than half of what I started with as I knocked off k/b/c with < 20 retests before tackling the bigger ones. If anyone wants to take S200, please let me know.

Presuming I don't find a missed prime, I would recommend suspending the double-check. I still think that it is possible that a prime was missed, but not feeling confident that I will find one. Not only am I double-checking CRUS work, I am also double-checking the Generalized Woodall search for n < 2000. I have found dozens of missed primes, which is really surprising to me. These were missed by PFGW 1.x, not PFGW 3.x. I also double-checked Generalized Cullens, but have not found any missing primes. I think that the older primes found by those projects might have been found by Proth, but I really don't know. The Generalized Cullen search is limited to b < 200. Only one of the missed primes for GW was for b < 200, so it is likely that the GC search didn't miss anything.

I have completed a double-check (a retest is probably more accurate) for all b >= 100 with the exception of S200, being done by CRGreatHouse. No primes were missed. I will not do anything for b < 100. If CRGreatHouse wants to abandon the retest of S200, he can do so. I don't expect to find any missed primes for anything not done.

Now back to S63...