I've upgraded all of my computers to version 3.3.6. My quad's current batch of work will be done in a week or so; after that I'll be able to put it to work doing re-testing where needed. Note that I have limited access to my quad and all work must be fed to it via PRPnet, though that shouldn't be too big a deal since most of the work will be above n=1000 and thus somewhat reasonably PRPnet-able.

[QUOTE=KEP;229303]Rogue, does this have anything to do with the constant switch between "All-complex fft lengths" and "zero padded fft lengths"? Also does it always constitute a problem, if the k*b^n+c test has been tested using an All-complex fft length, wich is usually 40 % faster? There has been especially many all complex fft length tests under the s58 conjecture.

Now I must admit, that I'm very very close to cancelling all my reservations and discard all work as "bogus". There is also no way that I can double check any of my S58, S60 and S70 results, since it will mean that I will first complete these 2 reservations + 1 re-reservation, some time in 2015. So do you have any stats on how many primes I can expect to have missed, below my current test depth of n=65K for S58?

I'll exam further explanations and come to a final conclusion, on weather or not to cancel my CRUS work, as new information sees the light of the day...

I'm not mad, to say it for once, but I for one, am not going to spend several CPU years on producing mallock work and doublechecking previous work. I will however see if I can find version 3.3.6 somewhere and upgrade immediately.[/QUOTE]

I cannot answer your first set of questions. If George is following this thread, maybe he can chime in.

I don't consider all work to be bogus. You only need to look at the k for which no prime has been found. I would estimate that somewhere between 3% and 10% of PRP tests will use a different FFT size when run with 3.3.6 vs. 3.3.5 and earlier. Even though a different FFT size will be chosen, most of the residues will match between the two versions of PFGW.

If you want me to, I can double-check whatever work you did before you upgraded to PFGW 3.3.6. All you need to do is e-mail me the remaining k and the limits (upper bound of n) to which you tested with older versions. If you have sieve files or a list of candidates tested, that would save me some time. With such low bases I think that double-checking should take no more than 1 week.

I have no idea how to compute how many primes might have been missed.

To get a feel for what's involved in checking a particular base for errors, I did a "dry run" on Riesel base 6, which only has 2 k's remaining and thus is a rather easy one on which to do this. I've outlined the procedure here so others can base their efforts on it.

[B]Stage 1: re-test up to n=1K[/B]
For a tiny base like this, this part went really fast. I first made a file called r6.abc2 (you can name it whatever you want, just don't put spaces in it since that will make it a little hard to plug into PFGW later) with the following contents:
[code]ABC2 $a*6^$b-1 // {number_primes,$a,1}
a: in { 1597 36772 }
b: from 1 to 1000[/code]
I then ran it through PFGW 3.3.6 with the command line:
[I]pfgw -f30 r6.abc2[/I]
It finished within seconds; no primes were found. If there had been any (indicating something missed the first time around), they would have been output to pfgw.log. In that case, I would need to verify the PRPs as usual with the -tp switch and post them in the forum so Gary can remove the appropriate k's from the web pages.

[B]Stage 2: sieve n=1K-25K and check for FFT differences[/B]
Because the base is so low, I didn't go up to the full p=10G that Mark recommended. However, for most bases, you will want to sieve that high. I used the command line:
[I]srsieve -n 1000 -N 25000 -P 1e10 -G 1597*6^n-1 36772*6^n-1[/I]
For multiple k's, you can string them up on the command line as I did, or if there's a ton of them, make a sequences file--a basic text file containing one sequence per line in the same format I used--and give its name to srsieve in place of the sequences. (Note that in this case, I manually stopped the sieve with Ctrl-C a little past p=1G.)

I now had a sieve file, t17_b6.prp. (For other bases, the b* part will change accordingly and it will be t16 for Sierp., t17 for Riesel.) Now I ran that through both PFGW 3.3.5 and 3.3.6 with the -F switch; in both cases the command line was:
[I]pfgw -F -l t17_b6.prp[/I]
Each produces a file pfgw.out with each line describing the FFT it would use on a particular number. After running this through v3.3.5, I renamed pfgw.out to pfgw335.out and re-ran with 3.3.6.

Now I ran a diff on the two files to see where they differed. I have Cygwin installed, so I can use this command locally; Linux users can do similarly. If you're on Windows without Cygwin, you can get diff.exe and grep.exe from [URL]http://unxutils.sourceforge.net/[/URL] which behaves exactly like the Unix one I used. I used the command:
[I]diff pfgw335.out pfgw.out | grep ">" >> diff.txt[/I]
Now diff.txt contains just the lines representing numbers that need to be retested. In this case, I got an empty file, i.e. no numbers were bad. All clear in the 1K-25K range--next! :smile:

[B]Stage 3: check residuals on file for 25K+[/B]
First, email Gary and ask him for all the n=25K+ residuals he has on file for the base you're doing. Obviously, you won't get a reply [I]immediately[/I], so for the purpose of this example I pulled a random results file out of the R6 drive thread and used it for the comparison.

The file that I grabbed was named pfgw.out to begin with, so to avoid confusion with the output files I'll be getting later, I renamed that to orig.out.

I then coded up a quick Perl script (attached here) which can be used to strip the residues from the results file, leaving just the number itself. In this case, I used the following command line:
[I]perl results-to-input.pl orig.out orig.in[/I]
which made a file orig.in containing just the numbers from orig.out.

Now, I repeate the process from stage 2 for checking which numbers PFGW v3.3.6 uses a different FFT size. That is, run both versions with the -F option, and use diff to compare them. The format output by my Perl script, namely that of one candidate per line in raw format, is accepted directly by PFGW, so you can use it just like the sieve file in stage 2.

In this case, as before, diff.txt was empty, indicating that this range is all clear. If any numbers had needed to be retested, it would have shown something like this (just an example, edited manually for the purposes of demonstration; neither version of PFGW actually wanted to use the 32K FFT on this number):
[code]
> Special modular reduction using FFT length 40K on 36772*6^150007-1
[/code]
Thus, you need to add just the number "36772*6^150007-1" to a new file on a line by itself. Repeat for any more such numbers found in diff.txt. I don't yet have a script for doing this but if we find that some bases produce enough potentially bad tests to make it a pain to do this by hand, I'll see what I can come up with. Edit: done. The script is called fft-to-input.pl and the syntax is just like results-to-input.pl.

That's all there is to it. Oh, and BTW: in the course of this "dry run", I [B]re-checked R6 up to n=25K[/B]. No bad results were found (as expected due to the low b value).

Max :smile:

[QUOTE=mdettweiler;229317]
In this case, as before, diff.txt was empty, indicating that this range is all clear. If any numbers had needed to be retested, it would have shown something like this (just an example, edited manually for the purposes of demonstration; neither version of PFGW actually wanted to use the 32K FFT on this number):
[code]1c1
< Special modular reduction using FFT length 32K on 36772*6^150007-1
---
> Special modular reduction using FFT length 40K on 36772*6^150007-1
[/code]
Thus, you need to add just the number "36772*6^150007-1" to a new file on a line by itself. Repeat for any more such numbers found in diff.txt. I don't yet have a script for doing this but if we find that some bases produce enough potentially bad tests to make it a pain to do this by hand, I'll see what I can come up with.[/QUOTE]

You can use a text editor (like TextPad) with the output from diff and strip the characters before the number on each line, then feed that through PFGW, with one candidate on each line. A perl script could probably do that even easier.

[quote=rogue;229318]You can use a text editor (like TextPad) with the output from diff and strip the characters before the number on each line, then feed that through PFGW, with one candidate on each line. A perl script could probably do that even easier.[/quote]
Yes, that would be pretty simple. What I was thinking might be a little trickier, though, is to ensure that the script only outputs each candidate to the file once--not twice as diff does it.

Is there a flag for diff that can tell it to only output one instance of each differing line?

Bases to be checked
 

Here are the Riesel bases to be checked:

3, 5, 6, 7, 10, 15, 19, 22, 23, 24, 25, 26, 27, 28, 30, 31, 35, 36, 37, 39, 40, 42, 45, 46, 48, 49, 51, 52, 53, 55, 58, 60, 61, 67, 70, 72, 75, 78, 79, 80, 82, 87, 88, 91, 93, 94, 95, 96, 97, 100, 102, 103, 105, 107, 108, 109, 111, 112, 115, 117, 121, 123, 133, 135, 136, 138, 143, 147, 148, 152, 157, 158, 159, 160, 161, 162, 163, 168, 170, 172, 173, 177, 181, 182, 186, 187, 191, 192, 198, 199, 200, 211, 212, 213, 214, 217, 218, 221, 231, 233, 234, 235, 236, 241, 243, 249, 253, 258, 268, 272, 273, 275, 277, 281, 287, 288, 290, 298, 304, 308, 315, 318, 319, 320, 321, 326, 327, 328, 332, 333, 334, 337, 338, 343, 347, 353, 354, 361, 362, 363, 367, 368, 373, 380, 383, 392, 394, 398, 401, 402, 406, 409, 410, 412, 416, 417, 422, 423, 424, 425, 433, 439, 444, 446, 447, 452, 458, 463, 468, 469, 470, 471, 480, 488, 492, 493, 496, 497, 499, 500, 504, 505, 510, 514, 516, 520, 523, 526, 530, 533, 534, 543, 548, 549, 550, 551, 552, 563, 564, 566, 572, 573, 579, 580, 581, 582, 588, 589, 590, 601, 602, 603, 610, 611, 615, 617, 619, 620, 622, 625, 633, 634, 635, 636, 638, 639, 641, 650, 653, 654, 662, 665, 666, 667, 668, 677, 678, 679, 684, 686, 688, 694, 695, 696, 702, 703, 709, 710, 716, 720, 724, 727, 729, 730, 731, 736, 737, 741, 743, 744, 746, 747, 752, 753, 754, 759, 761, 773, 774, 782, 783, 784, 785, 789, 790, 798, 800, 803, 812, 814, 815, 821, 828, 832, 833, 836, 841, 845, 850, 859, 864, 866, 867, 868, 872, 875, 879, 883, 887, 888, 889, 892, 894, 900, 904, 905, 908, 916, 920, 926, 928, 931, 932, 941, 945, 951, 954, 962, 964, 967, 968, 980, 984, 987, 988, 994, 995, 998, 999, 1001, 1003, 1009, 1010, 1017, 1019, 1021, 1025, 1029


Here are the Sierpinski bases to be checked:

3, 5, 6, 7, 9, 10, 12, 15, 17, 19, 22, 24, 25, 26, 27, 28, 30, 31, 33, 35, 37, 39, 40, 42, 43, 45, 46, 48, 49, 51, 52, 53, 55, 58, 60, 61, 63, 67, 68, 70, 72, 73, 75, 78, 80, 81, 82, 86, 87, 88, 91, 93, 95, 97, 100, 101, 102, 103, 105, 107, 108, 111, 112, 115, 117, 118, 122, 123, 133, 135, 136, 138, 140, 143, 145, 147, 148, 151, 155, 157, 161, 162, 163, 165, 168, 171, 173, 174, 177, 178, 182, 183, 185, 187, 189, 191, 192, 193, 198, 199, 200, 204, 207, 208, 211, 212, 213, 214, 217, 218, 220, 227, 228, 230, 236, 248, 249, 252, 255, 257, 259, 263, 264, 273, 288, 289, 290, 294, 298, 304, 308, 311, 315, 317, 318, 319, 320, 326, 328, 332, 334, 335, 338, 340, 341, 342, 343, 353, 354, 365, 368, 373, 379, 380, 383, 386, 392, 394, 395, 397, 401, 402, 405, 406, 409, 410, 412, 414, 416, 417, 422, 425, 426, 428, 436, 444, 450, 458, 461, 463, 467, 468, 470, 472, 476, 480, 482, 492, 496, 497, 500, 501, 510, 516, 518, 520, 529, 530, 533, 542, 550, 567, 573, 574, 577, 578, 579, 582, 588, 593, 596, 605, 617, 620, 622, 626, 638, 643, 647, 649, 653, 676, 683, 688, 695, 696, 701, 702, 703, 706, 707, 712, 724, 731, 736, 737, 740, 743, 758, 761, 766, 767, 773, 774, 778, 780, 781, 784, 785, 788, 789, 790, 797, 798, 800, 802, 803, 805, 811, 814, 816, 821, 825, 828, 832, 833, 834, 835, 836, 844, 845, 850, 853, 859, 866, 867, 872, 875, 878, 879, 883, 887, 889, 893, 905, 908, 914, 917, 920, 928, 930, 932, 934, 935, 939, 945, 947, 949, 953, 961, 964, 968, 980, 983, 985, 987, 993, 995, 998, 999, 1001, 1002, 1004, 1006, 1010, 1013, 1016, 1026, 1029

Bases 2, 4, 8, etc. are excluded since George implied that base 2 was not affected.

I removed bases that I have double-checked already on the Sierpinski side.

If someone can sticky this thread and move this list to the first post, that would be great.

[QUOTE=mdettweiler;229319]Is there a flag for diff that can tell it to only output one instance of each differing line?[/QUOTE]

I'm not aware of anything, but you can use grep to only include lines with '<' or '>', for example:

diff pfgw_3.3.5.out pfgw_3.3.6.out | grep ">"

[quote=rogue;229321]I'm not aware of anything, but you can use grep to only include lines with '<' or '>', for example:

diff pfgw_3.3.5.out pfgw_3.3.6.out | grep ">"[/quote]
Ah, perfect--that should do the trick quite nicely. I'll change the above directions accordingly.

[QUOTE=rogue;229320]Quote containing multiple of bases, that I have for the concern of space, removed from the quoted text :)[/QUOTE]

I can see that all my reservations after the crash of my Quad, is affected. However, thanks for your offer on DC Rogue. I have written a lot to you in a PM, and as mentioned in the PM, I'm going to continue as if nothing has happened, after an upgrade to version 3.3.6! However, I'll do some brief DC of S383 as I come home in a short while, simply to see if a k has actually been missed for n<=1K. After that I might do a DC to n=25K, even though the Primepattern seems credibel, it is better to be safe than sorry.

Regards

Kenneth

[quote=rogue;229289]3) For n > 25000, ... This should be a fairly low percentage (most likely under 3%).[/quote]

That's where almost all my reservations belong. I've now completely checked FFT sizes 3.3.5 vs 3.3.6 for all my reservations past and current.

n>25K, 16 bases (42<b<930), ~75 k's (2<k<12,000), lowest FFT size 12K, ~120,000 tests. 0 tests with differences in FFT sizes.

Also 10K<n<30K, Sierp 35, ~800 k's (46<k<215,000), lowest FFT size 4K, ~410,000 tests. 0 tests with differences in FFT sizes.

Most of my testing was done using LLR's N-1/N+1. I recall a post by J.[COLOR=Black]Penné[/COLOR] saying LLR lets FFT choice be handled by gwnum. If the same is true for PFGW then both programs should do the same test with the same FFT length (?) (if someone knows this to be wrong say so) and so all results I have submitted so far should be ok (i.e. done on the same FFT size that 3.3.6 would have chosen).

Was the 3% an estimate based on differences at much lower n levels? Based on my numbers it looks like (for n>25K) it will be much lower than that. We should know when others report some more stats, but if double-checking will be necessary for only small n's then the overall work should be reasonable.

[QUOTE=vmod;229335]That's where almost all my reservations belong. I've now completely checked FFT sizes 3.3.5 vs 3.3.6 for all my reservations past and current.

n>25K, 16 bases (42<b<930), ~75 k's (2<k<12,000), lowest FFT size 12K, ~120,000 tests. 0 tests with differences in FFT sizes.

Also 10K<n<30K, Sierp 35, ~800 k's (46<k<215,000), lowest FFT size 4K, ~410,000 tests. 0 tests with differences in FFT sizes.

Most of my testing was done using LLR's N-1/N+1. I recall a post by J.[COLOR=Black]Penné[/COLOR] saying LLR lets FFT choice be handled by gwnum. If the same is true for PFGW then both programs should do the same test with the same FFT length (?) (if someone knows this to be wrong say so) and so all results I have submitted so far should be ok (i.e. done on the same FFT size that 3.3.6 would have chosen).

Was the 3% an estimate based on differences at much lower n levels? Based on my numbers it looks like (for n>25K) it will be much lower than that. We should know when others report some more stats, but if double-checking will be necessary for only small n's then the overall work should be reasonable.[/QUOTE]

I'm somewhat surprised to learn that none of your tests would use different FFT lengths. Higher n have fewer problems than lower n. Could you please list the bases referenced above (where you say you have 16 bases)? Someone still needs to look at n < 25000 for those k. It should less than a day to verify S35 for n<10000 for the remaining k.

LLR and PFGW let gwnum choose the FFT size. PFGW can tell gwnum to choose a larger size using the -a switch or internally if it detects a roundoff error.

And of course if you haven't done so (and it sounds like you have), make sure all of your clients are running PFGW 3.3.6.