PFGW 3.6.x problems on small primes
 

[QUOTE=Lennart;377829]taking s66 to 50k

Lennart[/QUOTE]

Lennart,

Can you hold off on doing this? We have found many problems in the k's remaining on S66 and S79. I am trying to think of the best way to tackle the problem. Sorry.

[QUOTE=gd_barnes;377851]Lennart,

Can you hold off on doing this? We have found many problems in the k's remaining on S66 and S79. I am trying to think of the best way to tackle the problem. Sorry.[/QUOTE]


I am only sieving at this time.. I will not start LLR.

Are you redoing them from scratch ?

Lennart

[QUOTE=Lennart;377855]I am only sieving at this time.. I will not start LLR.

Are you redoing them from scratch ?

Lennart[/QUOTE]

I have not done anything with them yet but that is likely what we'll have to do. A bad version of PFGW, version 3.6.0, was used to script the bases. Many incorrect and missing small primes were found, around n=2, 3, 4, and 5; 100s of errors. It is a huge loss of work but I think it would be too risky to do anything else at this point.

This was pointed out to me by Phillip (MagentaSunrise) after he searched S79 for n=25K-50K and subsequently doublechecked all of the k's remaining to n=100. I think we'll also need to eventually look at many other large-conjectured bases, especially bases started since PFGW 3.6.0 was released. It's part of the reason that I doublechecked Riesel base 3 for k<1G to n=100K.

[QUOTE=gd_barnes;377860]I have not done anything with them yet but that is likely what we'll have to do. A bad version of PFGW, version 3.6.0, was used to script the bases. Many incorrect and missing small primes were found, around n=2, 3, 4, and 5; 100s of errors. It is a huge loss of work but I think it would be too risky to do anything else at this point.

This was pointed out to me by Phillip (MagentaSunrise) after he searched S79 for n=25K-50K and subsequently doublechecked all of the k's remaining to n=100. I think we'll also need to eventually look at many other large-conjectured bases, especially bases started since PFGW 3.6.0 was released. It's part of the reason that I doublechecked Riesel base 3 for k<1G to n=100K.[/QUOTE]

Do you know the scope of the problem? In other words do you know how many bases were impacted?

[QUOTE=rogue;377880]Do you know the scope of the problem? In other words do you know how many bases were impacted?[/QUOTE]

No. I haven't looked at other bases yet but I did catch some serious errors in a base 7 submission from Ian at one point and the problem was missing (or sometimes additional) teeny primes. There were clearly too many k's remaining in one of his searches. What I was thinking of doing to start with is looking at bases with more than ~500 k's remaining and doublechecking them to n=1000. I don't know which versions of PFGW were bad but I suspect that it may be limited to 3.6.0. I have "baseline" versions of PFGW (3.3.6 & 3.4.1) that I'm very confident are correct across all prime sizes, which I use for doublechecking.

I suppose the best thing to do would be to get a copy of every version of PFGW since 3.3.6 and run each of them on the script for a few 1000 k's on S66/S79 to n=1000 to see if there are any other erroneous versions so that people get rid of or update their bad versions. (I know that x.x.0 versions are a little more susceptible to problems since x.x.1-x.x.9 versions usually correct any bugs in extensive updates to the program.) I suspect that it may be related to updates in the GWNUM libraries. I know that George does extensive testing of those when doing LLR updates but testing on very small primes may not always be done.

This is such a time-consuming effort to either do or coordinate that I haven't wanted to take the time to do either. The error was first pointed out to me about 2-3 months ago.

[QUOTE=gd_barnes;377898]No. I haven't looked at other bases yet but I did catch some serious errors in a base 7 submission from Ian at one point and the problem was missing (or sometimes additional) teeny primes. There were clearly too many k's remaining in one of his searches. What I was thinking of doing to start with is looking at bases with more than ~500 k's remaining and doublechecking them to n=1000. I don't know which versions of PFGW were bad but I suspect that it may be limited to 3.6.0. I have "baseline" versions of PFGW (3.3.6 & 3.4.1) that I'm very confident are correct across all prime sizes, which I use for doublechecking.

I suppose the best thing to do would be to get a copy of every version of PFGW since 3.3.6 and run each of them on the script for a few 1000 k's on S66/S79 to n=1000 to see if there are any other erroneous versions so that people get rid of or update their bad versions. (I know that x.x.0 versions are a little more susceptible to problems since x.x.1-x.x.9 versions usually correct any bugs in extensive updates to the program.) I suspect that it may be related to updates in the GWNUM libraries. I know that George does extensive testing of those when doing LLR updates but testing on very small primes may not always be done.

This is such a time-consuming effort to either do or coordinate that I haven't wanted to take the time to do either. The error was first pointed out to me about 2-3 months ago.[/QUOTE]

In looking at the release notes, I would guess that versions up to 3.6.3 had the issue and that it was fixed in 3.6.4.

Did this result in erroneously reported primes, missing primes, or both? From what you've written it appears that this only affects numbers with small n. Is that correct? If so do you know the upper bound of n that was affected?

[QUOTE=rogue;377910]In looking at the release notes, I would guess that versions up to 3.6.3 had the issue and that it was fixed in 3.6.4.

Did this result in erroneously reported primes, missing primes, or both? From what you've written it appears that this only affects numbers with small n. Is that correct? If so do you know the upper bound of n that was affected?[/QUOTE]

It did both erroneous primes and missing primes, which is why it would be very risky not to start from scratch on these bases. Initially I thought it was just missing primes, which would mean it would be easy enough to remove them from k's remaining on the pages, but unfortunately the problem wasn't that simple. From what I can tell it only affected primes for n<10 but I can't be sure enough of it to say for sure. Most of the bad or missing primes were n=2,3,4,or5.

Probably anytime there is a change in GWNUM libraries and before releasing to the public, there should be a test run of several 100 or 1000 small primes (n<100 on many varying bases) to make sure nothing was affected at the small end of things.

I re-ran a good chunk of those two bases up to n=1000, although it's such a mess I haven't made an attempt to change the CRUS pages. I think I can provide both the "bad" and "good" files of primes for you if you'd like.

[QUOTE=gd_barnes;377921]It did both erroneous primes and missing primes, which is why it would be very risky not to start from scratch on these bases. Initially I thought it was just missing primes, which would mean it would be easy enough to remove them from k's remaining on the pages, but unfortunately the problem wasn't that simple. From what I can tell it only affected primes for n<10 but I can't be sure enough of it to say for sure. Most of the bad or missing primes were n=2,3,4,or5.

Probably anytime there is a change in GWNUM libraries and before releasing to the public, there should be a test run of several 100 or 1000 small primes (n<100 on many varying bases) to make sure nothing was affected at the small end of things.

I re-ran a good chunk of those two bases up to n=1000, although it's such a mess I haven't made an attempt to change the CRUS pages. I think I can provide both the "bad" and "good" files of primes for you if you'd like.[/QUOTE]

I would like to know what the upper bound for n is. You stated that most are for n <= 5, yet you tested to n=1000. Maybe you can narrow down the search range for any double-checking effort.

[QUOTE=rogue;377924]I would like to know what the upper bound for n is. You stated that most are for n <= 5, yet you tested to n=1000. Maybe you can narrow down the search range for any double-checking effort.[/QUOTE]

To the best of my knowledge in my doublecheck, all of the problems were for n<=5 (maybe n<=6) but I doublechecked all n<=1000. But I can't say for sure that all problems were for n<=6 because I didn't do an automated line-by-line comparison. I just visually compared the two.

The comparison is a bit tricky because sometimes it would miss a prime for n=4 and then later find a prime for the same k for, say, n=500. Other times it would not find a prime at all for that k. Other times for another k it would have a "prime" for n=4 when in fact the "prime" was composite.

It's a difficult task to sort out all of the scenarios, which is why I don't see any alternative but to start the bases completely over.

Edit: Perhaps I can send you both the good and bad files and you can come up with an automated comparison so that we're more confident in what we're dealing with.

[QUOTE=gd_barnes;377925]It's a difficult task to sort out all of the scenarios, which is why I don't see any alternative but to start the bases completely over.[/QUOTE]

I'm in support of this idea, since too much work has to be done in order to sort out wich primes is bad primes, wich primes were missed and wich k's that were legitimate as remaining.

However, before starting from scratch, I feel like we have to do following:

1. choose a base <1030
2. choose a conjecture <1000
3. find as many as possible versions of PFGW (both 32- and 64-bit)
4. test the choosen conjecture in each available version of PFGW 32 bit*
5. test the choosen conjecture in each available version of PFGW 64 bit*

* using the starting new bases script, going from n=1 to n=25000.

6. compare the different resultfiles to each other, to see wich files is simmilar and wich is different, thereby we can see wich versions of PFGW that is at least working similarly
7. create a k-list with all k's <conjectured k
8. sieve the range of k's from n=1 to n=25000, using srsieve and sr2sieve
9. LLR test the remaining candidates, using LLR 3.8.9 and StopOnPrimedK=1
10. Compare the primelist made by LLR with the primelist made by PFGW
11. compare/verify the mob list made by PFGW (manually)
12. compare/verify the trivial factors list made by PFGW (manually)
13. by finish of compare, make an official zip package, with a verified safe version of PFGW wich has to be used when starting new bases
14. start a drive to start all bases one-by-one from n=1 to n=25K (maybe the good folks at Primegrid can be of help here, since it could be a great PSA project)**

** This would mean, that everything is correct and doublechecked, such that future searchers should not be concerned about mallock material. Also it will make sure that no search ranges for the relative low bases with a very big conjecture is left unsearched for months or years at a time.

Maybe if a drive is setup, we should have 2 users search the same range of k's. Even though it is no easy task to compare millions of lines, I think I'll be able to offer a helping hand if you need it.

On a sad note, I just noticed that my R3 13G-14G range was searched using PFGW 32-bit version 3.6.3 ... so I guess that will make the range suspiscious...

Take care

Kenneth

Sounds great if someone has unlimited personal time to coordinate such an effort. I think this is something that we'll just have to do piece-meal as we have the time and inclination to do so. At some point, I think I'll need to add a doublecheck column to the CRUS pages so that we know what has been done.

As for R3 k=13G-14G, can you provide me with a small list of primes, perhaps k=13G-13.01G to n=10K or 25K using PFGW 3.6.3? I'll then run PFGW 3.3.6 or 3.4.1 against it and we'll compare the two.

[QUOTE=gd_barnes;377925]To the best of my knowledge in my doublecheck, all of the problems were for n<=5 (maybe n<=6) but I doublechecked all n<=1000. But I can't say for sure that all problems were for n<=6 because I didn't do an automated line-by-line comparison. I just visually compared the two.

The comparison is a bit tricky because sometimes it would miss a prime for n=4 and then later find a prime for the same k for, say, n=500. Other times it would not find a prime at all for that k. Other times for another k it would have a "prime" for n=4 when in fact the "prime" was composite.

It's a difficult task to sort out all of the scenarios, which is why I don't see any alternative but to start the bases completely over.

Edit: Perhaps I can send you both the good and bad files and you can come up with an automated comparison so that we're more confident in what we're dealing with.[/QUOTE]

If you e-mail me the good and bad files I can compare to see which n are impacted. I would need to know which version of pfgw those were run with. That will help me narrow get the upper limit of the size of the numbers that are impacted. I would then run with 3.7.7 and compare to the pre-3.6.0 output you have. If they don't match then more investigation is needed.

Do we know which bases are impacted or do you not trust any of them?

[QUOTE=rogue;377960]Do we know which bases are impacted or do you not trust any of them?[/QUOTE]

No I don't know which bases are impacted outside of S66/S79. R51/S51/S66/R79/S79 and some ranges of R3/S3/R7/S7/R15/S15 likely have the greatest chance of being impacted. I suppose the best thing to do would be to look at bases that were started around the time that PFGW 3.6.x was released.

I will send you "good" and "bad" files for S66 & S79. There is a tricky issue on comparing them: My doublecheck is for n<=1000. The original (bad) files are for n<=2500. Here is how I think a comparison would need to be done:

good file scenario / bad file scenario
1. has prime / has same prime : OK
2. has prime / has different prime : bad
3. has prime / has no prime : bad
4. has no prime / has prime n>1000 : OK
5. has no prime / has prime n<=1000: bad
6. has no prime / has no prime : OK

So in evaluating "true" differences, you're looking for 2 situations:
1. The good file has a prime but the bad file has a different prime or no prime at all.
2. The good file has no prime but the bad file has a prime for n<=1000.

The files for S66 are huge since the conjecture is k>20M. I'll attempt to send the whole files in gmail but if it won't take due to size, I'll cut the S66 files down to an appropriate size. There are so many differences that even a relatively small sample of primes should be enough for you to get an idea of the problems.

The "bad" files that I will send you were run with PFGW 3.6.0. The "good" (doublecheck) files that I will send you were run with PFGW 3.3.6.

I have to correct myself. The range tested by PFGW 3.6.3 is R3 11.0-12.0G and by tomorrow I'll have 11.00G to 11.01G send to you. The range will contain the primes for n<=25K.

[QUOTE=gd_barnes;377962]No I don't know which bases are impacted outside of S66/S79. R51/S51/S66/R79/S79 and some ranges of R3/S3/R7/S7/R15/S15 likely have the greatest chance of being impacted. I suppose the best thing to do would be to look at bases that were started around the time that PFGW 3.6.x was released.

I will send you "good" and "bad" files for S66 & S79. There is a tricky issue on comparing them: My doublecheck is for n<=1000. The original (bad) files are for n<=2500. Here is how I think a comparison would need to be done:

good file scenario / bad file scenario
1. has prime / has same prime : OK
2. has prime / has different prime : bad
3. has prime / has no prime : bad
4. has no prime / has prime n>1000 : OK
5. has no prime / has prime n<=1000: bad
6. has no prime / has no prime : OK

So in evaluating "true" differences, you're looking for 2 situations:
1. The good file has a prime but the bad file has a different prime or no prime at all.
2. The good file has no prime but the bad file has a prime for n<=1000.

The files for S66 are huge since the conjecture is k>20M. I'll attempt to send the whole files in gmail but if it won't take due to size, I'll cut the S66 files down to an appropriate size. There are so many differences that even a relatively small sample of primes should be enough for you to get an idea of the problems.

The "bad" files that I will send you were run with PFGW 3.6.0. The "good" (doublecheck) files that I will send you were run with PFGW 3.3.6.[/QUOTE]

OK. BTW, use 7-zip to compress. It will create smaller files than a normal zip.

Nevermind. I got them.

I have done a comparison of the results for these two bases and sent that information to Gary. He might want me to run the results for more bases, but I think in the end he will concur with what I've presented to him.

The problem only appears for numbers up to about 40 bits in size. pfgw 3.6.0 reports numbers as prime that are not prime, but it also misses primes. For these two bases the largest n with a bad or missing prime was 5.

I did not run those bases with pfgw 3.7.7. That needs to be done to verify correctness of the current version of pfgw.

This is what I would recommend (once we know that pfgw is working correctly):

1) Determine which bases are suspicious, i.e. those started after pfgw 3.6.0 was released.
2) Run those bases to n=100 (no reason at this time to go further).
3) Extract the primes where n <= 100 from the original submitted results.
4) If they are a match, done.
5) If there is a discrepancy, then use pl_remain.txt to do the following:
a) Remove k that have a prime for n > 100 from known results.
b) Remove known k that do not have a prime (as they have already been tested).
c) Run the remaining k to n=25000.
d) Run the remaining k to the conjecture's search limit.
e) Any remaining k get added back as needed to prove the conjecture.

I could write a program to make this easier. I would need the following:

1) A good version of pl_prime.txt (from step 2 above)
2) A good version of pl_remain.txt (from step 2 above)
3) A list of known primes, regardless of pfgw version. The program would ignore primes for n <= 100.
4) The current known list of k needed to prove the conjecture (although I could technically scrape that from the web pages).

I should be able to output a good list of primes (in k sequence) and a list of k needing further testing.

I concur with everything you said. That would be outstanding if we didn't have to start from scratch on the problem bases. When all is said and done, what I definitely need is one nice big file of "good" primes sorted by k, at least for primes n<2500. It looks like you can do that so that covers everything as far as I'm concerned. Considering the huge # of k's on these large-conjectured bases, I don't need more bases run to be convinced that the problem is only for very small primes.

Here is what I would suggest to start with: Write your program to do what you suggested and use it on the S66 and S79 files that I already sent you. I will also send you the rest of the primes for n=2.5K to 25K (S66) or 50K (S79) as well as the needed k's remaining files. The filenames with "good" in them are definitely good. The only difference vs. what you are suggesting that we do is that the "good" file has been searched to n=1000 vs. n=100. That shouldn't matter since the problem is for primes <= 40 bits.

[QUOTE=gd_barnes;377982]I concur with everything you said. That would be outstanding if we didn't have to start from scratch on the problem bases. When all is said and done, what I definitely need is one nice big file of "good" primes sorted by k, at least for primes n<2500. It looks like you can do that so that covers everything as far as I'm concerned. Considering the huge # of k's on these large-conjectured bases, I don't need more bases run to be convinced that the problem is only for very small primes.

Here is what I would suggest to start with: Write your program to do what you suggested and use it on the S66 and S79 files that I already sent you. I will also send you the rest of the primes for n=2.5K to 25K (S66) or 50K (S79) as well as the needed k's remaining files. The filenames with "good" in them are definitely good. The only difference vs. what you are suggesting that we do is that the "good" file has been searched to n=1000 vs. n=100. That shouldn't matter since the problem is for primes <= 40 bits.[/QUOTE]

Correct about 100 vs 1000. I was referring to any other bases that have problems. This way the retesting of those ranges will take far less time.

Slightly off this discussion, but I have a small suggestion.

Would it be possible to get the new bases script to print the version of PFGW used in all of the output files it produces? Then if we have a similar problem to this in the future it will be a much simpler case of just checking the output files to see which version they were produced with (assuming we can get people to use the latest version of the script!).