PFGW latest well-tested version

[gd_barnes]

Mark,

I have experienced the same thing when doing some double checking of Riesel base 3. Can I please make a suggestion? Please set up some test cases that test many known situations so that we don't keep introducing new issues whenever a previous one is fixed. That will also limit the # of releases, which will help us greatly. It will also make it to where I don't have to do a "work around" on the starting bases script. (See below.) Whenever any change is made, please don't test just what is being changed. No matter how small the change is, please run all of the test cases through it before adding another public release. I feel confident that in the long run, this will save you and all of us a lot of time.

You can have a limited # of test cases and it still wouldn't take long to test them all. You just have to make sure you have a wide variety. KEP's "composite" PRPs are a good starting point. Also recommended are: (1) Testing small bases (mainly bases 2 thru 5) for small and large composites/primes/PRPs. (2) Testing large bases > 1000 for large composites/primes/PRPs. (3) Testing rounding errors previously posted in this thread (which required use of the "former" -a1 switch) and making sure those now test correctly. (4) Test all of the above with various combinations of factoring switches such as -f0, -f10, -f30, and -f100. (5) Test problem "composite" PRP's (some that really are composite and some that are prime) with a combination of the -t, -tp, and -tc switches.

(4) is more important than you might think. The problems with "composite" PRP's (that really are prime) become more prevelant with less trial factoring. (5) is also important with the current problems that we are experiencing.

Back on what KEP was talking about, since the -a1 switch cannot be used with 3.3.0, then that is a moot point with the most recent version.

KEP, to verify whether the PRP's are prime or composite, don't use any version of PFGW. Considering using Alpertron's applet or some other software that is known bugfree for testing small primes. But I have a suggestion if you prefer using PFGW recent versions that helps a lot of the problems with it below.

This is definitely not a bug in the starting bases scripts and I have written a work-around for it with the newest version but I am still extensively testing it while I am out of town until next Thursday.

Here is what I did on the Riesel base 3 double checks while "manually testing" (i.e. without the starting bases script) using PFGW version 3.3.0: If it was a PRP and the -t or -tp switch found it to be PRP again (i.e. it just couldn't prove it) -or- if it was found to be composite, then I would use the -tc switch (a combined test). That seemed to correct ~90% of the problems. There were still a small percentage of PRPs shown composite that really were prime. I checked alptertron's applet to finally verify them. Therefore, KEP if using the latest PFGW, try the -tc switch (combined test) on your PRP's. It's slower since it does both sides but since we're talking small PRP's, it shouldn't take too much longer.

Because of these problems, like I said above, I have a modified version of the starting bases script that does the effect of the -tc switch (that is the PRIMEC script command) for proving PRP's in an attempt to much more accurately prove them. That is, if the -t or -tp switches (PRIMEM or PRIMEP script command) find the PRP to either still be PRP or to be composite, the script does not automatically throw it out as composite. It then runs a second proof test using the PRIMEC command. There are then 3 possibilities: (1) If it is composite, than it is written to the composite PRP file like the current public script does (You can then check for yourself if it really is composite.) and it continues searching the k. (2) If it is STILL PRP (Yes, that can happen if you have factoring set to something small like -f10.), I've created a new file that I'll humoriously refer to as PRP PRP's (for PRP's that can't be proven by PFGW but that it doesn't show as composite either). It will write those to the PRP PRP file. (3) If it is prime, then it writes to the regular primes file and continues normally with the next k. Jumping through these hoops made the primality proof and finding much more accurate even with the recent problems that were introduced on small tests in PFGW.

The above is the only way I know of to get around all of the differing issues with small PRPs/composites/primes since PFGW version 3.2.7. It has set my mind at ease quite a bit.

KEP, I know this is disconcerting on the conjectures but here is the good news: Even though PFGW seems to have a problem proving some "truly" prime PRPs, it is erroring on the side of caution. In other words, we are not getting false primes. We're getting false composites. Therefore although this might cause additional CPU time to be utilized to find larger primes, it won't make the conjectures "not proven" if we find primes for all of the k's. It just might take a little longer. So you need not worry about the proven conjectures actually not being proven.

The above said, one of the main pushes of the project has been to find the smallest prime for each k (both to save CPU time but also just to say we found the smallest primes possible) so it is hoped that these issues with small prime proofs be completely fixed at some point.


Thank you,
Gary

[gd_barnes]

Originally Posted by henryzz

It is possible that some of those prps are actually composite. Could you test the ones that never appear as prime with an old version of pfgw?

Why do that and continue taking such risk? Use different software such as Alpertron's applet.

Yes, they could really be composite but that's only the case for very small ones.

My experience: For base 3, the largest "truly composite" PRP that I have found had an exponent of n=~50, which equates to < 25 digits and I have run 10s of millions of k's with several dozen "truly composite" PRPs (and more recently) PRPs that were primes that were falsely found as composite.

If your PRP has > 25 digits and PFGW says it is composite when attempting a proof, there is > than a 99.99% chance that PFGW is wrong. That you can bank on! If it's < 15 digits, there is a very good chance that PFGW is correct.

My suggestion in a short form as alluded to in my long previous post:
(1) Prove with the the PFGW -tc switch first. If it shows prime, then it's definitely prime. If it shows PRP or composite, then go to (2).
(2) Try other software.
(3) Don't try different versions of PFGW. There is no guarantee what version will be correct. There have simply been too many releases to know for sure.


Gary

[rogue]

Originally Posted by KEP

Well I may look into see how old an version of WinPFGW that I has, but a fact is that all 589 k's turned out prime in the first place, using version 3.2.0, I'm sure that the explanations Rogue has provided, covers the issue regarding the PRP's being prime and then being actual composites, so I think there is no doubt anymore that the 589 k's is actual composites for those n that they are PRP.

Huh? Am I missing something in translation? Those 589 k's are most likely prime, but you need to use the -a1 switch when doing the primality test. If they are not prime with the -a1 switch, then I want to know.

[rogue]

Originally Posted by KEP

If you download the zip I attached, you can try the 61 PRPs that were found prime by version 3.2.0 but not by version 3.3.0, using -a1.

I think I am misunderstanding something here. I tested these with 3.3.0. All 589 numbers are prime when using -a1. Are you saying that you used -a1 with 3.3.0 and they were not prime?

Originally Posted by KEP

Well I may look into see how old an version of WinPFGW that I has, but a fact is that all 589 k's turned out prime in the first place, using version 3.2.0, I'm sure that the explanations Rogue has provided, covers the issue regarding the PRP's being prime and then being actual composites, so I think there is no doubt anymore that the 589 k's is actual composites for those n that they are PRP.

I'm not understanding your last sentence at all. Could you restate it?

Based upon Gary's comments, I am strongly considering a change to PFGW to automatically apply -a1 for all tests (not just those that PFGW detects a roundoff/sumout error on) until I have a new release of gwnum from George. As stated previously, this would slow down PFGW by a a couple percent, but it seems to be the safest thing to do.

The important for me is to have a solid collection of primes (such as those listed by KEP) that fail the initial primality test with 3.2.0 or 3.3.0 so that I can use them against any new release of gwnum.

Gary, could you do me a favor? Could you e-mail me a file of all primes found by CRUS? I presume you have an easy way to generate such a file. If you want to limit the size of the file, you can limit it by n or k. This is my plan:

1) Run the file through 3.2.0 with -t. Save numbers that do not come back as prime.
2) Take the remaining list from 3.2.0 and test with -a1. Investigate further if any are not prime.
3) Run the file through 3.3.1 with -t. Save numbers that do not come back as prime.
4) Take the remaining list from 3.3.1 and test with -a1. Investigate further if any are not prime.
5) Modify PFGW to apply -a1 to all tests.
6) Run file through that release with -t. If all numbers are prime, release it.

Based upon the size of the input I might ask for assistance for steps 1, 3, and 5. Certainly I could do all of the testing myself, but it would take a lot more time and would ultimately delay the release referenced by step 5.

With any new release of gwnum, I would revert the changes for step 5 and retest. Any release based upon a new release of gwnum must pass all primality tests without using -a OR PFGW must auto-detect the roundoff/sumout error and apply -a1 on the fly and retest. The problem with the current release is that PFGW is unable to auto-detect any errors because gwnum is not generating any. This means that either gwnum generates the roundoff/sumout error so that PFGW can auto-detect it or the gwnum has better logic for choosing the FFT size. Any changes to gwnum are strictly for George to make.

Thoughts?

[kar_bon]

Originally Posted by rogue

Gary, could you do me a favor? Could you e-mail me a file of all primes found by CRUS? I presume you have an easy way to generate such a file. If you want to limit the size of the file, you can limit it by n or k. (...)

For base-2 primes you can look at www.rieselprime.de (menu 'General' -> 'Downloads').
There're all primes and all twins on my pages, not the current state but enough to test base 2.

[KEP]

Originally Posted by rogue

I think I am misunderstanding something here. I tested these with 3.3.0. All 589 numbers are prime when using -a1. Are you saying that you used -a1 with 3.3.0 and they were not prime?



I'm not understanding your last sentence at all. Could you restate it?

Based upon Gary's comments, I am strongly considering a change to PFGW to automatically apply -a1 for all tests (not just those that PFGW detects a roundoff/sumout error on) until I have a new release of gwnum from George. As stated previously, this would slow down PFGW by a a couple percent, but it seems to be the safest thing to do.

The important for me is to have a solid collection of primes (such as those listed by KEP) that fail the initial primality test with 3.2.0 or 3.3.0 so that I can use them against any new release of gwnum.

Gary, could you do me a favor? Could you e-mail me a file of all primes found by CRUS? I presume you have an easy way to generate such a file. If you want to limit the size of the file, you can limit it by n or k. This is my plan:

1) Run the file through 3.2.0 with -t. Save numbers that do not come back as prime.
2) Take the remaining list from 3.2.0 and test with -a1. Investigate further if any are not prime.
3) Run the file through 3.3.1 with -t. Save numbers that do not come back as prime.
4) Take the remaining list from 3.3.1 and test with -a1. Investigate further if any are not prime.
5) Modify PFGW to apply -a1 to all tests.
6) Run file through that release with -t. If all numbers are prime, release it.

Based upon the size of the input I might ask for assistance for steps 1, 3, and 5. Certainly I could do all of the testing myself, but it would take a lot more time and would ultimately delay the release referenced by step 5.

With any new release of gwnum, I would revert the changes for step 5 and retest. Any release based upon a new release of gwnum must pass all primality tests without using -a OR PFGW must auto-detect the roundoff/sumout error and apply -a1 on the fly and retest. The problem with the current release is that PFGW is unable to auto-detect any errors because gwnum is not generating any. This means that either gwnum generates the roundoff/sumout error so that PFGW can auto-detect it or the gwnum has better logic for choosing the FFT size. Any changes to gwnum are strictly for George to make.

Thoughts?

Well I may have misunderstood what you said, becuse I somehow understood, that if they were composites in both versions, they would without a doubt be composite PRPs. I've not used the -a1 flag, and I've not tested in other ways than previously mentioned. I will go home and retest, using -a1, as long as it works in 3.3.0, which it appears to Garys long statement, that it is not possible to do. I'll let you know, if they doesn't appear as primes when using the -a1.

Regards

KEP

Edit: Using -a1 for PFGW version 3.3.0 did also make them all prime at me.
Using -tc still made them all composites. So it appears, that using -a1 should be done automatically. Now I'll just have to stop all 14 hours of work on the 589 prime k's, since they are actual primes or can't they be trusted to be prime even when using -a1?

[rogue]

Originally Posted by KEP

Well I may have misunderstood what you said, becuse I somehow understood, that if they were composites in both versions, they would without a doubt be composite PRPs. I've not used the -a1 flag, and I've not tested in other ways than previously mentioned. I will go home and retest, using -a1, as long as it works in 3.3.0, which it appears to Garys long statement, that it is not possible to do. I'll let you know, if they doesn't appear as primes when using the -a1.

Regards

KEP

Edit: Using -a1 for PFGW version 3.3.0 did also make them all prime at me.
Using -tc still made them all composites. So it appears, that using -a1 should be done automatically. Now I'll just have to stop all 14 hours of work on the 589 prime k's, since they are actual primes or can't they be trusted to be prime even when using -a1?

PFGW scripts cannot set the -a switch. I presume this is what Gary was referring to. Using -tc does both an N+1 and N-1 test. Since N+1 cannot prove primality for k*b^n+1 and N-1 cannot prove primality for k*b^n-1, I'm not concerned, unless -tc with -a1 returned composite.

If a number is PRP (with a PRP test) and prime (with or without -a1), they are prime. I do not see how it could be possible that a number is PRP (with a PRP test), prime (with a primality test), but is really composite. The PRP and primality tests use unrelated methods for their test. If both tests produce the same, but incorrect results, that would be almost impossible. If that did happen, then I would know about it almost immediately.

It is not reasonable for me to test all numbers from the Prime Pages, NPLB, and all of the other projects out there. It would take a number of years for me to test all of those numbers, even if I had a couple of quads to work on it. These problems seem to be manifesting themselves with smaller FFTs so that is where I will focus my testing.

[KEP]

@ Rogue:

Thanks for your help. I've also come to the same conclusion that the PRPs failing a -t, -tp or -tc test without the use of -a1 seems to be those with very small n's with small FFTs. Using version 3.3.0 with the following commandline:

pfgw.log -tc -a1

gave 589 prime pairs. So I think now, that I should change my routines when it comes to proving the primes, especially if I come along a PRP being composite situation like this in the future.

Just glad that it appears that all work can be trusted So now I'm continuing happily.

Finally Gary, thanks for your Idea about using -tc, however I think that if I should ever encounter an PRP being composite message/situation again, I'll most likely try using the "-t -a1" for sierpinski numbers, before going through a slower -tc test

Regards

KEP

[gd_barnes]

I have updated the 1st post here to PFGW version 3.3.4. This version includes the most recently updated GWNUM library and apparently corrects previous problems related to the proof of smaller PRPs. Many PRPs that were actually prime in the n=50 to n=125 range were showing up as composite.

This version is highly recommended if you are frequently testing very large-conjectured bases starting from scratch such as 3, 7, 15, or 63. I have found it particularly helpful in doublechecking R3, S3, and S63.


Gary

[gd_barnes]

I have updated the latest well-tested links in the 1st posting here to PFGW 3.3.6.

[gd_barnes]

I have updated the latest well-tested links in the 1st posting here to PFGW 3.4.0 & 3.4.1.