Accuracy of our work [k*2^n-1, k<300]

[Kosmaj]

A few days ago I was surprised to find the following post of July 18, 2004 by David Broadhurst on the primeform mailing list. The post lists 23 primes k*2^n-1 missing at the time from our list of primes in ranges declared as processed. I know that we are not perfect and that some mistakes could creep in but 23 primes spread over 16 k's is a bit too much. I think we should try to track down what caused these mistakes. A few things come to mind: (1) poor book-keeping of correctly found primes, (2) hardware errors due to over-heating, faulty memory etc. and (3) software errors, for example corrupting and losing result files or bugs in LLR (primes with zero residue not listed in primes.txt?). If it's (1) we have to improve book-keeping, but if it's (2) or (3) we will have to process again affected ranges.

I'll appreciate if the people who worked on affected ranges reply to this post and tell us what happened. I tried to locate people who reserved and worked on some k's but without much success. I found that SB2 worked on k=35, justinsane on k=53, and Thomas on k=95 but I'm not sure whether they worked in affected ranges of these k's. I worked on k=53 and k=77 but not in affected ranges.

Finally, whatever the reasons of the mistakes I think we deserve some credit for our work because when we started to work on k*2^n-1, k<300 many k's were badly under-processed (only to n=32k or 40k) and almost nobody worked actively on this form. Not to mention many primes that we found.

I was thinking to write privately to Joss or Slash Dude about this but I realized that we must have this explained on a public page. Becuse people who read David's post and came here to find out more couldn't find anything until now. So please when you have time kindly reply. This is no witch-hunting, just trying to locate problems so that we can avoid them in the future. Thank you and happy prime-hunting!

[SB2]

My eyes must be decieving me as I do not see k=35 listed on the post in question? However I do see k=165 which I also tested, but it is listed in my files as being prime, so my guess is it skipped over when the primes for k=165 were added to the statisics.

2, 3, 5, 6, 8, 11, 12, 15, 18, 23, 27, 36, 39, 44, 45, 56, 59, 63, 81, 84, 150, 188, 264, 275, 282, 299, 321, 338, 390, 552, 657, 722, 930, 1139, 1388, 1491, 1625, 2196, 2519, 2541, 2736, 2766, 3630, 4202, 4875, 5643, 6150, 10134, 20328, 20805, 22902, 25770, 30918, 30978, 33389, 40901, 54311, 62145, 67224, 83811, 89465, 93846, 109719, 148758, 167457, 173396,

[marc]

Also notice that most of the primes listed have very small n. Most of the work by the "(largely anonymous) LLR informants" was started higher up in the range of n.

Also from my k=103 lresults.txt:

Using IBDWT : Mersenne fftlen = 1280, Proth fftlen = 2560, Used fftlen = 2048
fftlen seems to be too small, using next fftlen...
Using IBDWT : Mersenne fftlen = 1280, Proth fftlen = 2560, Used fftlen = 2560
103*226927-1 is prime!  Time : 2.879 sec.

so this does look like there could be a problem with the lresults to stats stage rather than the unreliable anonymous informants.

[jocelynl]

I take all the blame on this. Some were missed from work on multiple computers, (I missed a range and realized it only after David told me about it). Others were from overheat when using multiple instance of LLR (Bad thing). Other times, I was mis-informed. (Can't blame me for that) I could have check there results files and there sieve file with log numbers removed (Yes I could have done that!) Other time, it was typo (can't blame me for that, the light in the office is too low and I need glasses ... )

Joss

[Kosmaj]

Thanks to everybody who replied. SB2, you are right, it's k=165 not k=35.

Then what shall we do? David checked all k's to n=80k, I think it will be the best to double-check all "suspicious" k's on David's list in the 80-200k range except those where typo's or mis-information were involved. Joss, can you tell us:

1) What k's are on the David's list due to typos and/or bad reports. Like k=103 checked by Marc and k=165 checked by SB2. Provided that all typos were corrected we can consider those as processed accurately.

2) Is there any possibility that some sieve files we got from the 15k.org site were erroneous or corrupted and if so for which k's? If necessary we'll have to sieve again those k's.

3) What k's are on the list as the result of missing ranges or over-heating? I think it will be the best to double-check those k's in the 80-200k range.

I volunteer to coordinate double-checking and will do some tests by myself.

[jocelynl]

All of the false report due to overheat were below 80k, and since David as checked them with PFGW there's no need to double check again. As for typo, the current list has been checked. For k=297 it had been missed by the searcher. My guess is that he reported his work from PrimeSearch. The missing primes have been reported to them as well. I don't know if David is checking them higher. He seemed insulted from our work from the begining and from Mark(Slashdude) first report of missing primes in Prof Keller's list. No offence was intended. I would suggest to hold on to your lresults file when doing search.
The sieve file on the site were correct, since I used the verify results when generating them. If, for example, someone started k=35 without the verify results on, he would have missed many primes.

[Kosmaj]

Joss, on the "Riesel list k<300" page you say that "At the end of the project, all primes, from this project also from Wilfrid Keller's prior effort and from recently PrimeSearch, will be verified again". For k's reported by Braodhurst as erroneous I think now is the best time to double-check them. The longer we postpone it, the less likely that we'll ever do it.

OTOH, if you are certain that double-checking is not necessary then we can as well do nothing.

BTW, there was one with a large n missing, 59*2^134240-1. Who did k=59 and who did k=297, why is this one on our page at all since you suggested that we "switch off the lights" at k=247, if I remember correctly.

Finally, I'm puzzled why Masser, who is from time to time crunching with us, asked David Broadhurst to post missing primes?? If he knew some primes were missing he could inform us beforehand or at the time he informed David.

I can report that I double-checked sieving files I got for k=53 and k=77 and I found that they are correct.

[masser]

[QUOTE=Kosmaj]

Finally, I'm puzzled why Masser, who is from time to time crunching with us, asked David Broadhurst to post missing primes?? If he knew some primes were missing he could inform us beforehand or at the time he informed David.

[QUOTE]

I was just curious to see if any of the k that I tested were among the k values listing incorrect results. Also, If I recall correctly, Broadhurst only complained that we had missed or incorrectly stated so many primes; his comments would have been of little use to us if we didn't know what mistakes we had made.

Best regards,
Masser

If I recall correctly, Broadhurst only complained that we had missed or incorrectly stated so many primes; his comments would have been of little use to us if we didn't know what mistakes we had made

Correct, and I added that alot of those had to have been LLR errors.
David should have already been aware of this, but I guess not.

[Kosmaj]

Well, I can report that TTn is RIGHT! It was alleged but never proved that the first new version of LLR, date stamped March 22, 2004 is missing primes. I dug up the version and checked the primes posted by David Broadhurst. The results are given below.

              Athlon  P-M
k/n pair      P-3     P-4
----------------------------
 53 25230     prime   no
 59 134240    prime   prime
 61 6039      prime   prime
 77 2080      no      no
 77 8320      no      no
 95 1536      no      no
103 26927     prime   prime
105 896       no      prime
109 3227      prime   prime
133 26655     prime   prime
165 33389     prime   no
167 576       no      prime
229 8443      prime   prime
233 640       no      no
241 13217     prime   prime
241 16707     prime   no
269 2304      no      prime
269 3840      no      no
297 60114     prime   prime
297 60282     prime   no
297 61370     prime   no
297 65456     prime   prime
297 82782     prime   no
----------------------------
  Missed       8       11

As many as 11 out of 23 primes are falsely declared composites on Pentium-4 and Pentium-M and 8 on Pentium-3/Athlon. Only in 9 cases LLR works correctly on both platforms.

I'm sure this contributed to missed primes reported by David Broadhurst. This version of LLR was revoked not because of this, since at the time nobody tested it on known primes but because some primes were declared composites with zero residues. Jean Penne mentioned that Proth mode was not initialized correctly but in some cases above when primes were declared composites the Proth mode was not used. BTW, the next version of April 11 works well.

I wonder can Joss and others who missed primes check whether they used this erroneous, prematurely released version of LLR.

Related links
New LLR announced, March 24 2004.
Harvey reports a zero residue false negative, April 9
New correct version announced, April 12
LLR not reliable? , May 18
"Buggy version not released" , May 24
DB points out two missed primes k=297, July 6 2004.

[jocelynl]

I just don't know. I updated all my boxes when the new version was out. I never saw any zero residue with the any version. I have done several test on missing numbers with the old version I had and got conflicting results on all of them when using two instances. All were correct when using only one instance of LLR. I also recall using multiple instances on LLRNet. I might have missed some there too.

Joss