Double checking of Results

pinhodecarlos · 2017-11-29, 20:07

....was wondering if we need to double check below 4000e15.

rudy235 · 2017-11-29, 21:58

Quote:

Originally Posted by pinhodecarlos

Anyway, was wondering if we need to double check below 4000e15.

That work was done by Tomás Oliveira e Silva of Aveiro University. While he only double checked randomly 400 x 10¹⁵ subranges or 10% of the 4000 x 10¹⁵ subranges, he contrasted the number of Primes discovered against the number of primes π(n) obtained by variants of the Meissel-Lehmer algorithm. In all 4000 intervals there was no discrepancy at all.

danaj · 2017-11-29, 22:18

It would be a possibly useful double check on our program, but perhaps best done as spot checks rather than duplicating the whole thing.

rudy235 · 2017-11-29, 23:07

Quote:

Originally Posted by danaj

It would be a possibly useful double check on our program, but perhaps best done as spot checks rather than duplicating the whole thing.

That is something that always concerned me. I believe that a totally random ( blind ) test of 10% of the ranges would be useful.

We have 6000 e15 subranges. We could chose 600 e15 random ranges, I believe Mathematica has a Random Number Generation, and test them, stipulating that no one would check it's own work. That would go a great length to guarantee the soundness of this work.

danaj · 2017-11-30, 00:58

Seems like a new thread for this makes sense. This is going far off topic.

I was thinking just check around windows of the gaps < 4e18. Our program should find the gap. I'm unsure as to the value but it is cheap as long as the windows are reasonably sized.

For the spot check of *our* results, we could do the same (but I see no point), but what we probably want is checking that the recheck finds the same gaps as the original.

Alternately as you suggest just do random ranges. What you want is random selections without duplication, which can be done using Perl/ntheory's randperm, Mathematica's RandomSample, numpy's choice (though it can easily blow up in time and memory), and more. Ensuring no double check of own work is probably manual with our current non-automated methods.

In theory it'd seem like we'd also want more like "a random 10% of each partipants work" rather than completely random, or we will likely do no double checking of some people's work.

While it'd be nice to have full results from everything already done, I'm not sure how many people sent that info. What I believe we do have is the "largest gap" for reasonably sized subranges, so one could go through all the results and turn them into [version, parameters, expected value(s), participant id] tuples for small chunks. Version here is distinguishing between the "A,B" ranges vs. "A,B,M1,M2" modulo ranges. Then it would be reasonable to do the sampling from that set and assign the double checks easily enough.

rudy235 · 2017-11-30, 03:03

Danaj : if you feel this requires a new thread perhaps you could open a new one. I do have some comments of my own but I will keep then in pectore for the time being.

robert44444uk · 2017-11-30, 07:54

The weakest element of our work:

we have not automated the reporting of results
the lack of a centralised data repository
the manual extraction of the largest gap in any given range of 10 "res" from the generated report of highest gap for each "res"
The incompleteness of the reporting of gaps >=1000
We are humans (last time I looked) and it is possible someone could be omitting or forging results

All of these weaknesses make the job of double checking hard because even if an error is found it may not impact the first instance gaps we find i.e. the robustness of the 25 or so gaps of this nature found to date and most will relate to the 3rd element listed above.

The most critical error would be if there is a hole in the logic for Robert G's program, which would lead to systematic errors requiring a total recheck of the range. The program has been looked at in detail by any number of people so I have not listed this as a weakness. If there was one it would most likely relate to the way ranges are stitched together (is this guaranteed seamless?); to the changing of program variables during the testing of a range; or the generation of the "worktodo" file in stressful situations such as power cuts.

Automating the reporting would go a long way to overcoming most of the weaknesses I have listed.

There are 234,057,667,276,344,606 prime gaps below 10e18. I for one will want to leave these be for now! Onwards and upwards!

rudy235 · 2017-11-30, 16:32

My principal concern lies with the fact that probably nobody will audit the results ever.

In other words if (for example) the gap 1382 lies somewhere among the 6e18 and 10e18 and we have let it slip then when someone does find it at a higher range and an exhaustive search is done this find will go as a CFC, when it might simply be just a simple gap without much significance i.e. the second (or worse third) occurrence of a specific gap.

Of the many elements mentioned by Robert S I can only comment this.

As to the intentional forging of data I truly don't believe that to be a major concern. There is almost no benefit in doing that. Any record will be checked by both Robert S and (presumably) Dr. Nicely and checking this is very easy to do. So, only the intentional omission of a find could be done without the chance of being found out. But what benefit would that do to the culprit?
The other comment by Robert S does seem to be more pertinent.
The lack of a centralized data repository

I hope something can be done about this.

danaj · 2017-11-30, 18:00

Robert: The weakest element of our work:

we have not automated the reporting of results

I was hoping to do an automated setup, but I don't have time at the moment. It's on my todo list but I unless I get a big chunk of free time to work through things, I won't be able to do it. Unless we're going to go past 2^64 we only have 8 ranges to go, so we can debate the worth at this point (albeit it would make Robert's life easier).
the lack of a centralised data repository

Robert, did you have ideas? I think I could work on this with you but I'd like to know if someone has already collected all the various zip files people have attached.
the manual extraction of the largest gap in any given range of 10 "res" from the generated report of highest gap for each "res"

With a repo of the results, this shouldn't be manual. I use a Perl snippet to generate the list from the gap_solutions.txt files, not gap_results.txt or a manual process.

If we had a central repo then all the row results could be generated and compared to what we have.
The incompleteness of the reporting of gaps >=1000

My understanding is that this isn't possible with the current program (IIRC when we started it would report at 1000+ but guarantee results only for 1050+), but is complete only at a slightly higher number. Our current m1 is 1190, so what is the guarantee?
We are humans (last time I looked) and it is possible someone could be omitting or forging results

All gaps go through multiple verifications. The endpoints are proved with multiple software methods, and the interior checked similarly. So the gaps we find are real. But indeed someone could be intentionally or accidently missing results. This is a benefit of doing a comprehensive double check -- checking that all the expected results are found. If those pass, then their results seem to be working. Just doing spot checks hoping to find missing large results is more dubious. Running either different software or using different parameters would be nice as well. I don't think we have any other software fast enough, but we could at least use some different parameters as long as they would get the same results (e.g. lower m1).

robert44444uk · 2017-12-01, 07:31

Quote:

Originally Posted by danaj

With a repo of the results, this shouldn't be manual. I use a Perl snippet to generate the list from the gap_solutions.txt files, not gap_results.txt or a manual process.

Could you share the perl snippet? That would make my job easier!

pinhodecarlos · 2017-12-08, 22:15

Out of cutiosity, are we going to search beyond 16950e15?

2017-11-29, 20:07	#1
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 2³·641 Posts	Double checking of Results ....was wondering if we need to double check below 4000e15. Last fiddled with by robert44444uk on 2017-11-30 at 08:01

2017-11-30, 07:54	#7
robert44444uk Jun 2003 Suva, Fiji 2³·3·5·17 Posts	The weakest element of our work: we have not automated the reporting of results the lack of a centralised data repository the manual extraction of the largest gap in any given range of 10 "res" from the generated report of highest gap for each "res" The incompleteness of the reporting of gaps >=1000 We are humans (last time I looked) and it is possible someone could be omitting or forging results All of these weaknesses make the job of double checking hard because even if an error is found it may not impact the first instance gaps we find i.e. the robustness of the 25 or so gaps of this nature found to date and most will relate to the 3rd element listed above. The most critical error would be if there is a hole in the logic for Robert G's program, which would lead to systematic errors requiring a total recheck of the range. The program has been looked at in detail by any number of people so I have not listed this as a weakness. If there was one it would most likely relate to the way ranges are stitched together (is this guaranteed seamless?); to the changing of program variables during the testing of a range; or the generation of the "worktodo" file in stressful situations such as power cuts. Automating the reporting would go a long way to overcoming most of the weaknesses I have listed. There are 234,057,667,276,344,606 prime gaps below 10e18. I for one will want to leave these be for now! Onwards and upwards!

2017-11-30, 18:00	#9
danaj "Dana Jacobsen" Feb 2011 Bangkok, TH 909₁₀ Posts	Robert: The weakest element of our work: we have not automated the reporting of results I was hoping to do an automated setup, but I don't have time at the moment. It's on my todo list but I unless I get a big chunk of free time to work through things, I won't be able to do it. Unless we're going to go past 2^64 we only have 8 ranges to go, so we can debate the worth at this point (albeit it would make Robert's life easier). the lack of a centralised data repository Robert, did you have ideas? I think I could work on this with you but I'd like to know if someone has already collected all the various zip files people have attached. the manual extraction of the largest gap in any given range of 10 "res" from the generated report of highest gap for each "res" With a repo of the results, this shouldn't be manual. I use a Perl snippet to generate the list from the gap_solutions.txt files, not gap_results.txt or a manual process. If we had a central repo then all the row results could be generated and compared to what we have. The incompleteness of the reporting of gaps >=1000 My understanding is that this isn't possible with the current program (IIRC when we started it would report at 1000+ but guarantee results only for 1050+), but is complete only at a slightly higher number. Our current m1 is 1190, so what is the guarantee? We are humans (last time I looked) and it is possible someone could be omitting or forging results All gaps go through multiple verifications. The endpoints are proved with multiple software methods, and the interior checked similarly. So the gaps we find are real. But indeed someone could be intentionally or accidently missing results. This is a benefit of doing a comprehensive double check -- checking that all the expected results are found. If those pass, then their results seem to be working. Just doing spot checks hoping to find missing large results is more dubious. Running either different software or using different parameters would be nice as well. I don't think we have any other software fast enough, but we could at least use some different parameters as long as they would get the same results (e.g. lower m1).

2017-12-08, 22:15	#11
pinhodecarlos "Carlos Pinho" Oct 2011 Milton Keynes, UK 12010₈ Posts	Out of cutiosity, are we going to search beyond 16950e15? Last fiddled with by pinhodecarlos on 2017-12-08 at 22:16

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Double checking	gd_barnes	Riesel Prime Search	71	2022-10-02 17:21
Double Checking my own work?	evanh	Software	3	2018-01-07 01:05
What about double-checking TF/P-1?	137ben	PrimeNet	6	2012-03-13 04:01
Double checking	Unregistered	Information & Answers	19	2011-07-29 09:57
Any glory in double checking?	Quacky	Lounge	5	2003-12-03 02:20

2017-11-29, 22:18	#3
danaj "Dana Jacobsen" Feb 2011 Bangkok, TH 3²×101 Posts	It would be a possibly useful double check on our program, but perhaps best done as spot checks rather than duplicating the whole thing.

2017-11-30, 00:58	#5
danaj "Dana Jacobsen" Feb 2011 Bangkok, TH 3²·101 Posts	Seems like a new thread for this makes sense. This is going far off topic. I was thinking just check around windows of the gaps < 4e18. Our program should find the gap. I'm unsure as to the value but it is cheap as long as the windows are reasonably sized. For the spot check of our results, we could do the same (but I see no point), but what we probably want is checking that the recheck finds the same gaps as the original. Alternately as you suggest just do random ranges. What you want is random selections without duplication, which can be done using Perl/ntheory's randperm, Mathematica's RandomSample, numpy's choice (though it can easily blow up in time and memory), and more. Ensuring no double check of own work is probably manual with our current non-automated methods. In theory it'd seem like we'd also want more like "a random 10% of each partipants work" rather than completely random, or we will likely do no double checking of some people's work. While it'd be nice to have full results from everything already done, I'm not sure how many people sent that info. What I believe we do have is the "largest gap" for reasonably sized subranges, so one could go through all the results and turn them into [version, parameters, expected value(s), participant id] tuples for small chunks. Version here is distinguishing between the "A,B" ranges vs. "A,B,M1,M2" modulo ranges. Then it would be reasonable to do the sampling from that set and assign the double checks easily enough.

2017-11-30, 03:03	#6
rudy235 Jun 2015 Vallejo, CA/. 1149₁₀ Posts	Danaj : if you feel this requires a new thread perhaps you could open a new one. I do have some comments of my own but I will keep then in pectore for the time being.

2017-11-30, 16:32	#8
rudy235 Jun 2015 Vallejo, CA/. 3·383 Posts	My principal concern lies with the fact that probably nobody will audit the results ever. In other words if (for example) the gap 1382 lies somewhere among the 6e18 and 10e18 and we have let it slip then when someone does find it at a higher range and an exhaustive search is done this find will go as a CFC, when it might simply be just a simple gap without much significance i.e. the second (or worse third) occurrence of a specific gap. Of the many elements mentioned by Robert S I can only comment this. As to the intentional forging of data I truly don't believe that to be a major concern. There is almost no benefit in doing that. Any record will be checked by both Robert S and (presumably) Dr. Nicely and checking this is very easy to do. So, only the intentional omission of a find could be done without the chance of being found out. But what benefit would that do to the culprit? The other comment by Robert S does seem to be more pertinent. The lack of a centralized data repository I hope something can be done about this.