Error Rates - Page 2

Prime95 · 2002-08-20, 03:24

Quote:

Originally Posted by outlnder

I believe this to be the correct Forum since I am asking George.

Isn't it up to George whether or not I can request certain exponents??

To make life easier on me, I let the server assign triple-checks at random. Once every couple of months I collect a list of exponents that need triple-checking and let the server give them out.

I will let you reserve an exponent if you have a hunch it is prime or just want to test it for some reason, but only if it is beyond the ones currently being handed out (16.5 million or so today).

svempasnake · 2002-08-26, 10:21

[quote="Prime95"]

Quote:

Originally Posted by outlnder

Thus, you can calculate prime95's error rate by analyzing a range of exponents (say 3 million to 4 million) and count the number of unique entries in bad.zip and divide that by the entries in lucas_v and bad.

I had a quick look into this, at the exonents around 6 - 6.5 millions. According to my rough counting, I find the error rate there to be about 2% - 2.5%! Note that the FAQ suggests about 1% error rate. So I believe the current estimates (on the status page) for double checks to find a new mersenne prime might be too pessimistic!?

Prime95 · 2002-08-26, 14:43

Quote:

Originally Posted by svempasnake

I had a quick look into this, at the exonents around 6 - 6.5 millions. According to my rough counting, I find the error rate there to be about 2% - 2.5%! Note that the FAQ suggests about 1% error rate. So I believe the current estimates (on the status page) for double checks to find a new mersenne prime might be too pessimistic!?

Thanks for the data!

I guess I'm interested in the overall error rate (your 2 - 2.5%) as well as the error rate when prime95 reports a clean run. A clean run is defined as version WVn or later and the last 4 hex digits of the error count field is zero. The error count field is the last field in the line. I suppose it would also be interesting to know the error rate when a non-clean run is reported (version WVn or later and last 4 digits non-zero).

Also, you'll want to analyze a lower range, say 4.5 - 5.0 million. The reason is the two tables do not include results awaiting triple-checks. Obviously, half of the results awaiting triple-check are wrong!

svempasnake · 2002-08-26, 15:10

Quote:

Originally Posted by Prime95

This anaylsis would help answer the question "What is the probability of a Mersenne prime being discovered out of order?"

Sorry, my first "rough counting" was not right. Also chosing 6.0 - 6.5 M was out of topic, since this intervall isn't 100% double checked yet. So please forget that result (2-2.5%).

So I counted again:
This time I count the numbers of entries in the range 5.00 - 5.25 Millions. lucas_v.txt entries =6267, bad file entries = 470. If I got it right this time, we have an error rate here at 470 / 6267 = 7.5%.(!)

Kinda remarkable figure, compared to the 1% stated in FAQ. Feel free to spam me if my counting was way off this time... :?

Prime95 · 2002-08-26, 15:35

Quote:

Originally Posted by svempasnake

This time I count the numbers of entries in the range 5.00 - 5.25 Millions. lucas_v.txt entries =6267, bad file entries = 470. If I got it right this time, we have an error rate here at 470 / 6267 = 7.5%.(!)

A quick count (of distinct entries with program Wxx) in that range I got: 447 bad, 12172 good. Error rate = 447 / (12172 + 447) = 3.5%. More analysis is required to find the error rate on a clean run. The 1% I quote in the FAQ is for a clean run - although I may not state that. The error rate has probably gone up over time too.

svempasnake · 2002-08-26, 17:47

Quote:

Originally Posted by Prime95

A quick count (of distinct entries with program Wxx) in that range I got: 447 bad, 12172 good. Error rate = 447 / (12172 + 447) = 3.5%.

Dear George, thanks for your attention. I think your count is more accurate in one way, mine was semi-manual, based on sorting and counting lines manually. So, for example I was too lazy to count some double entries in the bad file. But I think your automatic counting didn't consider one important detail: Each exponent normally occurs *twice* in lucas_v.txt. I believe 12172 has to be adjusted, probably to 6086 (half your figure)? If that's right, we have 7 % rather than 3.5 % here.
A rough manual calculation looking at the status page confirms 12000 is out of order: Double checked exponents from 5.25 to 6.52 millions are about 30400. So 6000 corresponds to the 0.25 million range pretty well.

PS. Hmmm ! Now I think I got it :idea:

ops: : 3.5% is the figure for error rate per LL test, which is the topic. 7% corresponds to error per exponent, which is out of topic...

Sorry about the confusion.

2002-08-30, 14:37

Hi,

According to the statistical data given the error rate is increasing slightly but not proportional to the effort necessary to test an exponent of a certain size because computers become more reliable. Nevertheless, there is quite a lot of work lost if a LL test of a big exponent turns out to be wrong.
Does it make sense to collect intermediate residuals during a LLtest? I think, it could be useful if Prime95 automatically collects these every 1M or 5M iterations and submits them to the primenet server.
I know, there is already a feature which outputs a residual every N iteration to the results.txt file, but if the primenet server collected such data, it would be able detect an error before a doublecheck is finished (the most recent residual can be compared while updating the primenet server) and only one part of the LLtest has to be done again (unless the error occurred in the first LLtest).
If the firt time LLtest and the double check is done at the same time, they can verify themselves mutually and not triple check is necessary any more.
I guess this feature needs a big effort to update the software and server.

PageFault · 2002-08-30, 19:32

Well if there is an error the triple check will have to be done anyways - George needs two identical runs to declare it properly tested. As for handing out the exponent twice initially - that would stall progress in a big way. We need to cover the first time range as rapidly as possible. The probability of finding a prime in a doublecheck is rather remote compared to the probability of finding a new one in the first time range.

The added bonus of doublechecking much later in the game is that many of them get tested on a different architecture. Imagine if a hardware bug ruined the northwood results and the error was reproducible - if DC's were done concurrently then twice the amount of work would be wasted.

Quote:

Originally Posted by kollegi

If the firt time LLtest and the double check is done at the same time, they can verify themselves mutually and not triple check is necessary any more.

Prime95 · 2002-08-31, 00:15

This issue came up on the mailing list long ago. The optimal GIMPS architecture is to hand out the same exponent to two users, they simultaneously test as long as their residues match. If they divurge, one or both go back to the last matching save file and try again until they agree. This continues until completion.

This minimizes the number of wasted iterations.

However, it takes away one of the great joys of GIMPS. When you are assigned a first time test, you "own" the exponent. If it turns out to be a Mersenne prime, you will be the first person to know it and you will receive the recognition.

Certainly, some wasted work is a small price to pay for this human enjoyment.

Daffy · 2002-08-31, 14:25

I wonder why Prime95 keeps running to the end when it already knows that it made errors ?

Shouldn't it go back to previous backups (assuming we start savings more than just the last one) and re-run that portion to double check that it doesn't make the same error ? And if it does, shouldn't it simply give up, declare an hardware problem and maybe start running stress tests again until the user "repair" his computer ?

I think it is a waste to let Prime95 report a result when it knows it made mistakes days or weeks earlier.

Also, I understand Prime95 is able to detect some errors related to the size of the FFT used. Why not allowing it to change the size and re-start the calculation from the beginning instead of going to the end for nothing ?

I think we are all insterested in having valid results posted, not simply having computers running 24 hours a day.

Digital Concepts · 2002-08-31, 17:36

Daffy,

Prime95 does what you are asking - whenever it can. When it finds an error, it does rerun from the last checkpoint. And if it detects an error related to FFT size, a recent improvement does implement the necessary algorithm change. It doesn't just run to the end without doing anything.

But there are things that can go wrong with memory, etc. that Prime95 has no way to know about, which is why residues don't match and a triple check is necessary.

2002-08-31, 14:25	#21
Daffy Aug 2002 1F₁₆ Posts	Error rates I wonder why Prime95 keeps running to the end when it already knows that it made errors ? Shouldn't it go back to previous backups (assuming we start savings more than just the last one) and re-run that portion to double check that it doesn't make the same error ? And if it does, shouldn't it simply give up, declare an hardware problem and maybe start running stress tests again until the user "repair" his computer ? I think it is a waste to let Prime95 report a result when it knows it made mistakes days or weeks earlier. Also, I understand Prime95 is able to detect some errors related to the size of the FFT used. Why not allowing it to change the size and re-start the calculation from the beginning instead of going to the end for nothing ? I think we are all insterested in having valid results posted, not simply having computers running 24 hours a day.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Electrical Service Rates	storm5510	Hardware	70	2019-10-14 21:14
error rates and P-1 test	drakkar67	Prime Sierpinski Project	9	2008-05-26 14:29
error rates	drakkar67	Prime Sierpinski Project	12	2006-04-21 17:26
LL Test Rates and GIMPS Promotion	Primenut	Lounge	14	2003-06-09 09:32
Error rates revealed	Prime95	Math	1	2002-09-01 00:10

2002-08-30, 14:37	#18
kollegi 7,177 Posts	Hi, According to the statistical data given the error rate is increasing slightly but not proportional to the effort necessary to test an exponent of a certain size because computers become more reliable. Nevertheless, there is quite a lot of work lost if a LL test of a big exponent turns out to be wrong. Does it make sense to collect intermediate residuals during a LLtest? I think, it could be useful if Prime95 automatically collects these every 1M or 5M iterations and submits them to the primenet server. I know, there is already a feature which outputs a residual every N iteration to the results.txt file, but if the primenet server collected such data, it would be able detect an error before a doublecheck is finished (the most recent residual can be compared while updating the primenet server) and only one part of the LLtest has to be done again (unless the error occurred in the first LLtest). If the firt time LLtest and the double check is done at the same time, they can verify themselves mutually and not triple check is necessary any more. I guess this feature needs a big effort to update the software and server.

2002-08-31, 00:15	#20
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 16546₈ Posts	This issue came up on the mailing list long ago. The optimal GIMPS architecture is to hand out the same exponent to two users, they simultaneously test as long as their residues match. If they divurge, one or both go back to the last matching save file and try again until they agree. This continues until completion. This minimizes the number of wasted iterations. However, it takes away one of the great joys of GIMPS. When you are assigned a first time test, you "own" the exponent. If it turns out to be a Mersenne prime, you will be the first person to know it and you will receive the recognition. Certainly, some wasted work is a small price to pay for this human enjoyment.

2002-08-31, 17:36	#22
Digital Concepts Aug 2002 2·3³ Posts	Daffy, Prime95 does what you are asking - whenever it can. When it finds an error, it does rerun from the last checkpoint. And if it detects an error related to FFT size, a recent improvement does implement the necessary algorithm change. It doesn't just run to the end without doing anything. But there are things that can go wrong with memory, etc. that Prime95 has no way to know about, which is why residues don't match and a triple check is necessary.