mersenneforum.org > Data Error rate plot
 Register FAQ Search Today's Posts Mark Forums Read

 2015-12-24, 04:42 #67 ewmayer ∂2ω=0     Sep 2002 República de California 32×1,303 Posts Is the error rate really dropping as the exponents get larger, or is that an artifact of the sampling methodology?
2015-12-24, 06:43   #68
Serpentine Vermin Jar

Jul 2014

52·7·19 Posts

Quote:
 Originally Posted by ewmayer Is the error rate really dropping as the exponents get larger, or is that an artifact of the sampling methodology?
My own (probably unscientific) prediction is that error rates will increase with the larger exponents. For the simple reason that the longer a test takes, the more likely a memory or CPU error will creep in.

On the other hand, these larger tests are done by (generally) more reliable systems which may balance things out.

FYI, hopefully we'll see a curious "bump" in proven bad results as a direct result of our effort to find these bad CPUs and do double-checking ahead of the curve.

When we find those bad systems, they're still technically unverified until a triple-check is done, so it may still be a while until those get done.

2015-12-24, 07:15   #69
Dubslow

"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

1C3516 Posts

Quote:
 Originally Posted by ewmayer Is the error rate really dropping as the exponents get larger, or is that an artifact of the sampling methodology?
I believe that's the sampling methodology -- those tests simply have been largely not double checked, so anything with zero error code is assumed to be good, even though that's not true.

2015-12-24, 21:00   #70
Serpentine Vermin Jar

Jul 2014

332510 Posts

Quote:
 Originally Posted by Dubslow I believe that's the sampling methodology -- those tests simply have been largely not double checked, so anything with zero error code is assumed to be good, even though that's not true.
In my analysis of known bad results, if it had a zero error code there's something like less than 1% bad. If it's a non zero error, it's more like a 50% chance of being bad.

I should clarify that by "non zero" I mean any error code that would mark it as "suspect". There are non-zero errors that are normal... like repeatable rounding errors during the run. I should say "suspect" and "non-suspect" instead of zero/non-zero.

Well, I forget the actual rates, but it was something like that.

2015-12-24, 21:47   #71
Serpentine Vermin Jar

Jul 2014

1100111111012 Posts

Quote:
 Originally Posted by Madpoo In my analysis of known bad results, if it had a zero error code there's something like less than 1% bad. If it's a non zero error, it's more like a 50% chance of being bad. I should clarify that by "non zero" I mean any error code that would mark it as "suspect". There are non-zero errors that are normal... like repeatable rounding errors during the run. I should say "suspect" and "non-suspect" instead of zero/non-zero. Well, I forget the actual rates, but it was something like that.
Just checked again...

If the result is marked "clean" (not suspect), counting up all of the known good/bad results, the odds of it being bad are just a hair under 2%.

(I don't consider the still-unknowns when calculating, i.e. I just use (bad)/(bad+good)).

Here are the raw stats on those "clean" results:
Code:
Unknown	Bad	Good
648674	29698	1501012
For the dirty/suspect results, 54% of the suspect results end up being bad.

Raw stats:
Code:
Unknown	Bad	Good
5904	31827	27562
EDIT: To be clear, there are certain suspect error codes that always end up with a bad result. For example, if you see an error code of "00000304" then beware, your machine hosed up. :) But on average, results marked suspect are bad more often than good.

Last fiddled with by Madpoo on 2015-12-24 at 21:56

 2015-12-24, 23:07 #72 Madpoo Serpentine Vermin Jar     Jul 2014 1100111111012 Posts Another couple data points... If the error code actually is just "00000000" the odds of it being bad are currently 1.84% The *total* known error rate no matter what the result code = 3.87% Last fiddled with by Madpoo on 2015-12-24 at 23:10
2015-12-24, 23:26   #73
patrik

"Patrik Johansson"
Aug 2002
Uppsala, Sweden

52·17 Posts

Quote:
 Originally Posted by diep Great! Do i correctly interpret the red as that from all verified tests between about 2% to 4% in the 40M range gives an error? So (P( has error AND was verified test ) / P( all verified tests) ) = [2% ; 4%] Did i read this correct? If so - excuse my ignorance, how can the green be above the red in the estimate?
No, the red curve does not contain information about the error code of the test. Instead it is the number of known bad LL results in an interval divided by the number of all1 LL results (good + bad) in the same interval, i.e.

( # known bad LL results ) / ( # known bad LL results + # known good LL results )

A result is known to be bad when two other LL results for the same exponent don't match the result but still match each other.

I use the publicly available information at mersenne.org to collect information (Reports->Detailed Reports->LL Results).

The green curve is above the red one since I count a number of unverified tests as bad: When there are still unverified exponents in the interval (i.e. there are exponents left, for which there are still not two matching tests), if there are n non-matching tests for an exponent, I count n-1 of them as bad. The red curve does not use the unverified tests at all.

1Not including unverified tests, though, if any.

2015-12-24, 23:43   #74
patrik

"Patrik Johansson"
Aug 2002
Uppsala, Sweden

1101010012 Posts

Quote:
 Originally Posted by ewmayer Is the error rate really dropping as the exponents get larger, or is that an artifact of the sampling methodology?
I think that the green curve does not accurately describe the error rate. I don't understand, though, why it looks as it does. The effect of the decreasing green curve was present in my second post in this thread, but in the latest plot the effect has disappeared in that range.

Still, a fairly constant error rate at around 4% would mean that the probability per iteration of an error is dropping at the same time as the amount of computations (per iteration) is increasing (due to the larger FFT:s).

 2015-12-26, 20:14 #75 henryzz Just call me Henry     "David" Sep 2007 Liverpool (GMT/BST) 25×11×17 Posts It would be interesting to see the graph with fft boundaries added to see if they match the variations in the graph at all.
2015-12-26, 21:11   #76
chalsall
If I May

"Chris Halsall"
Sep 2002

35·43 Posts

Quote:
 Originally Posted by henryzz It would be interesting to see the graph with fft boundaries added to see if they match the variations in the graph at all.
That would indeed be interesting.

Are you willing to work it?

2015-12-26, 22:30   #77
henryzz
Just call me Henry

"David"
Sep 2007
Liverpool (GMT/BST)

598410 Posts

Quote:
 Originally Posted by chalsall That would indeed be interesting. Are you willing to work it?
If I had the data I could persuade R to produce a graph displaying this information.

 Similar Threads Thread Thread Starter Forum Replies Last Post ixfd64 Hardware 4 2011-04-12 02:14 S485122 PrimeNet 15 2009-01-16 11:27 GP2 Data 3 2003-12-01 20:24 dsouza123 Data 6 2003-10-23 22:26 GP2 Data 5 2003-09-15 23:34

All times are UTC. The time now is 12:11.

Sat May 21 12:11:36 UTC 2022 up 37 days, 10:12, 0 users, load averages: 1.22, 1.37, 1.45