Newer milestone thread

[NBtarheel_33]

Originally Posted by Uncwilly

Just as a note: we are now at 0.500 expected new primes in the 79.3 range.

Current expected new primes in the 79.3M range: 0.467.
Change since January 26, 2015 (63 days ago): 0.500 - 0.467 = 0.033.
Change since January 1, 2014 (453 days ago): 0.705 - 0.467 = 0.238.

Expected time to next prime in the 79.3M range is therefore estimated by:

63 days/0.033 primes = 1,909 days (from today), or June 20, 2020 OR
453 days/0.238 primes = 1,903 days (from today), or June 14, 2020.

Seems that our throughput has been remarkably stable over the last fifteen months or so.

[Uncwilly]

Originally Posted by NBtarheel_33

Expected time to next prime in the 79.3M range is therefore estimated by:

63 days/0.033 primes = 1,909 days (from today), or June 20, 2020 OR
453 days/0.238 primes = 1,903 days (from today), or June 14, 2020.

Seems that our throughput has been remarkably stable over the last fifteen months or so.

But if you will note:

Originally Posted by Uncwilly

The estimated "completion date" for the classic 79.3M range has been hovering around Jan-Feb 2018 for almost 5 months. Going back to 2013 it has been in the same range +- 2.5 months.

That is based upon the change in P90 years remaining and the rate of change (based upon a floating period of ~8-13 weeks).

[Madpoo]

Originally Posted by ATH

The one at 96.2% will expire ~ June 4th 2015 + 3.33 days for every extra percent done "plus a grace period if close to finished" (unspecified grace period).

The one at 64.1% will expire ~ May 15th 2015 + 3.33 days for every extra percent done

The one at 63.4% will expire ~ May 13th 2015 + 3.33 days for every extra percent done

From what you said they are doing less than 0.3% per day (1% in 3.33 days), so they will eventually expire, except the one at 96.2% might just make it if it does 0.1% every 3 days and depending on the grace period.

Revisiting this... I was curious what the grandfather rules actually look like, as the server itself applies it when expiring assignments.

I think George posted this elsewhere but anyway, the basic thing for first-time LL checks boils down to:

• Grace period only applies to assignments prior to 2014-03-01
• Only applies to assignments in the critical range (as of this message it's anything below 58629120)
• Grandfathered assignments higher than the critical threshold get a free pass...of course that threshold is always increasing over time
• Calculates a "grace" percent done threshold
• If the current percent done is below the "grace" threshold *and* it's below that critical threshold, it recycles that assignment

Here's the part I didn't really look at, and that's how the "grace percentage" is calculated. It's somewhat straightforward once I stared at it a while:

• There's a 10% deduction in % done right off the top
• You get one "free" year (365 days)
• Anything past the first 365 days, it expects 1% progress per day

The way it calculates, it takes the difference between right now and the date it was assigned, and then subtracts that 365 days. For example, exponent 54357769 was assigned "2013-08-21 02:17:38.490" which was 595 days ago, or 230 days once we grant that one year bonus.

Take 230 days and divide by 3.33 and it would expect you to be at least 69.1% done by then, but add another 10% to that for an expectation that after a grand total of 595 days, you *should* be at 79.1% complete.

That assignment is, in fact, 97.7% done so yay, it gets a reprieve. But pretend for a moment that it does still crawl along at mind-numbingly slow speeds. At some point the "grace percentage" actually goes over 100% so even if it were at 99.9% done it would expire.

That will happen when an exponent is over 665 days old: (665-365)/3.33 + 10 = 100.1.

When that happens (as of today, that's any assignment earlier than June 12, 2013), as long as it's below that critical threshold it will expire.

In the example of M54357769 that will happen in 70 more days from today.

There are currently 14 first-time, grandfathered assignments in that critical range. It's hard to say if any of them are in particular danger of expiring since I don't know how fast they're actually progressing. The closest is M58403539 which is just 7.28% ahead of the cut-off.

Grand total, there are 3,294 grandfathered first-time checks. 2,622 of them are above 100,000,000 which are well ahead of the critical area and will be safe for some time to come. Most of the other 672 are in the 60-70M range with only 104 below 60M.

So... that makes me feel better, knowing some of the slower systems out there can't actually hold things up indefinitely. They will eventually expire.

[retina]

Originally Posted by Madpoo

it expects 1% progress per day
...
Take 230 days and divide by 3.33

To me that says 0.3% per day.

Originally Posted by Madpoo

ou *should* be at 79.1% complete

In the example of M54357769 that will happen in 70 more days from today.

And again to me that says 0.3% per day.

[Madpoo]

Originally Posted by retina

To me that says 0.3% per day....And again to me that says 0.3% per day.

Lack of sleep. Yes, carry the one, borrow a seven, multiply by pi and divide by zero. You'd think I didn't know a dividend from a hole in the ground.

Okay: (# of days / 3.33) implies it's expecting 0.3% daily.

Anyway, besides that, the rest of the math should be correct. If a grandfathered exponent reaches the ripe old age of 665 days old, it will expire no matter what (assuming it's below the "critical" threshold, which has it's own algorithm).

That's more generous than I probably would be with exponents in that critical range.

[Madpoo]

Originally Posted by Madpoo

Okay: (# of days / 3.33) implies it's expecting 0.3% daily.

Incidentally, I'm trying to think of something that the server could be doing to not only track the current % done, but also gather some kind of velocity. It may be somewhat basic to keep it simple, like just doing a delta of the last check-in and the current one. Anything fancier that tracks it more thoroughly would give better results but also be a database burden and be kind of a chore.

Basically, the database is big enough as-is without trying to keep track of the % done and timestamps for every time a client updates itself. It would be interesting though to have some basic stats, because the current ETA based on the client's "best guess" can be wildly optimistic, to say the least.

It's not high on my to-do list but maybe when I have some spare time I can noodle around with some ideas... if anyone has any thoughts on that, let me know.

[retina]

Originally Posted by Madpoo

Incidentally, I'm trying to think of something that the server could be doing to not only track the current % done, but also gather some kind of velocity. It may be somewhat basic to keep it simple, like just doing a delta of the last check-in and the current one. Anything fancier that tracks it more thoroughly would give better results but also be a database burden and be kind of a chore.

This is one of those things that appears simple at first and turns out to be fraught with so many problems and assumptions that the results are only good for a small percentage of tasks. People turn of their computers at night, at weekends, when they are on vacation holiday, when they are out doing fieldwork, etc. People also do computational intensive things at irregular times for irregular periods that prevent lower priority tasks running. Computers get hot and throttle, or crash. Power goes out. And probably a million other things that affect the throughput. I can see it becoming yet another ignored piece of data along with the existing ETA values.

[axn]

Originally Posted by Madpoo

Basically, the database is big enough as-is without trying to keep track of the % done and timestamps for every time a client updates itself.

So only do it for exponents of interest, say the lowest 100 (or earliest 100) active assignments (LL / DC). Dump them into a separate table, and do a linear regression to get more reliable ETAs. Once the assignment is over, clean out the table.

[Madpoo]

Originally Posted by axn

So only do it for exponents of interest, say the lowest 100 (or earliest 100) active assignments (LL / DC). Dump them into a separate table, and do a linear regression to get more reliable ETAs. Once the assignment is over, clean out the table.

Yeah, I'm kind of leaning towards something like that. Reason being, I don't want to do anything that would require changes on the way these check-ins are handled by the server. As in, I don't want to mess with any current functionality and risk breaking whatever.

My best guess at an approach right now would be to setup a whole new table in the DB that run some scheduled job that takes the current dates and progress and stores them. Then the website could examine that in whatever way it wants to get some idea of the real progress going on.

To your point, it could indeed be relegated to just the stuff that might show up in whatever milestone reports we're interested in at the time. If that were the case, it could track maybe a couple weeks worth of check-ins for a subset of work and it wouldn't be too bad to manage.

Well, it's a thought for sure. First step is collecting that data which really isn't too hard. Next would be doing something with it.

[ATH]

Originally Posted by Madpoo

Incidentally, I'm trying to think of something that the server could be doing to not only track the current % done, but also gather some kind of velocity. It may be somewhat basic to keep it simple, like just doing a delta of the last check-in and the current one. Anything fancier that tracks it more thoroughly would give better results but also be a database burden and be kind of a chore.

This is what I suggested a while back except I suggested the average over last 3 checkin to better average out peoples irregular behaviour:
http://www.mersenneforum.org/showpos...postcount=1548

I actually meant the 4th last checkin in that formula so we get 3 gaps between 4 checkins, but 3 is just a suggestion maybe another number of gaps is better.

[Madpoo]

Originally Posted by Madpoo

Lack of sleep. Yes, carry the one, borrow a seven, multiply by pi and divide by zero. You'd think I didn't know a dividend from a hole in the ground.

Okay: (# of days / 3.33) implies it's expecting 0.3% daily.

Anyway, besides that, the rest of the math should be correct. If a grandfathered exponent reaches the ripe old age of 665 days old, it will expire no matter what (assuming it's below the "critical" threshold, which has it's own algorithm).

That's more generous than I probably would be with exponents in that critical range.

And while I'm at it... the double-check grandfather rules are similar.

Whereas first-time checks assume the work will be at least 10% done after the first year, double-checks assume they'll be at least 60% done in the first year.

First time assumes 0.3% progress daily, and double-checks assume 0.333% daily.

What it boils down to for double-checks is that it will expire no matter what, even at 99.99%, once the exponent reaches an age of 485 days (~ 1 year+4 months, compared to ~ 1 year+10 months for first time checks).

Just like first time grandfathered assignments, this only applies to work in the critical range. Right now that means exponents below 34505378. As it turns out, there aren't any grandfathered DC assignments below that anyway.

There are just 69 of them right now...

• 6 in the 35-38M range
• 16 between 40M-50M
• 40 between 50M-60M
• 3 between 60M-70M
• and then this motley bunch to round it out: 85000043,100000237,100000379,100000609,112401617

I can say that of the 4 grandfathered assignments in the 35M range, once the critical threshold reaches them, they would all get expired right away. Some of them aren't even really close, like 16-20%. I mean, *maybe* by the time 35040547 is in the critical area it will have moved past the expiration threshold, but I guess we'll see. As of today, that threshold is 78% for that exponent, and it's only 16.4% done.

I'm not sure when to expect that one to be in the critical range based on the current progress, but it has some catching up to do.