mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > PrimeNet

Reply
 
Thread Tools
Old 2016-03-23, 18:59   #1
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

7,159 Posts
Default Modifications to LL assignment rules!!!

This thread is for discussion of modifications to the LL assignment rules. After a plan of action is decided upon we will re-visit DC assignment rules. Refer to milestones thread for some of the recent discussions on this issue. As a result of those discussions, I've proposed in this post a new SQL query to run nightly that will give us more metrics for more intelligent decision making.


Background:

1) Cat 1 & 2 assignments were reserved for those users that promised to complete assignments quickly.
2) We assumed GIMPS completed 100,000 LL tests a years. Thus, by setting the cat 3 cutoff at the first 100,000 exponents, a user getting a cat 4 assignment would have at least a year to complete before getting in the way of a milestone.
3) We selected 90/150/270/360 days to complete an assignment. Exponents not recycled until they reach category 1.

Current problems:

1) The number of computers signed up for cat 1 & 2 assignments is too low.
2) Some cat 1 assignments are going to computers that are too slow, suspect, or have a history of abandoning assignments.
3) The very smallest exponents (a.k.a. cat 0) might need even stricter limits to advance milestones faster are discourage poaching.

Old metrics:

1) CPU speed (adjusted by rolling average, hours-per-day, and number of workers)
2) Number of recent LL completions
3) Computer's days-of-work-get value. NOTE, we do not have access to the days between-checkins value.
4) Computer's reliability value -- a rather gross estimate of how error-prone the computer is.

New metrics (from a new SQL query run once a day):

1) Computer's rank ordered by LL/DC GHZ-days completed over the last 90 days.
2) Computer's number completed LL/DC attempts for last 90 days.
3) Computer's number of suspect LL/DC results for the last 90 days.
4) Computer's number of expired assignments for the last 90 days.

Data:

Today's cat cutoffs (and available assignments, not including those that GPU-to-72 has):
cat 3: 76.15M 15,866
cat 2: 68.83M 2,962
cat 1: 67.76M 3,137

Same info one year ago:
cat 3: 71.46M 62,752
cat 2: 64.45M 5,833
cat 1: 58.46M 2,141

Assignments in the last 30 days:
cat 1: 837 (639 active, 198 completed)
cat 2: 16 (all still active)
cat 3: 6244 (5924 active, 320 completed)
cat 4: 5128 (5020 active, 108 completed)

Total completed LL tests in last 30 days grouped by exponent's category the day it was assigned:
cat 1: 799
cat 2: 22
cat 3: 4655
cat 4: 3036


Goals:

1) Regular progress on milestones
2) Reduce the vast number of available cat 1/2/3 assignments (i.e. compress the ranges if possible)
3) Allow users with slower computers to get LL tests without getting in the way of milestones.
4) If necessary, adjust manual testing ranges
5) Don't piss off users by recycling assignments that are making decent enough progress.
6) Where possible, rules should be self-adjusting. Examples: rather than "computer should have a CPU speed of 3GHz" we have a rule of "computer is ranked in the top 1000 producers over the last 90 days". Another example: instead of "cat 2 is the first 10000 exponents", we could try "cat 2 is the number of exponents we expect the top 500 computers could complete in the next 120 days".

Last fiddled with by Prime95 on 2016-03-25 at 09:52
Prime95 is offline   Reply With Quote
Old 2016-03-23, 19:07   #2
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

157678 Posts
Default

I will update the first post as the discussion develops. Thus, you can refer back to see the current state of affairs without re-reading the entire thread.

First up, do we need more data to help in our thinking?
Prime95 is offline   Reply With Quote
Old 2016-03-23, 19:26   #3
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

932710 Posts
Default

Quote:
Originally Posted by Prime95 View Post
First up, do we need more data to help in our thinking?
Thanks for re-opening this discussion George. Much needed.

Just to put on the table, GPU72 has recently been focusing on building up a buffer of candidates TF'ed to 75 bits to give to P-1'ers. Some of these are in the high Cat 3 range (75M), with the rest in the low Cat 4 range (76M & 77M) to feed to the "churners" (which have now largely stabilized after the latest MP announcement).

The goal has been to ensure that no candidate is assigned for P-1'ing without first being appropriately TF'ed, and that no candidate is assigned for LL'ing without first being appropriately P-1'ed.

We have also been working on TF'ing the low Cat 3 range (which already has P-1 done). I've redirected some additional amount of our firepower down there as our aforementioned "to be P-1'ed" buffer is looking pretty comfortable at the moment.
chalsall is offline   Reply With Quote
Old 2016-03-23, 19:36   #4
Mark Rose
 
Mark Rose's Avatar
 
"/X\(‘-‘)/X\"
Jan 2013

55428 Posts
Default

I would restrict access to Cat 1/2 DC to computers who have produced verified results in two of the last three results, and no suspect or bad results in the last three results.
Mark Rose is offline   Reply With Quote
Old 2016-03-23, 23:54   #5
Uncwilly
6809 > 6502
 
Uncwilly's Avatar
 
"""""""""""""""""""
Aug 2003
101×103 Posts

17·523 Posts
Default

Can we also open discussion on what folks call Cat 0?

When an exponent in cat 1 expires (or a DC has a mismatch), it gets thrown back in the pool of cat 1's for another 90 days. This has happened twice to the same exponent. That stretches the time out to nearly 270 days.

Also, exponents that are picked up in Cat 2 and expire after showing progress then stalling become 'late' cat 1 entries. Then if they expire, they are hold outs.

A cat 1* or cat 1' or cat 0 would be for such stragglers (Cat 1 expirees and Cat 2's that expire into the lower part of Cat 1). They would get handed out to top 25 machines or to forumites that have pledged 25 day max turn around (on some sort of fair basis).
Uncwilly is offline   Reply With Quote
Old 2016-03-24, 01:05   #6
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

1BF716 Posts
Default

Quote:
Originally Posted by Uncwilly View Post
Can we also open discussion on what folks call Cat 0?
Yes, kinda. I'd like the discussion somewhat limited to LL rules. So question #1 is do we feel there is a need for a cat 0 in first-time LL testing?? If we agree there is a need, then let's come up with proposed rules keeping in mind that it would be great if such a rule would also make sense for DC.

That said, I like your solution. Something like a top 100 machine in addition to a reduced time limit. Possibly in addition to limiting it to users that have checked the smallest exponents box.
Prime95 is offline   Reply With Quote
Old 2016-03-24, 01:14   #7
retina
Undefined
 
retina's Avatar
 
"The unspeakable one"
Jun 2006
My evil lair

5,881 Posts
Default

Quote:
Originally Posted by Prime95 View Post
5) Don't piss off users ...
I think this is the most important rule. The rule to rule them all. Without the users you got nothing.
retina is offline   Reply With Quote
Old 2016-03-24, 02:26   #8
cuBerBruce
 
cuBerBruce's Avatar
 
Aug 2012
Mass., USA

2·3·53 Posts
Default

Quote:
Originally Posted by Prime95 View Post
1) Computer's rank ordered by LL/DC GHZ-days completed over the last 90 days.
I have no idea where my computers currently rank among all the GIMPS computers. Would the CPUs page be modified to show us the ranking of each computer in the last 90 days?

With the upward surge in the cat 1 exponent wavefront in the last few months, some older computers of mine are nearing the point where they might not finish them in the required time limit, using 1 core per worker. I've been thinking of switching them to two cores per worker, but under the proposed new rules, I don't know if that will allow them to qualify for cat 1 LLs. I may be forced to switch them to cat 2 work.

Quote:
Originally Posted by Prime95 View Post
Yes, kinda. I'd like the discussion somewhat limited to LL rules. So question #1 is do we feel there is a need for a cat 0 in first-time LL testing?? If we agree there is a need, then let's come up with proposed rules keeping in mind that it would be great if such a rule would also make sense for DC.
With at least four users poaching very low exponents in recent weeks to (apparently) prevent them from the possibility of another "lengthy" assignment, and at least two exponents being double-poached, I kind of think this suggests something is needed.
cuBerBruce is offline   Reply With Quote
Old 2016-03-24, 03:24   #9
axn
 
axn's Avatar
 
Jun 2003

7·683 Posts
Default

Quote:
Originally Posted by Prime95 View Post
1) The number of computers signed up for cat 1 & 2 assignments is too low.
I don't think this assertion is correct. From the other thread, there was a datapoint that suggested appr 25 completed cat1 LLs/day, so < 6 months to advance thru the cat 1 range. That seems entirely acceptable.
I don't know about cat 2 completion rate, but I have yet to hear an argument about what _should_ be the progress of cat 1/2 waves (and justifications for it). Perhaps we should start there.
Quote:
Originally Posted by Prime95 View Post
2) Some cat 1 assignments are going to computers that are too slow, suspect, or have a history of abandoning assignments.
This would definitely be a problem. IMO, the whole idea of category system was to avoid "sticking nails" exponents blocking milestones for a long time. Anything that can be done to reduce this problem would be good. But even here, things have improved drastically. Right now, we're looking at a few months to achieve a milestone instead of years.
Quote:
Originally Posted by Prime95 View Post
3) The very smallest exponents (a.k.a. cat 0) might need even stricter limits to advance milestones faster are discourage poaching.
Since the number of exponents that needs babysitting are small, manual intervention is perfectly acceptable. We don't need to automate every little thing.

Quote:
Originally Posted by Prime95 View Post
1) Progress on milestones faster
No. Keeping _regular_ progress of milestones is the goal (i.e. preventing extreme hold ups). Actual milestone progress is a function of the combined LL completion from all the categories. It might appear that cat 1 & 2 are the ones achieving milestones, but that is short-sighted view. If the cat 3 & 4 have done a good job, then cat 1 & 2 will go thru that range faster when they eventually get to it. Moving machines from cat3/4 to cat1/2 only helps with milestones in the short run. To reiterate "Keeping _regular_ progress of milestones is the goal"

Prime95: Noted and fixed in original post

Quote:
Originally Posted by Prime95 View Post
2) Reduce the vast number of available cat 1/2/3 assignments (i.e. compress the ranges if possible)
This is something that can be easily done. There is nothing sacrosanct about 4k/10k cutoffs. We can adjust it up/down based on the (perceived) progress of the milestones.

Quote:
Originally Posted by Prime95 View Post
3) Allow users with slower computers to get LL tests without getting in the way of milestones.
The correct focus!


Quote:
Originally Posted by Prime95 View Post
First up, do we need more data to help in our thinking?
Absolutely. I am yet to see an actual well-reasoned argument that there is a problem with the current system. It seems to be a case of "perfect is the enemy of good enough". More data can surely help.

Quote:
Originally Posted by cuBerBruce View Post
With at least four users poaching very low exponents in recent weeks to (apparently) prevent them from the possibility of another "lengthy" assignment, and at least two exponents being double-poached, I kind of think this suggests something is needed.
Four is a very small number. We have to ask ourselves if it is really worth it to address it.

Last fiddled with by Prime95 on 2016-03-24 at 22:22
axn is online now   Reply With Quote
Old 2016-03-24, 04:03   #10
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

3×3,109 Posts
Default

Quote:
Originally Posted by axn View Post
I don't think this assertion is correct. From the other thread, there was a datapoint that suggested appr 25 completed cat1 LLs/day, so < 6 months to advance thru the cat 1 range. That seems entirely acceptable I don't know about cat 2 completion rate, but I have yet to hear an argument about what _should_ be the progress of cat 1/2 waves (and justifications for it). Perhaps we should start there.
When you look at the current situation and see that there are 6,163 candidates in the Cat 1 and Cat 2 ranges awaiting to be assigned (plus another 1,013 Cat 2s about to be processed and released by GPU72 for assignment) out of the 10,000 candidates in the two ranges, it seems clear (at least to me) that something isn't optimal.

Please note (as also mentioned previously) that /very/ few Cat 2s are currently being assigned; almost all current Cat 2s were originally Cat 3s when assigned. Further, currently any Cat 2s which are assigned almost immediately become Cat 1s.

What this means is that at the current rate of ~27 completions of Cat 1s a day it's actually going to take ~370 days to finish up to where the Cat 3 range starts currently. Do we really want to still be working below 69M in a years time?
chalsall is offline   Reply With Quote
Old 2016-03-24, 04:42   #11
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

29×113 Posts
Default

Quote:
Originally Posted by axn View Post
...
Since the number of exponents that needs babysitting are small, manual intervention is perfectly acceptable. We don't need to automate every little thing.
...
Yeah, people may have noticed I have no problem with doing manual interventions when I see a user is working on an assignment and will miss their expiration by just a few days. In cases like that I would personally like to see manual interventions in the way of "stretching" the expiration by a few days to allow them to complete the work.

The alternative in those situations is to let it expire, go to someone else, but then the original assignee finishes anyway in a few days and the new person gets "poached".

Some of these things could be fixed by merely making sure only the "quality" machines are getting those things in the first place, but even then there will be the odd instance of a category 4 assignment taking really long and making its way into a category zero situation, where it's the last one left blocking a milestone (or several). It's happened.

I guess that would mean we were completing cat 1/2/3 work too fast, if a cat 4 assignment caught up with them before 1 year had lapsed. Nice problem to have?
Madpoo is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
PrimeNet Assignment Rules S485122 PrimeNet 7 2018-06-08 14:49
Modifications to DC assignment rules Prime95 PrimeNet 74 2017-01-18 18:36
Understanding assignment rules Fred PrimeNet 3 2016-05-19 13:40
Tweak to assignment rules Prime95 PrimeNet 11 2014-11-17 02:43
Tweaked assignment rules Prime95 PrimeNet 16 2012-03-19 20:24

All times are UTC. The time now is 11:28.

Tue Dec 1 11:28:45 UTC 2020 up 82 days, 8:39, 1 user, load averages: 2.31, 2.07, 1.72

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.