A little help please?

Prime95 · 2005-02-21, 00:25

You guys are great at downloading the databases, assignments, cleared reports and analyzing the data.

Would anyone be interested in sending me a report of exponents that have either

a) been successfully double-checked but are still assigned to someone
b) had an error-free first test and are still assigned to someone for a first test
c) trial factoring has finished and it is still assigned to someone.

These SQLServer crashes are keeping assignments from being cleared. If I knew the exponents I could delete them from the server database to prevent them from expiring and being reassigned.

I'd need these reports after every update to the databases on status.htm

nfortino · 2005-02-21, 04:17

I could do that. I assume you want these reports by e-mail, but I will post my original results here incase anyone wants to double check. I should be able to add it to automated scripts I have already by the next data file update (the delay in the data file update due to M42 should help). The attached files list exponents satisfying these criteria using the February 12, 2005 data files and 21 Feb 2005 01:00 (Feb 20 2005 6:00PM Pacific) status.txt.

A couple of notes. For the purposes of filtering error prone tests, an error prone machine was defined as a machine with a confirmed error rate of greater than .333. I have not yet looked up if this definition is exactly the same as was used when error prone exponents were released as first time tests, but I know it is close.
I am not sure whether I correctly understand criterion c. I have worked off the assumption you meant exponents for which a factor has been found, but are still being tested.
I am surprised by how many exponents satisfy criterion b, but I have checked and re-checked, and my results seem to be accurate.

garo · 2005-02-21, 15:46

nfortino and George,
Yes, quite a few tests from "error-prone" machines did not have any errors and were re-released for a seconf first time tests and therefore show up in yout list. You may want to search for a post by GP2 which listed the exponents that were supposed to be released under this criterion and exclude them from your list.

PrimeFun · 2005-02-21, 17:18

The exponents in criteria B) would just be converted from first time tests to double checks by the server (right?). I have about 70 or so of these assigned my account at the moment (all < 13M).

shaneamy

nfortino · 2005-02-21, 19:16

Quote:

Originally Posted by garo

nfortino and George,
Yes, quite a few tests from "error-prone" machines did not have any errors and were re-released for a seconf first time tests and therefore show up in yout list. You may want to search for a post by GP2 which listed the exponents that were supposed to be released under this criterion and exclude them from your list.

I had already done that. All tests that were been returned by an error prone machine (error rate of greater than .333 and at least 2 bad tests) were removed along with tests with tests with harmful error codes when determining whether a good first time check had been run. While there is in fact some overlap between the list I have provided and the exponents intentionally released for early double check, there are only 16, and this is due to machines formerly flagged as error-prone returning good results and being removed from the error prone list. This still leaves the reason for there being 1588 exponents satisfying criterion b unexplained.

garo · 2005-02-22, 09:44

OK then a possible explanation is the recent server synchronizations. It seems that after each synchronization the server "forgets" tests that have already been completed. So the following situation can cause tests in category B. Assume a test expired and was reassigned and thus had two assignees at one time or the other. Now one of them completes the test and returns it to the server and then the server synchs. Now, if the other assignee reports to the server saying that the test is in progress the server does not send a message back saying that the test has been already done but instead inserts this exponent in the assigned list again.
This behaviour has been occuring rather frequently since last September when server synchs started happening on a monthly basis.

moo · 2005-02-23, 01:32

i wrote something that can get what p1 factors still need to be run its written in torque so a copy of the torque engine needs to be obtained ;) i used tribes 2 with its dos server but it takes 28 days to complete...

dswanson · 2005-02-23, 05:54

I'm the guilty party for the large majority of nfortino's "category B" exponents less than 13M. Every single one of the ones assigned to me or shaneamy is actually an exponent that requires a triplecheck.

Each time George updates the stats files I compare the exponents in HRF3.TXT against the exponents in the cleared exponents file. It's very simple to identify recently-returned results that don't match prior results. These are exponents that need a triplecheck.

When George resynchs the database each month he doesn't immediately move exponents needing triplechecks into the "available for doublechecking" queue. Because of the resynch they're no longer "cleared". But they're not assigned, either; they're just in limbo. So I toss them into a worktodo file and check in to the server. The server recognizes that they're not assigned to anyone, so it gives them to me, but it identifies them as first-time tests rather than doublechecks.

I don't see this as a particular problem. When I report the result back it still ends up being treated as if it were a doublecheck. It just makes the status report look a little odd. Frankly, I'd be disappointed if this "feature" were fixed, since it makes it easy for me and shaneamy to knock off the low triplechecks very early.

As for nfortino's observation that there are lots of category B exponents: The first time I tried this trick (sometime last fall) I collected somewhat more than 200 exponents in the 11M-12M range. The remains from that initial stash (mostly above 11.7M) are still in my and shaneamy's work queue. They should all be done within a month or so.

The "category A" exponents (cleared but still assigned) belong to a mishmash of users. My guess is that something very similar to what I've described happened with these as well: A user checks into the server with an exponent that's already been doublechecked and removed from the cleared exponent list. Since it is neither "cleared" nor assigned, the server lets them have it, marking it as a first-time test. This is the process garo describes.

If they eventually complete the exponent, it will end up being a redundant triplecheck. I'm not sure what happens to these exponents if they expire instead -- Does the server automatically reassign them, or do they go back into limbo? I always check the exponents the server assigns to me to make sure they're not already doublechecked. I've run across a few that are, and I report these back to George. I've never bothered to check if these are "category A" exponents. I'll try to remember to do so next time it happens.

garo · 2005-02-23, 11:05

Yes, the server does automatically reassign them. I've had quite a few of these recently as well. And I think that's the reason why George is asking for the lists as he doesn't want them to be reassigned after expiry thus causing redundant triple and quadruple checks.

Consider the case when a user checks in once in 6 months. The way the server is currently configured it is quite possible that every time the exponent expires, it is reassigned, completed, synched out of Primenet and then reported as being in progress again only to re-expire and be reassigned....

Prime95 · 2005-02-23, 21:41

Thanks nfortino. With a little tweaking, maybe we can get the right set of data mined.

I took your has-been-double-checked and has-been-factored data and removed those exponents from the server. So, one useful file on a regular basis would be these two files combined:

All exponents that have been either successfully double-checked or a factor has been found and are still assigned to a user.

I'm a little uncertain what to do with the wrong-queue data. I hate to just delete the exponents from the server as those doing error checks or dswanson's triple-checks won't get CPU credit. I might try to turn off the "available for LL testing flag". Hopefully, this would prevent the exponent from being reassigned if it expires.

I also need data for exponents that are currently assigned for factoring and have already been trial factored up to their prime95 limit.

Thanks again for everyone's help. It's been a busy week here...

Prime95 · 2005-02-23, 22:08

OK, I've tried clearing the available-for-LL-assignment flag on the 1400 exponents in the wrong queue list that are over 13.5 million. Can someone watch these exponents and see if they get reassigned if they expire?

Dswanson, when you check in your triple-checks have you verified that they don't get reassigned as double-checks? I wouldn't think so, but I want to make sure.

2005-02-21, 00:25	#1
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2×53×71 Posts	A little help please? You guys are great at downloading the databases, assignments, cleared reports and analyzing the data. Would anyone be interested in sending me a report of exponents that have either a) been successfully double-checked but are still assigned to someone b) had an error-free first test and are still assigned to someone for a first test c) trial factoring has finished and it is still assigned to someone. These SQLServer crashes are keeping assignments from being cleared. If I knew the exponents I could delete them from the server database to prevent them from expiring and being reassigned. I'd need these reports after every update to the databases on status.htm

2005-02-23, 05:54	#8
dswanson Aug 2002 2³·5² Posts	I'm the guilty party for the large majority of nfortino's "category B" exponents less than 13M. Every single one of the ones assigned to me or shaneamy is actually an exponent that requires a triplecheck. Each time George updates the stats files I compare the exponents in HRF3.TXT against the exponents in the cleared exponents file. It's very simple to identify recently-returned results that don't match prior results. These are exponents that need a triplecheck. When George resynchs the database each month he doesn't immediately move exponents needing triplechecks into the "available for doublechecking" queue. Because of the resynch they're no longer "cleared". But they're not assigned, either; they're just in limbo. So I toss them into a worktodo file and check in to the server. The server recognizes that they're not assigned to anyone, so it gives them to me, but it identifies them as first-time tests rather than doublechecks. I don't see this as a particular problem. When I report the result back it still ends up being treated as if it were a doublecheck. It just makes the status report look a little odd. Frankly, I'd be disappointed if this "feature" were fixed, since it makes it easy for me and shaneamy to knock off the low triplechecks very early. As for nfortino's observation that there are lots of category B exponents: The first time I tried this trick (sometime last fall) I collected somewhat more than 200 exponents in the 11M-12M range. The remains from that initial stash (mostly above 11.7M) are still in my and shaneamy's work queue. They should all be done within a month or so. The "category A" exponents (cleared but still assigned) belong to a mishmash of users. My guess is that something very similar to what I've described happened with these as well: A user checks into the server with an exponent that's already been doublechecked and removed from the cleared exponent list. Since it is neither "cleared" nor assigned, the server lets them have it, marking it as a first-time test. This is the process garo describes. If they eventually complete the exponent, it will end up being a redundant triplecheck. I'm not sure what happens to these exponents if they expire instead -- Does the server automatically reassign them, or do they go back into limbo? I always check the exponents the server assigns to me to make sure they're not already doublechecked. I've run across a few that are, and I report these back to George. I've never bothered to check if these are "category A" exponents. I'll try to remember to do so next time it happens. Last fiddled with by dswanson on 2005-02-23 at 05:59

2005-02-21, 15:46	#3
garo Aug 2002 Termonfeckin, IE 5314₈ Posts	nfortino and George, Yes, quite a few tests from "error-prone" machines did not have any errors and were re-released for a seconf first time tests and therefore show up in yout list. You may want to search for a post by GP2 which listed the exponents that were supposed to be released under this criterion and exclude them from your list.

2005-02-21, 17:18	#4
PrimeFun Jul 2003 11110₂ Posts	The exponents in criteria B) would just be converted from first time tests to double checks by the server (right?). I have about 70 or so of these assigned my account at the moment (all < 13M). shaneamy

2005-02-22, 09:44	#6
garo Aug 2002 Termonfeckin, IE 2²×691 Posts	OK then a possible explanation is the recent server synchronizations. It seems that after each synchronization the server "forgets" tests that have already been completed. So the following situation can cause tests in category B. Assume a test expired and was reassigned and thus had two assignees at one time or the other. Now one of them completes the test and returns it to the server and then the server synchs. Now, if the other assignee reports to the server saying that the test is in progress the server does not send a message back saying that the test has been already done but instead inserts this exponent in the assigned list again. This behaviour has been occuring rather frequently since last September when server synchs started happening on a monthly basis.

2005-02-23, 01:32	#7
moo Jul 2004 Nowhere 809 Posts	i wrote something that can get what p1 factors still need to be run its written in torque so a copy of the torque engine needs to be obtained ;) i used tribes 2 with its dos server but it takes 28 days to complete...

2005-02-23, 11:05	#9
garo Aug 2002 Termonfeckin, IE 2²×691 Posts	Yes, the server does automatically reassign them. I've had quite a few of these recently as well. And I think that's the reason why George is asking for the lists as he doesn't want them to be reassigned after expiry thus causing redundant triple and quadruple checks. Consider the case when a user checks in once in 6 months. The way the server is currently configured it is quite possible that every time the exponent expires, it is reassigned, completed, synched out of Primenet and then reported as being in progress again only to re-expire and be reassigned....

2005-02-23, 21:41	#10
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2×53×71 Posts	Thanks nfortino. With a little tweaking, maybe we can get the right set of data mined. I took your has-been-double-checked and has-been-factored data and removed those exponents from the server. So, one useful file on a regular basis would be these two files combined: All exponents that have been either successfully double-checked or a factor has been found and are still assigned to a user. I'm a little uncertain what to do with the wrong-queue data. I hate to just delete the exponents from the server as those doing error checks or dswanson's triple-checks won't get CPU credit. I might try to turn off the "available for LL testing flag". Hopefully, this would prevent the exponent from being reassigned if it expires. I also need data for exponents that are currently assigned for factoring and have already been trial factored up to their prime95 limit. Thanks again for everyone's help. It's been a busy week here...

2005-02-23, 22:08	#11
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2·53·71 Posts	OK, I've tried clearing the available-for-LL-assignment flag on the 1400 exponents in the wrong queue list that are over 13.5 million. Can someone watch these exponents and see if they get reassigned if they expire? Dswanson, when you check in your triple-checks have you verified that they don't get reassigned as double-checks? I wouldn't think so, but I want to make sure.