Countdown meter

[gd_barnes]

Hi all,

We'd like to add a countdown meter that shows the # of pairs remaining to be tested on the 5th, 6th, and 7th drives to n=800K and to n=1M. This should give us some targets to shoot for. Later on, we'll add a countdown to n=900K for these drives as well as post many more drives here.

Latest update: 2010/10/06 09:30 GMT

5th drive k=400-600:

n=1.0M: 321,204  (78.38% done)

(Goal for completion to n=1M: Dec. 31st, 2010 (-> 3735 pairs/day to test))

6th drive k=600-800:

n=800K: 134,594  (81.56% done)
n=1.0M: 866,421  (40.72% done)
 
(Goal for completion to n=1M: Dec. 31st, 2011 (-> 1921 pairs/day to test))

7th drive k=800-1001:

n=800K: 136,357  (81.51% done)
n=1.0M: 874,283  (40.73% done)
 
(Goal for completion to n=1M: Dec. 31st, 2011 (-> 1939 pairs/day to test))

Counts include pairs remaining in the servers plus all remaining pairs not yet loaded into the servers.

I've asked the other admins to assist in keeping this thread up to date. I'd like to shoot for at least once every 2 days.

Gary

[gd_barnes]

I just now updated only the k=400-600 counts.

I attached how I came up with the counts. It's a little tricky because it's not immediately obvious how to come up with the pairs remaining in the PRPnet server. I just took the last few results and assumed there were no holes up to that point; then looked in the sieve file at what was remaining after those results. There probably are some holes but with Lennart the only one on there, I'm assuming that they are minimal and are filled fairly quickly.

Thanks for keeping them all up to date Karsten.

[henryzz]

Originally Posted by gd_barnes

I just now updated only the k=400-600 counts.

I attached how I came up with the counts. It's a little tricky because it's not immediately obvious how to come up with the pairs remaining in the PRPnet server. I just took the last few results and assumed there were no holes up to that point; then looked in the sieve file at what was remaining after those results. There probably are some holes but with Lennart the only one on there, I'm assuming that they are minimal and are filled fairly quickly.

Thanks for keeping them all up to date Karsten.

looking at the prpnet webpages should tell you how many pairs are remaining
it will be at most 20 minutes out

[mdettweiler]

Originally Posted by henryzz

looking at the prpnet webpages should tell you how many pairs are remaining
it will be at most 20 minutes out

Actually, the "count untested" figures are real-time; it's only the primes that can take up to 20 minutes to show up. The problem is that you have to add 50 separate figures together (one for each k) to get the overall # remaining.

My "real life" load lightened up a little and should stay so for the forseeable near future. Sometime soon I'm going to write some code in my PRPnet status page script to query the database and generate all the info we'd normally expect from an LLRnet status page. Namely, I'll try to get the following:
-Lowest outstanding k/n pair
-Highest outstanding k/n pair
-Total # of k/n pairs remaining to be tested
-Possibly a list of all pairs remaining a la LLRnet's knpairs.txt. We actually don't have these copied off to the web for LLRnet servers, but IronBits' old servers did this and it was rather handy.

Everything else not listed above is either already covered by the status page script, or is already provided by PRPnet's built-in status pages in the format we want.

[TimSorbet]

Originally Posted by mdettweiler

Actually, the "count untested" figures are real-time; it's only the primes that can take up to 20 minutes to show up.

Actually, I think he's right. If I'm not mistaken, those are part of the group stats that are recomputed every [some number] (20, probably...) minutes, or when you log in with prpadmin and issue "6. Recompute server statistics", so they aren't real-time unless you just ran that command.

[mdettweiler]

Originally Posted by Mini-Geek

Actually, I think he's right. If I'm not mistaken, those are part of the group stats that are recomputed every [some number] (20, probably...) minutes, or when you log in with prpadmin and issue "6. Recompute server statistics", so they aren't real-time unless you just ran that command.

Hmm...I'm not so sure about that. Over the last few days, I was watching a Sierp. base 6 range near completion on my personal server, and when I saw a batch was near completion I'd refresh the server's all.html page and it would immediately update the "Count untested" and "Count tested" columns, at the very least. I think the web pages are generated on demand from the DB.

What's not generated on demand, however, is a separate table in the DB containing all primes found, from which the web pages (and email sender) pull their prime-related data. That table is updated from the main tables every 10 minutes, which is why it can take up to that long for a newly found prime to be shown on the web pages and for the email to be sent out.

[TimSorbet]

Originally Posted by mdettweiler

Hmm...I'm not so sure about that.

I am. I just checked with my private PRPnet 3.2.4 setup. It doesn't update immediately. Quickly, (a matter of minutes, maybe even less than 20) but not immediately.

Originally Posted by mdettweiler

Over the last few days, I was watching a Sierp. base 6 range near completion on my personal server, and when I saw a batch was near completion I'd refresh the server's all.html page and it would immediately update the "Count untested" and "Count tested" columns, at the very least. I think the web pages are generated on demand from the DB.

What version are you running? IIRC the behavior was changed somewhat recently, so I wouldn't be surprised to see that in a < 3.2.0 (or so) version. If, however, you're running 3.2.4 or 3.2.5, I'm rather confused, as we're seeing rather different behaviors. There is no option to change this, (unless servertype changes it, which I have set to 3) so the only reasonable explanation I could think of then is that you have changed the code to make it like this.

Originally Posted by mdettweiler

What's not generated on demand, however, is a separate table in the DB containing all primes found, from which the web pages (and email sender) pull their prime-related data. That table is updated from the main tables every 10 minutes, which is why it can take up to that long for a newly found prime to be shown on the web pages and for the email to be sent out.

Not 100% sure on this one, but if you have unhideprimehours set to 0 and it says it's hiding one (instead of seeming to be ignorant of a prime at all), that should've been fixed in 3.2.4. 3.2.4's changelog includes "If unhideprimehours is set to 0, show primes immediately on user_primes.html page." I know that was a bug where it would act like you had unhideprimehours set to something like 10 minutes even if you have it set to 0 (hours).

[mdettweiler]

Originally Posted by Mini-Geek

I am. I just checked with my private PRPnet 3.2.4 setup. It doesn't update immediately. Quickly, (a matter of minutes, maybe even less than 20) but not immediately.

What version are you running? IIRC the behavior was changed somewhat recently, so I wouldn't be surprised to see that in a < 3.2.0 (or so) version. If, however, you're running 3.2.4 or 3.2.5, I'm rather confused, as we're seeing rather different behaviors. There is no option to change this, (unless servertype changes it, which I have set to 3) so the only reasonable explanation I could think of then is that you have changed the code to make it like this.

I'm running 3.2.5 on servertype=1 (Sierpinski/Riesel).

But, yeah, I'm beginning to wonder if you're right--yesterday I saw it was almost done with finishing Riesel base 182 and there were (as I recall) 14 pairs currently assigned to clients, but the "count untested" column said 18. Within a few minutes I refreshed the page and saw the pending tests were down to 10, and "count untested" had updated to reflect that as well.

Not 100% sure on this one, but if you have unhideprimehours set to 0 and it says it's hiding one (instead of seeming to be ignorant of a prime at all), that should've been fixed in 3.2.4. 3.2.4's changelog includes "If unhideprimehours is set to 0, show primes immediately on user_primes.html page." I know that was a bug where it would act like you had unhideprimehours set to something like 10 minutes even if you have it set to 0 (hours).

I do know for a fact that I've seen the "there are hidden primes" message on the port 9000 public server after some of Lennart's primes, and the primes then appeared within 10 minutes. That server has been running 3.2.5 from the start with unhideprimehours=0.

I have all the servers I administrate set to debuglevel=1, despite the huge logfiles it produces in 3.x versions, since I've been bitten way too many times by issues that showed up that could have been debugged if only I had more info on what went on. On this mode I've actually observed the server checking each k in the database for primes every 10 minutes. (That's the primary thing that seems to make the logfiles grow...)

[kar_bon]

I've updated post #1 and added two graphs.

[gd_barnes]

Karsten,

When you get a chance, can you update post 1 here? I'm curious to see how much effect the rally had on our goal for reaching n=1M for k=400-600 this year. My guess is that we are now in pretty good shape.

Having all k<600 at n>=1M will be great!


Gary

[kar_bon]

Working on it, but it could take some time:

I've to catch more than 600 double-tests due to the issue during the rally.
So delete them from the result-files and documenting for history.