NPLB Database - Page 5

IronBits · 2009-01-08, 07:54

Happy Belated B-day BlisteringSheep! May you have many many more!

I appreciate the patience everyone has had as I dial in the stats system!

I've been joining a lot of scripts together so I have less physical scripts to deal with.
Initially we only had one port, so everything was rather simple.
Then we added another, then another, then there was crus (gone!) and then another port was added and then one was put into retirement, and another one is on hold for future drives. (8000) which adds up to a lot of darn scripts in two directories.

Instead of having a script for each port, x2, one for hourly updates and one for midnight rollovers, for each specific task, I have been changing them so that there is one script per task, that can do both hourly and rollover tasks.

Example tasks are, counting of pairs, looking for primes, email auto-notification, web page updates, hourly stats and yet another that copies the files around so they are available to anyone on the website so we can all keep an eye on what they are, or are not, doing.

I have re-written the majority of it now so that each task can do all ports. I also converted as much DOS batch file work into vbscript that I can at this stage.
Now I'm adding the rollover part into each one of those task scripts so that one task script can do both hourly updates and handle midnight rollover tasks. fun! ;)

When the dust settles, I'm hoping to port the whole thing over to perl, put it on the Server, throw it into cron and be done with it.

BlisteringSheep · 2009-01-10, 03:59

Thank you all! I wish I could say I was wiser, but I don't think my wife & kids would necessarily agree...

I hope my sporadic bursts of llrnet-ing aren't too disruptive. Of course, ol' reliable (

) has still been trudging along on port 400, about 8 hours/test.

Code:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 5
model           : 4
model name      : Pentium MMX
stepping        : 3
cpu MHz         : 199.908
fdiv_bug        : no
hlt_bug         : no
sep_bug         : no
f00f_bug        : yes
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr mce cx8 mmx
bogomips        : 398.95

mdettweiler · 2009-01-10, 05:34

Quote:

Originally Posted by BlisteringSheep

Thank you all! I wish I could say I was wiser, but I don't think my wife & kids would necessarily agree...

I hope my sporadic bursts of llrnet-ing aren't too disruptive. Of course, ol' reliable (

) has still been trudging along on port 400, about 8 hours/test.

Code:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 5
model           : 4
model name      : Pentium MMX
stepping        : 3
cpu MHz         : 199.908
fdiv_bug        : no
hlt_bug         : no
sep_bug         : no
f00f_bug        : yes
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr mce cx8 mmx
bogomips        : 398.95

You might want to try port 9000--the tests are a lot smaller, and your Pentium would probably be able to finish them in only an hour or less.

em99010pepe · 2009-01-10, 10:36

IB,

The latest IB9000 results.txt file from here is not correct. It should be a file with >15M in size. When you get the time please check the script. Thank you.

Carlos

IronBits · 2009-01-10, 14:58

That is correct, the database did not get updated last night.
We are in the process of converting everything over to perl, and putting all the tasks onto the Servers to do everything.
AMDave is looking into the fetch script that we use to get Gary's csv files.
I'll make a manual stats run this morning and we'll get it all updated.

IronBits · 2009-01-10, 18:08

Database stats updated now.

IronBits · 2009-01-12, 06:19

I believe the Database stats are updated every 15 minutes now, for all server ports including Gary's.

mdettweiler · 2009-01-12, 16:32

Quote:

Originally Posted by IronBits

I believe the Database stats are updated every 15 minutes now, for all server ports including Gary's.

I'm not quite sure what you mean in this case by "database stats"--do you mean that the results are imported into the database every 15 minutes?

IronBits · 2009-01-13, 00:40

I think we decided to drop back to once per hour.
Sure, we can get a copy of your 'copy off results.txt' and my 'copy of results.txt' making the database pretty much live now.
In fact, you can check out the latest new addition by going directly to the stats server now.
http://stats.ironbits.net/statsnew/

mdettweiler · 2009-01-13, 01:34

Quote:

Originally Posted by IronBits

I think we decided to drop back to once per hour.
Sure, we can get a copy of your 'copy off results.txt' and my 'copy of results.txt' making the database pretty much live now.
In fact, you can check out the latest new addition by going directly to the stats server now.
http://stats.ironbits.net/statsnew/

Nice!

One thing to keep in mind: the GB servers copy off their results files at 00:01 every day, but from the timestamps on the results files range from 21:56 to 00:01, so I'm not sure exactly when things are happening. In fact, I'm wondering if the times are getting messed up somehow...because for the status page scripts, cron always seems to be right on the nose regarding what time it runs the script, so I can't imagine why it would mess up this egregiously for the results file script.

Thus, I'm not quite sure if using the 15-minute results file updates can be trustworthy for importing stuff into the database...it's possible that, depending on when you're copying off my results files, you may potentially miss 15 or, on a bad day, 30 minutes of results. Thus, only the daily results files would be sure to include everything.

I'll have to investigate the results file copying-off process more closely to see exactly what's going on with those timestamps...some of them are looking pretty weird. I mean, one of them was copied off at 21:56? What the heck? It's not like there was anything weird like a daylight savings time change on Dec. 30th (the day when the 21:56 results file occurred).

I'm wondering if it actually *is* being copied off at the correct time, but somehow the times are being messed up after the fact...I don't know. Anyone else got any ideas on why this might be happening?

IronBits · 2009-01-13, 01:52

The daily results files are also processed, to make sure we don't miss anything.
MySQL prevents the duplications based on k and n values :)
The dates/times are taken from within the results.txt files themselves, not the date of the results.txt file itself...

Which gives a tad heartburn being that your server is one timezone behind me ;)

2009-01-08, 07:54	#45
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 459₁₆ Posts	Happy Belated B-day BlisteringSheep! May you have many many more! I appreciate the patience everyone has had as I dial in the stats system! I've been joining a lot of scripts together so I have less physical scripts to deal with. Initially we only had one port, so everything was rather simple. Then we added another, then another, then there was crus (gone!) and then another port was added and then one was put into retirement, and another one is on hold for future drives. (8000) which adds up to a lot of darn scripts in two directories. Instead of having a script for each port, x2, one for hourly updates and one for midnight rollovers, for each specific task, I have been changing them so that there is one script per task, that can do both hourly and rollover tasks. Example tasks are, counting of pairs, looking for primes, email auto-notification, web page updates, hourly stats and yet another that copies the files around so they are available to anyone on the website so we can all keep an eye on what they are, or are not, doing. I have re-written the majority of it now so that each task can do all ports. I also converted as much DOS batch file work into vbscript that I can at this stage. Now I'm adding the rollover part into each one of those task scripts so that one task script can do both hourly updates and handle midnight rollover tasks. fun! ;) When the dust settles, I'm hoping to port the whole thing over to perl, put it on the Server, throw it into cron and be done with it. Last fiddled with by IronBits on 2009-01-08 at 07:55

2009-01-10, 03:59	#46
BlisteringSheep Oct 2006 On a Suzuki Boulevard C90 2×3×41 Posts	Thank you all! I wish I could say I was wiser, but I don't think my wife & kids would necessarily agree... I hope my sporadic bursts of llrnet-ing aren't too disruptive. Of course, ol' reliable () has still been trudging along on port 400, about 8 hours/test. Code: processor : 0 vendor_id : GenuineIntel cpu family : 5 model : 4 model name : Pentium MMX stepping : 3 cpu MHz : 199.908 fdiv_bug : no hlt_bug : no sep_bug : no f00f_bug : yes coma_bug : no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr mce cx8 mmx bogomips : 398.95

2009-01-13, 01:52	#55
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 3·7·53 Posts	The daily results files are also processed, to make sure we don't miss anything. MySQL prevents the duplications based on k and n values :) The dates/times are taken from within the results.txt files themselves, not the date of the results.txt file itself... Which gives a tad heartburn being that your server is one timezone behind me ;) Last fiddled with by IronBits on 2009-01-13 at 01:54

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
NPLB.net down	Mini-Geek	No Prime Left Behind	17	2009-09-25 18:51
Servers for NPLB	gd_barnes	No Prime Left Behind	0	2009-08-10 19:21
NPLB on BOINC?	jasong	No Prime Left Behind	17	2008-12-01 10:25
two ideas for NPLB	Mini-Geek	No Prime Left Behind	16	2008-03-01 23:32
One month of NPLB	em99010pepe	No Prime Left Behind	5	2008-02-24 14:37

2009-01-10, 10:36	#48
em99010pepe Sep 2004 2×5×283 Posts	IB, The latest IB9000 results.txt file from here is not correct. It should be a file with >15M in size. When you get the time please check the script. Thank you. Carlos

2009-01-10, 14:58	#49
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 459₁₆ Posts	That is correct, the database did not get updated last night. We are in the process of converting everything over to perl, and putting all the tasks onto the Servers to do everything. AMDave is looking into the fetch script that we use to get Gary's csv files. I'll make a manual stats run this morning and we'll get it all updated.

2009-01-10, 18:08	#50
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 459₁₆ Posts	Database stats updated now.

2009-01-12, 06:19	#51
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 459₁₆ Posts	I believe the Database stats are updated every 15 minutes now, for all server ports including Gary's.

2009-01-13, 00:40	#53
IronBits I ♥ BOINC! Oct 2002 Glendale, AZ. (USA) 10001011001₂ Posts	I think we decided to drop back to once per hour. Sure, we can get a copy of your 'copy off results.txt' and my 'copy of results.txt' making the database pretty much live now. In fact, you can check out the latest new addition by going directly to the stats server now. http://stats.ironbits.net/statsnew/