mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > No Prime Left Behind

Reply
 
Thread Tools
Old 2021-08-20, 04:44   #595
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

33×17×23 Posts
Default

Stats are back in maintenance mode.
gd_barnes is offline   Reply With Quote
Old 2021-08-20, 14:10   #596
AMDave
 
AMDave's Avatar
 
Jan 2006
deep in a while-loop

2·32·37 Posts
Default

At some point there was an event that caused corruption the MySQL and that corruption was retained in the backups so restoring from back up could not excise the problem.
The old prpnet1470 database was already unrecoverable, but I had to forcibly remove it to restore integrity.

The processing logs were really quite perplexing until the timestamps revealed the picture.
Somewhere in MySQL was enough of a problem to cause the sql steps to take longer and longer to complete, until they were too long.
Eventually the daily and hourly processes overlapped.
As one process created the update tables another one removed them.
That is how the stats summaries disappeared.

I re-optimised the configuration of the MySQL server itself,
I have re-optimised all of the databases: both ports and stats.
It was close to optimal but I have refined it further.
Bounced it and bench marked it several times.
Its looking better, for an old dog. (It is a great grandfather now!)

I have merged the daily and hourly stats processes so there is now only one process.
That comes at a small cost in processing time.
The original hourly process target was 3 minutes and the daily process time was 10 minutes.
The single process will now run every hour including midnight and take up to 20 minutes depending on how many results you can pump into it :)
I have removed the weakest point in the process (the 'blind-update-and-switch' summary step) which is where the web stats were disappearing from.

And finally I have added a 'hard' semaphore that prevents the process from running over itself.
So the process will attempt to run each hour, as before, but if a massive slowdown occurs again it will run to completion without interruption and will not start again until the start of the hour after it has completed.

I will have to come back and change that hard semaphore to a soft semaphore at a more convenient time.

Back out of maintenance mode.
Monitoring the log files closely.

Last fiddled with by AMDave on 2021-08-20 at 14:11
AMDave is offline   Reply With Quote
Old 2021-08-20, 20:09   #597
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

33×17×23 Posts
Default

Thank you very much, Dave, for that tremendous effort.

I had seen that prpnet1470 caused a big issue over a year ago. Hence I never attempted to run it again. It's too bad that its corruption caused the backups to become corrupted.

To everyone: The NPLB stats pages look good now!

Last fiddled with by gd_barnes on 2021-08-20 at 20:09
gd_barnes is offline   Reply With Quote
Old 2021-08-27, 05:19   #598
odicin
 
Sep 2011
Potsdam, Germany

11111002 Posts
Default

Very nice... I noticed that there are still some incorrect entries on the free-dc stats page (Top 10 Users, Show All entries). Possibly, the export for this also has to be touched again.


Regards Odi
odicin is offline   Reply With Quote
Old 2021-09-08, 12:44   #599
AMDave
 
AMDave's Avatar
 
Jan 2006
deep in a while-loop

2×32×37 Posts
Default

I have posted on FDC asking Bok to refresh the project stats from the latest NPLB extracts
AMDave is offline   Reply With Quote
Old 2021-09-09, 11:47   #600
AMDave
 
AMDave's Avatar
 
Jan 2006
deep in a while-loop

2×32×37 Posts
Default

Bok advised that his new stats server no longer refreshes the stats of non-BOINC projects.
However he is looking into making NPLB an exception.
There is a bit of work and goodwill involved so if it happens don't forget to tell him thank you :)
AMDave is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
prime64 causing severe unresponsiveness Freightyard Software 14 2011-11-11 00:22

All times are UTC. The time now is 09:02.


Sun Dec 5 09:02:27 UTC 2021 up 135 days, 3:31, 0 users, load averages: 1.25, 1.67, 1.60

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.