mersenneforum.org 3,748+ c204 Smaller-but-Needed
 Register FAQ Search Today's Posts Mark Forums Read

2021-10-23, 02:29   #133
SethTro

"Seth"
Apr 2019

2·5·41 Posts

Quote:
 Originally Posted by VBCurtis Pretty sure Seth open-sourced his code for that page. He's still active on the forum, maybe PM him and he's likely to help you set up the web-page monitor.
Thanks for remembering me. The code is used it at https://github.com/sethtroisi/factoring-ui
It also used a simple driver script to pull over ssh/scp the db file which looked something like this

Code:
date
cd "/home/eights/Projects/Factoring"
number="2_1165+.c217"
sshpass -p '<PASSWORD>' rsync -v <USERNAME>@<MACHINENAME>:/<PATH>/2_1165sieve3/${number}.{db,log} . db="${number}.db"
dated_db="${number}.$(date +%F).db"
echo "Testing for ${dated_db}" #if [[ ! -f "${dated_db}" ]]; then
#  echo
#  echo "Backing up ${db} to${dated_db}"
#  rsync --info=progress2 -v "${db}" "${dated_db}"
#fi

./log_processor.py -n "$number" -g 7.0e7 \ -s "db:sqlite3:$number.db" -l "\$number.log" \
--min-workunits 5 \
--host-startswith "localhost,DESKTOP,R5"
#    --client-joined "fivemack:birch:barley:butternut:oak:pineapple:pumpkin" \
the CADO team was willing to merged this but it needs to be tested on a MySQL system which I failed to find time to do that (if someone had such a system I would be grateful for the opportunity to test on it briefly).

 2021-10-24, 21:45 #134 kruoli     "Oliver" Sep 2017 Porta Westfalica, DE 84610 Posts Over the weekend, there have been a lot of timed out WUs and the server stopped today. I bumped the limit to 10,000. Additionally, there is now a status page.It should get refreshed every ten minutes. It should state if the server stopped/crashed/etc. If the whole machine hangs, the status page will not get updated anymore. If the status page itself cannot be reached, that means that my internet connection and/or my in-house web server is down. The percentage is not fully correct. 1.3G relations will be reachend when the displayed percentage is about 86 %. The ETA is not correct currently since I there is only one client that is not me. Before the restart today, the ETA was 21st of November for 100 %.
 2021-10-24, 22:58 #135 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 2×11×191 Posts My machines stopped communicating late Friday. I let them keep trying until last night. Many of them may have had late returns at some point. I went ahead and pulled them all down today. If you think things are settled, I'll start adding them back in a few at a time. If you're going to use a modified version of the script I posted to generate the status page for now, I'll add in some more details. Let me know if there is something you'd like me to try to add. Client details like SethTro's info would be a bit too much for me right now, but simple things should be attainable.
 2021-10-25, 10:12 #136 kruoli     "Oliver" Sep 2017 Porta Westfalica, DE 2×32×47 Posts Regarding Friday: There was a time when the server just idled. I do not know why. Without any interaction, it recovered and served WUs again. Weird. Beside that, everything seems to be back at normal. Now, there are already a lot of clients back. If I find some more time, I will try to use Seth's script. It is way more complex than our current approach, so for simplicity's sake I set up EdH's script to have something available while I experiment with other things.. In the mean time, if there is something important missing, I'll try to add it into the existing script. There is also the question of how much information should be publicly (not only for us, but everyone who can access my web server) available. I think there is a point where it could get too much for some of the participants.
 2021-10-25, 11:52 #137 EdH     "Ed Hall" Dec 2009 Adirondack Mtns 2·11·191 Posts I was thinking along the lines of adding in a line with the largest (and maybe smallest) number of relations returned for a WU and maybe a count of the unique clients that returned WUs within the previous 24 hours. This would not include identification of those clients. I will not be able to pursue any additions today. It was nice to see the ETA change from Apr to Dec. SethTro's approach added a sense of competitiveness. I'm sure that helped us reach the finish line sooner.
 2021-10-25, 12:38 #138 kruoli     "Oliver" Sep 2017 Porta Westfalica, DE 2·32·47 Posts These two additions will be a nice scripting exercise for me, so I hope you do not mind if I leapfrog you and try to implement them this evening. Yes, I see the point of competitiveness. If everyone is okay with this, we can show a ranking with Seth's script. Edit: I will also try to implement a feature to display relations found per day. Last fiddled with by kruoli on 2021-10-25 at 12:55 Reason: Additions.
2021-10-25, 20:24   #139
charybdis

Apr 2020

2·13·23 Posts

Quote:
 Originally Posted by kruoli Yes, I see the point of competitiveness. If everyone is okay with this, we can show a ranking with Seth's script.
Happy to have a ranking as long as it doesn't give the details of my individual clients. It should be pretty obvious to you which are mine

Edit: should be fine to do the switch to adjust_strategy=2 now.

Last fiddled with by charybdis on 2021-10-25 at 20:26

 2021-10-25, 20:53 #140 SethTro     "Seth" Apr 2019 19A16 Posts The only details of the client displayed are the name and statistics like last work unit finished, number of work units, .. In the case that a people wish names the name to be hidden it's easy to group a set of clients (or possible a single client) under a new name with --client-joined ":::"
 2021-10-25, 20:53 #141 VBCurtis     "Curtis" Feb 2005 Riverside, CA 5·1,033 Posts I don't mind my production info on a web page like Seth's, by forum name or client name. I'm still running the same two machine-names from that 2019 job, anyway.
 2021-10-26, 08:06 #142 kruoli     "Oliver" Sep 2017 Porta Westfalica, DE 2·32·47 Posts I have been asked to wait with changing the sieve strategy until somebody (who should know that he is meant) was able to took pull his machines. If you are ready, please PM me so that we can restart the server with the new settings!
2021-10-26, 14:04   #143
kruoli

"Oliver"
Sep 2017
Porta Westfalica, DE

84610 Posts

Quote:
 Originally Posted by charybdis Edit: should be fine to do the switch to adjust_strategy=2 now.
Done!

 Similar Threads Thread Thread Starter Forum Replies Last Post fivemack Factoring 3 2017-09-19 08:52 skan YAFU 6 2013-02-26 13:57 akruppa Factoring 114 2012-08-20 14:01 fortega Data 2 2005-06-16 22:48 marc Factoring 6 2004-10-09 14:17

All times are UTC. The time now is 17:24.

Fri Jan 28 17:24:24 UTC 2022 up 189 days, 11:53, 1 user, load averages: 0.90, 1.01, 1.18