Planning & Coordination for 2,2330L

[thome]

Originally Posted by SethTro

I'm going to start saving daily copies of the database for cloudygo. At ~100mb I can store 180 days for 18 gigs and reduce this concern.

Thome, would you be interested in a new server flag that saves a last N days of the database? I'm imagining copying it every night and deleting any copies older than N days, or maybe saving a new copy every X WUs (so that during periods of no work it doesn't delete the older copies)

That would rather be a functionality of the database management system that probably exists and could be leveraged.

Now there are multiple ways a cado-nfs database can have problems. It's very rare that the database becomes corrupt from the DBMS point of view, and I'm not really talking of database corruption in that sense (for which DBMS-level backups seem to be the way to go). But for example, when the number of failed WUs increases a lot, cado-nfs refuses to resume. Likewise if some of the counts diverge (I think it's in the 'sieving' table -- these counts shouldn't exist in the first place, IMO). There's no doc, not even a cookbook, as to what can be done in such cases. Lacking that, from the user point of view, a database that cado-nfs.py refuses to work with for this kind of reason is just as useful in the end of the day as a corrupt database...

Originally Posted by SethTro

The whole site is around 1000 lines of code (https://github.com/sethtroisi/factoring-ui), if you think that it or something similar would be interesting to add to the project I'm happy to write up a proposal for that and work with you to get it commit.

We can discuss that, yes.

[Mumps]

Originally Posted by VBCurtis

We're back up and running; our host box did power up on its own.

And here we are now with "Last Work Unit Returned" from over an hour ago. *sigh* :)

[VBCurtis]

Originally Posted by Mumps

And here we are now with "Last Work Unit Returned" from over an hour ago. *sigh* :)

Pretty sure that was a website downtime- note the "last updated" time matched the "last workunit returned" time. The server continued to issue and receive WUs all morning.

[VBCurtis]

We had another hiccup this morning; one of the team caught it and kicked the server again.

I've just changed to I=15, as of Q=180.07M. Workunits are now 10k in length.

I=16 produced 1304M raw relations from Q=8-180M, for an average yield of 7.58.

I=15 is expected to average yield of 2.0 from 180 to ~880M for the other 1400M relations.

[EdH]

With the change to I=15, I'm looking at possibly adding a couple clients that would not be 24 hour machines. Since the server disregards and reassigns overdue WUs, suspension seems of no value. This means interruption by CTRL-C or letting an las session complete after suspension, when it will just get discarded.

I may insert a break in the client script between sieving runs, triggered by a stop file (as I have done elsewhere), but is there a clean way to tell the client to stop at the end of a WU? Possibly a switch I haven't discovered?

[SethTro]

Originally Posted by EdH

With the change to I=15, I'm looking at possibly adding a couple clients that would not be 24 hour machines. Since the server disregards and reassigns overdue WUs, suspension seems of no value. This means interruption by CTRL-C or letting an las session complete after suspension, when it will just get discarded.

I may insert a break in the client script between sieving runs, triggered by a stop file (as I have done elsewhere), but is there a clean way to tell the client to stop at the end of a WU? Possibly a switch I haven't discovered?

I'm actually working on a patch for cado to do exactly that (github) sadly it's not quite as easy as I expected (signals get passed to children so I'm going to have to open a raw tty which I really didn't want to do)

[xilman]

Originally Posted by SethTro

I'm actually working on a patch for cado to do exactly that (github) sadly it's not quite as easy as I expected (signals get passed to children so I'm going to have to open a raw tty which I really didn't want to do)

Can't you pass an otherwise unused signal to the child and have its signal handler do The Right Thing?

[ET_]

never mind...

[thome]

is there a clean way to tell the client to stop at the end of a WU? Possibly a switch I haven't discovered?

-single maybe ?

[EdH]

Originally Posted by thome

-single maybe ?

I am trying this "--single" right now. Thanks! If this works, I can build a short script around it.

My description was actually a bit poor. What I really wanted was a way to have the client run in its normal loop for many iterations, but break it at the end of a WU when it became close to suspend time. I should be able to do just that.

Unfortunately, each run of single will create a separate instance for the cloudygo site to track, since each instance will be seen as a different client.


I will also have to look at the github version posted by SethTro.

[Mumps]

Yay! We crossed 50% completion! Keep it up folks!