CADO NFS

[VBCurtis]

Yes, but it's hard to understand how. It seems to be the database side that's failing, yet bigger numbers gather relations and hand out WUs more slowly than smaller jobs. So, things *should* function better, because the rate of transactions is smaller... at least, I would think? (well, I do think... but I'm wrong)

In the Cunningham 2330L C207 thread last summer, one of the CADO contributors suggested a different database backend. I have zero experience with such things, so I didn't consider exactly what he suggested, but that's why I think it's something about the server/backend rather than a sieve parameter; certainly, one might trigger the other for reasons unknown.

[EdH]

I think I might have to set my laziness aside and join the CADO-NFS mailing list to see if I can gain a little more insight into the package. I'm a very amateur programmer, so I can't readily figure out in depth programs, but I do sometimes try alterations.

A "pet peeve" with CADO-NFS is its manner of crashing (dying) instead of gracefully ending, if you use "tasks.filter.run = false." Now I have 30+ pieces of "farm" equipment that are still looking for assignments. I'll probably have to restart without the "false" and crash that run, to "gracefully" stop the clients, rather than going to each one to issue CTRL-C..

Another difficulty is that I was unable to get a remote Colab install to run las on its own because it didn't like something about the roots1 data.

I do realize it's ME, but a lot of these things I keep trying, appear to be just opposite what the programmers intended. If I just had a better understanding (and a longer attention span). . .

[EdH]

This is speculation, but I think I have discovered why CADO-NFS stops issuing WUs.

If WUs are tardy, the server tries to wait for the late WUs until timedout is reached, whereas it reissues them. But, it appears that if too many are late, it stops sending all others until they are caught up.

I have changed my timedout to 43200 (12 hours) to cover the sleeping time for some of my machines and have had no noticeable instances of WUs not being handed out for my current run.

Let's see if this theory is disproven now that I have posted it. . .

[Dylan14]

So I ran into a problem with the cado-nfs-client.py file, in particular, it couldn't find my las executable:

FileNotFoundError: [Errno 2] No such file or directory: "'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'"

Looking into the code and adding a print statement in the run_command function I got this as the command_list:

["'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'", '-poly', "'download/sean198.c198.poly'", '-q0', '11290000', '-A', '30', '-q1', '11292000', '-lim0', '536000000', '-lim1', '536000000', '-lpb0', '33', '-lpb1', '33', '-mfb0', '64', '-mfb1', '95', '-ncurves0', '30', '-ncurves1', '15', '-fb1', "'download/sean198.c198.roots1.gz'", '-out', "'dylan-xps159570.79cb1ba5.work/sean198.c198.11290000-11292000.gz'", '-t', '6', '-stats-stderr']

The path inside all the quotes is indeed the location of my las executable, but somehow all the quotes confused it. So, to fix this I did the following:
(*) imported the shlex module
(*) replaced

command_list = command if isinstance(command, list) else command.split(" ")

with

command_list = shlex.split(command_str)

and then this worked (at least to start the run up. Waiting to see if it submits successfully - which it did!)

[henryzz]

Originally Posted by Dylan14

So I ran into a problem with the cado-nfs-client.py file, in particular, it couldn't find my las executable:

FileNotFoundError: [Errno 2] No such file or directory: "'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'"

Looking into the code and adding a print statement in the run_command function I got this as the command_list:

["'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'", '-poly', "'download/sean198.c198.poly'", '-q0', '11290000', '-A', '30', '-q1', '11292000', '-lim0', '536000000', '-lim1', '536000000', '-lpb0', '33', '-lpb1', '33', '-mfb0', '64', '-mfb1', '95', '-ncurves0', '30', '-ncurves1', '15', '-fb1', "'download/sean198.c198.roots1.gz'", '-out', "'dylan-xps159570.79cb1ba5.work/sean198.c198.11290000-11292000.gz'", '-t', '6', '-stats-stderr']

The path inside all the quotes is indeed the location of my las executable, but somehow all the quotes confused it. So, to fix this I did the following:
(*) imported the shlex module
(*) replaced

command_list = command if isinstance(command, list) else command.split(" ")

with

command_list = shlex.split(command_str)

and then this worked (at least to start the run up. Waiting to see if it submits successfully - which it did!)

You might want to report this error to the devs. It looks like the sort of thing that could be broken python 2/3 compatibility. I believe CADO uses python 2 but they aim for it to work in 3 as well. I don't have much python experience so I may be wrong.

[Dylan14]

Originally Posted by henryzz

You might want to report this error to the devs. It looks like the sort of thing that could be broken python 2/3 compatibility. I believe CADO uses python 2 but they aim for it to work in 3 as well. I don't have much python experience so I may be wrong.

Just submitted the issue on their repository. Waiting for a response.

[Dylan14]

The issue is fixed in a new commit, basically it was a conflict of version between a "new" client and a "old" server.

[EdH]

I'm happy to mention that I have the git version:

commit a1dbe6b800a0bef436f0723e62c5b502955e0c2a
Author: Emmanuel Thom� <Emmanuel.Thome@inria.fr>
Date:   Sat Aug 22 18:56:34 2020 +0200

running on all my machines, including the Core2 ones that wouldn't compile last year.

[VBCurtis]

Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!

[EdH]

Originally Posted by VBCurtis

Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!

My Core2s are Intel Quads (Q8400 and Q9???). They wouldn't compile the then current git last year, but worked fine as clients with an earlier commit until just recently when there was change that forced me to update everything - none of my clients would communicate with the newer version server and I was experiencing an occasional failure with the one I was running. ATM, all is well.

[EdH]

Originally Posted by VBCurtis

Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!

I must have jinxed it! Don't upgrade just yet. More later.