mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > CADO-NFS

Reply
 
Thread Tools
Old 2020-04-16, 06:00   #474
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

106048 Posts
Default

Yes, but it's hard to understand how. It seems to be the database side that's failing, yet bigger numbers gather relations and hand out WUs more slowly than smaller jobs. So, things *should* function better, because the rate of transactions is smaller... at least, I would think? (well, I do think... but I'm wrong)

In the Cunningham 2330L C207 thread last summer, one of the CADO contributors suggested a different database backend. I have zero experience with such things, so I didn't consider exactly what he suggested, but that's why I think it's something about the server/backend rather than a sieve parameter; certainly, one might trigger the other for reasons unknown.
VBCurtis is online now   Reply With Quote
Old 2020-04-16, 13:43   #475
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3,449 Posts
Default

I think I might have to set my laziness aside and join the CADO-NFS mailing list to see if I can gain a little more insight into the package. I'm a very amateur programmer, so I can't readily figure out in depth programs, but I do sometimes try alterations.

A "pet peeve" with CADO-NFS is its manner of crashing (dying) instead of gracefully ending, if you use "tasks.filter.run = false." Now I have 30+ pieces of "farm" equipment that are still looking for assignments. I'll probably have to restart without the "false" and crash that run, to "gracefully" stop the clients, rather than going to each one to issue CTRL-C..

Another difficulty is that I was unable to get a remote Colab install to run las on its own because it didn't like something about the roots1 data.

I do realize it's ME, but a lot of these things I keep trying, appear to be just opposite what the programmers intended. If I just had a better understanding (and a longer attention span). . .
EdH is offline   Reply With Quote
Old 2020-04-23, 21:10   #476
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3,449 Posts
Default

This is speculation, but I think I have discovered why CADO-NFS stops issuing WUs.

If WUs are tardy, the server tries to wait for the late WUs until timedout is reached, whereas it reissues them. But, it appears that if too many are late, it stops sending all others until they are caught up.

I have changed my timedout to 43200 (12 hours) to cover the sleeping time for some of my machines and have had no noticeable instances of WUs not being handed out for my current run.

Let's see if this theory is disproven now that I have posted it. . .
EdH is offline   Reply With Quote
Old 2020-04-29, 17:11   #477
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

21016 Posts
Default

So I ran into a problem with the cado-nfs-client.py file, in particular, it couldn't find my las executable:

Code:
FileNotFoundError: [Errno 2] No such file or directory: "'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'"
Looking into the code and adding a print statement in the run_command function I got this as the command_list:

Code:
["'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'", '-poly', "'download/sean198.c198.poly'", '-q0', '11290000', '-A', '30', '-q1', '11292000', '-lim0', '536000000', '-lim1', '536000000', '-lpb0', '33', '-lpb1', '33', '-mfb0', '64', '-mfb1', '95', '-ncurves0', '30', '-ncurves1', '15', '-fb1', "'download/sean198.c198.roots1.gz'", '-out', "'dylan-xps159570.79cb1ba5.work/sean198.c198.11290000-11292000.gz'", '-t', '6', '-stats-stderr']
The path inside all the quotes is indeed the location of my las executable, but somehow all the quotes confused it. So, to fix this I did the following:
(*) imported the shlex module
(*) replaced
Code:
command_list = command if isinstance(command, list) else command.split(" ")
with
Code:
command_list = shlex.split(command_str)
and then this worked (at least to start the run up. Waiting to see if it submits successfully - which it did!)
Dylan14 is offline   Reply With Quote
Old 2020-04-30, 07:14   #478
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

34×71 Posts
Default

Quote:
Originally Posted by Dylan14 View Post
So I ran into a problem with the cado-nfs-client.py file, in particular, it couldn't find my las executable:

Code:
FileNotFoundError: [Errno 2] No such file or directory: "'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'"
Looking into the code and adding a print statement in the run_command function I got this as the command_list:

Code:
["'/home/dylan/bin/cado/cado-nfs/build/dylan-xps159570/sieve/las'", '-poly', "'download/sean198.c198.poly'", '-q0', '11290000', '-A', '30', '-q1', '11292000', '-lim0', '536000000', '-lim1', '536000000', '-lpb0', '33', '-lpb1', '33', '-mfb0', '64', '-mfb1', '95', '-ncurves0', '30', '-ncurves1', '15', '-fb1', "'download/sean198.c198.roots1.gz'", '-out', "'dylan-xps159570.79cb1ba5.work/sean198.c198.11290000-11292000.gz'", '-t', '6', '-stats-stderr']
The path inside all the quotes is indeed the location of my las executable, but somehow all the quotes confused it. So, to fix this I did the following:
(*) imported the shlex module
(*) replaced
Code:
command_list = command if isinstance(command, list) else command.split(" ")
with
Code:
command_list = shlex.split(command_str)
and then this worked (at least to start the run up. Waiting to see if it submits successfully - which it did!)
You might want to report this error to the devs. It looks like the sort of thing that could be broken python 2/3 compatibility. I believe CADO uses python 2 but they aim for it to work in 3 as well. I don't have much python experience so I may be wrong.
henryzz is online now   Reply With Quote
Old 2020-04-30, 14:34   #479
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

24·3·11 Posts
Default

Quote:
Originally Posted by henryzz View Post
You might want to report this error to the devs. It looks like the sort of thing that could be broken python 2/3 compatibility. I believe CADO uses python 2 but they aim for it to work in 3 as well. I don't have much python experience so I may be wrong.

Just submitted the issue on their repository. Waiting for a response.
Dylan14 is offline   Reply With Quote
Old 2020-04-30, 16:42   #480
Dylan14
 
Dylan14's Avatar
 
"Dylan"
Mar 2017

24·3·11 Posts
Default

The issue is fixed in a new commit, basically it was a conflict of version between a "new" client and a "old" server.

Last fiddled with by VBCurtis on 2020-04-30 at 19:08
Dylan14 is offline   Reply With Quote
Old 2020-08-26, 19:55   #481
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3,449 Posts
Default

I'm happy to mention that I have the git version:
Code:
commit a1dbe6b800a0bef436f0723e62c5b502955e0c2a
Author: Emmanuel Thom´┐Ż <Emmanuel.Thome@inria.fr>
Date:   Sat Aug 22 18:56:34 2020 +0200
running on all my machines, including the Core2 ones that wouldn't compile last year.
EdH is offline   Reply With Quote
Old 2020-08-26, 20:41   #482
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

22×19×59 Posts
Default

Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!
VBCurtis is online now   Reply With Quote
Old 2020-08-26, 21:25   #483
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3,449 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!
My Core2s are Intel Quads (Q8400 and Q9???). They wouldn't compile the then current git last year, but worked fine as clients with an earlier commit until just recently when there was change that forced me to update everything - none of my clients would communicate with the newer version server and I was experiencing an occasional failure with the one I was running. ATM, all is well.
EdH is offline   Reply With Quote
Old 2020-08-27, 18:01   #484
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

3,449 Posts
Default

Quote:
Originally Posted by VBCurtis View Post
Excellent! That means I can update my Core2-Xeons this winter for another factorization. They sat unused this past winter, as the version of CADO installed on them wouldn't work as a client with the current version as server and the then-current version wouldn't compile. I missed out on all that "free" room heating!
I must have jinxed it! Don't upgrade just yet. More later.
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
CADO-NFS on windows jux CADO-NFS 22 2019-11-12 12:08
CADO help henryzz CADO-NFS 4 2017-11-20 15:14
CADO and WinBlows akruppa Programming 22 2015-12-31 08:37
CADO-NFS skan Information & Answers 1 2013-10-22 07:00
CADO R.D. Silverman Factoring 4 2008-11-06 12:35

All times are UTC. The time now is 21:03.

Fri Nov 27 21:03:54 UTC 2020 up 78 days, 18:14, 3 users, load averages: 1.23, 1.25, 1.26

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.