CADO NFS

[VBCurtis]

Originally Posted by Dubslow

That's ~100 minutes vs ~138 for CADO with Curtis' improved C130 settings, am I understanding that right? So in total, CADO still has some ground to make up, even if some of its parts are winners?

Yes, that is my experience as well. I looked into the data a little bit, and the biggest slowdown CADO faces is that for a given LP-bound size, CADO requires quite a lot more relations to build a matrix. That's where I got the idea to reduce CADO's target density; alas, it hasn't closed the gap far enough to justify using CADO for all production.

I do believe the sievers and poly select are up to msieve/GGNFS standards & speeds. So, there's hope that a future improvement to CADO filtering can close much of the remaining gap, or even surpass GGNFS.

[EdH]

Of note, remember that I set the cut-off for poly select at 5 minutes with 8 threads. If I had let msieve run what it wanted, it would have easily used up the extra time:

... poly select deadline: 52159
... time limit set to 14.49 CPU-hours

[VBCurtis]

Originally Posted by VBCurtis

I've been re-reading the factorization-announcement papers from the CADO group, and I think one of them mentioned that one of the three CADO matrix phases can be parallelized across gigabit; I'll have a look one of these evenings to see if I can find and quote that passage.

Here's that paper:
https://hal.inria.fr/inria-00502899/file/grid.pdf

The first and third phases can be split to not-well-connected machines, at the cost of copying the matrix and some other files from machine to machine. The CADO docs make gentle reference to doing so, but I could not make out an actual way to trigger the CADO package to do so. Perhaps that's coming soon in 3.0!

[EdH]

@Curtis: I'm getting ready to do some more testing this week. I'm looking at RSA150, but I might just grab a C150 from one of the unreserved Aliquot sequences.

Anyway, have you done any parameter work for CADO-NFS for C150s?

I'll do a run with CADO-NFS and another with my gnfs scripts for comparison. If you have another set of CADO-NFS params, I can do a run with those as well. I hope to run them all this week, but I often procrastinate on these tests.

Ed

[VBCurtis]

Ed-
I haven't yet, but I could try a draft based on what has worked up to 140 digits. I've been using my CADO resources on poly select for some big GNFS jobs the past month. I'm still working on my tax returns, but by Thursday or so I should be able to guess at some improved parameters.

Gotta start somewhere; we can start together! I have a hypothesis that CADO scales better than GGNFS/msieve, so I'm hoping to find that a C150 is closer than C130 between the two packages; if so I'll be quite encouraged to try future GNFS-160 class jobs with CADO, with the hope of exceeding GGNFS/msieve performance.

[EdH]

Curtis,

I'll be doing some testing throughout the week to see what machines and configurations I can use. A minor annoyance is that about half of my machines go to sleep at night, so I can't use them for any benchmarking if I can't fit in, at least, all the work up to LA, before slumber.

I'm going to use a C150 from the db that has survived YAFU's battery of ECM:

http://www.factordb.com/index.php?id...00001092093149

from Aliquot Sequence 281592

More later...

Ed

[EdH]

I seem to be having a bit of trouble. I've tried twice without success.

I'm trying to run the C150 described earlier and have factored it via my home brew scripts and msieve/ggnfs. It has three primes, if that is of any significance.

I've tried CADO-NFS twice with the same results.

I'm running 1 server, (obvously), and 37 clients. All seems fine until I get to 98% of the way through the Polynomial Selection. Then the server stops:

...
Info:Polynomial Selection (size optimized): Marking workunit c150_polyselect1_394000-395000 as ok (97.5% => ETA Thu Apr 19 08:18:55 2018)
Info:Polynomial Selection (size optimized): Parsed 138 polynomials, added 0 to priority queue (has 100)
Info:Polynomial Selection (size optimized): Marking workunit c150_polyselect1_390000-391000 as ok (97.8% => ETA Thu Apr 19 08:18:56 2018)
Info:Polynomial Selection (size optimized): Parsed 146 polynomials, added 0 to priority queue (has 100)
Info:Polynomial Selection (size optimized): Marking workunit c150_polyselect1_397000-398000 as ok (98.0% => ETA Thu Apr 19 08:19:17 2018)

All my clients are at idle:

...
ERROR:root:Download failed, URL error: HTTP Error 404: No work available
ERROR:root:Waiting 10.0 seconds before retrying (I have been waiting since 7120.0 seconds)
ERROR:root:Download failed, URL error: HTTP Error 404: No work available
ERROR:root:Waiting 10.0 seconds before retrying (I have been waiting since 7130.0 seconds)
ERROR:root:Download failed, URL error: HTTP Error 404: No work available
ERROR:root:Waiting 10.0 seconds before retrying (I have been waiting since 7140.0 seconds)
ERROR:root:Download failed, URL error: HTTP Error 404: No work available
ERROR:root:Waiting 10.0 seconds before retrying (I have been waiting since 7150.0 seconds)
... and counting ...

Could a misbehaving client cause this, or is it solely a server borne trouble? I will be turning my clients away from the project momentarily and probably the server as well, but will keep whatever files I can from the server.

Any thoughts appreciated...

[Dubslow]

You could always try attaching pdb to it?

[VBCurtis]

I once had a poly-select range that timed out repeatedly; msieve has a time-limit for any particular coeff, but it seems CADO does not. In my case, the offending range was under 1000, so I set admin=1e3 in the params file and all was well.

You could also try increasing the time-out limit from default (I think 3600 sec).

[EdH]

Originally Posted by Dubslow

You could always try attaching pdb to it?

My ignorance won't allow me to try this yet...

Originally Posted by VBCurtis

I once had a poly-select range that timed out repeatedly; msieve has a time-limit for any particular coeff, but it seems CADO does not. In my case, the offending range was under 1000, so I set admin=1e3 in the params file and all was well.

You could also try increasing the time-out limit from default (I think 3600 sec).

I've added tasks.polyselect.admin = 1e3 to the params.c150 Polynomial selection section. I'll try a restart tomorrow morning. If it doesn't run tomorrow, I'll try RSA150 and see if it will get past polyselect.

Thanks...

[EdH]

The admin = 1e3 appears to have done the trick. The process has moved into Lattice Sieving without any noticed trouble.

Thanks!