CADO NFS

[Dubslow]

Error: maxlevel should be positive and less than 32

From your pasted code.

Also, gah the error printing needs some formatting work.

[KingsAlpaca]

Thanks Dubslow,

I changed my "tasks.filter.maxlevel" to 32 and indeed it looks ok now. Although I have no idea why the initial value of "tasks.filter.maxlevel" is 40.

I also noticed the problem after posting the previous message. I am still a newbie on factorization. I will try to improve in the future! Thanks again for your advice.

[EdH]

I'm coming up with many things to question and some machines that don't work as clients even though they seem to run well as a standalone. Is it better to address everything here or would it be better to use the discussion group?

I'm kind of up in the air over whether I will replace all my factmsieve.py distributive scripts with CADO-NFS. My scripts are cumbersome and need rewriting, but they may have an edge on CADO-NFS. My overall scripts distribute ECM as well and initiate poly selection on an ancient GPU while ECM is running. I could still use the ECM portion and replace the gnfs part with CADO-NFS. If I read the READMEs correctly, I could still run the poly selection on the GPU and import it to CADO-NFS. Perhaps I will experiment with that...

Any insights?

[Dubslow]

What edge may your scripts have over CADO?

[EdH]

Originally Posted by Dubslow

What edge may your scripts have over CADO?

I was thinking mostly of factmsieve.py and ggnfs having an edge over CADO. As to the scripts, they run continuously awaiting to either perform ECM segments or sieving, going idle during off-times. Whether they exist or not is of no significance to the host and the main script doesn't really care if they aren't working properly, other than possibly receiving bad relations that may need to be culled. There is also a minimum of communication, consisting mainly of work requests and acknowledgments,

I must admit to not having run the scripts for a few months and I would have to familiarize myself with them again to better describe them. And, I did have collisions on occasion that duplicated an occasional bit of work. They need some refinement.

I used them mostly as an automated Aliqueit system. I ran quite a few sequences with them a while back with over 20 machines of various level contributing to the whole.

[EdH]

I currently have one host machine and three client machines all running properly, as far as I can tell. This, to me, means I must have something right. All are i7 2600 systems.

However, I have an i7 920 system that runs fine by itself, but won't play well with others. It mostly works as a host, but the client shows messages like:

ERROR:root:Existing file download/c95.roots.gz has wrong checksum ... Deleting file.

When I try to run it as a client, it won't work at all:

INFO:root:Attaching file math69.1ba8791a.work/c95.94000-96000.gz to upload
ERROR:root:Could not read file math69.1ba8791a.work/c95.94000-96000.gz: [Errno 2] No such file or directory: 'math69.1ba8791a.work/c95.94000-96000.gz'
INFO:root:Attaching stderr for command 0 to upload
INFO:root:Sending result for workunit c95_sieving_94000-96000 to http://math79.local:13531/cgi-bin/upload.py
INFO:root:Cleaning up for workunit c95_sieving_94000-96000
INFO:root:Removing result file math69.1ba8791a.work/c95.94000-96000.gz
ERROR:root:Could not remove file: [Errno 2] No such file or directory: 'math69.1ba8791a.work/c95.94000-96000.gz'
INFO:root:Removing workunit file download/WU.math69.1ba8791a
INFO:root:Downloading http://math79.local:13531/cgi-bin/getwu?clientid=math69.1ba8791a to download/WU.math69.1ba8791a (cafile = None)
INFO:root:download/c95.roots.gz already exists, not downloading
INFO:root:download/c95.poly already exists, not downloading
INFO:root:download/las already exists, not downloading
INFO:root:Result file math69.1ba8791a.work/c95.96000-98000.gz does not exist
INFO:root:Running 'download/las' -I 13 -poly 'download/c95.poly' -q0 96000 -q1 98000 -lim0 3660220 -lim1 2758600 -lpb0 23 -lpb1 23 -mfb0 22 -mfb1 45 -ncurves0 2 -ncurves1 13 -fb 'download/c95.roots.gz' -out 'math69.1ba8791a.work/c95.96000-98000.gz' -t 2 -stats-stderr
ERROR:root:Command resulted in exit code 132
ERROR:root:Stderr: Illegal instruction (core dumped)

Any words of wisdom?

[henryzz]

Is the client downloading the siever from the server? Can you stop that behaviour?

[VictordeHolland]

CADO-NFS works on my ARM board (Odroid-U2)!
Compiles with just a simple 'make' command with GCC 6.3
So kudo's to the people making the make file!

Tested it with RSA-100

PID27408 2018-03-01 18:32:52,010 Info:Square Root: finished
PID27408 2018-03-01 18:32:52,011 Info:Square Root: Factors: 40094690950920881030683735292761468389214899724061 37975227936943673922808872755445627854565536638199
PID27408 2018-03-01 18:32:52,012 Info:Square Root: Total cpu/real time for sqrt: 419.33/145.825
PID27408 2018-03-01 18:32:52,014 Info:Polynomial Selection (size optimized): Total time: 1138.84
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Aggregate statistics:
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Total time: 338.18
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Rootsieve time: 336.08
PID27408 2018-03-01 18:32:52,017 Info:Generate Factor Base: Total cpu/real time for makefb: 5.19/1.45861
PID27408 2018-03-01 18:32:52,017 Info:Generate Free Relations: Total cpu/real time for freerel: 145.56/40.3167
PID27408 2018-03-01 18:32:52,020 Info:Lattice Sieving: Total CPU time: 28356.3s
PID27408 2018-03-01 18:32:52,020 Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 16.03/43.1087
PID27408 2018-03-01 18:32:52,021 Info:Filtering - Duplicate Removal, splitting pass: Aggregate statistics:
PID27408 2018-03-01 18:32:52,021 Info:Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 42.7s
PID27408 2018-03-01 18:32:52,022 Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 80.83/58.5983
PID27408 2018-03-01 18:32:52,023 Info:Filtering - Singleton removal: Total cpu/real time for purge: 53.38/32.7059
PID27408 2018-03-01 18:32:52,023 Info:Filtering - Merging: Total cpu/real time for merge: 292.17/277.597
PID27408 2018-03-01 18:32:52,024 Info:Filtering - Merging: Total cpu/real time for replay: 25.63/30.9723
PID27408 2018-03-01 18:32:52,024 Info:Linear Algebra: Total cpu/real time for bwc: 12419.8/0.000614882
PID27408 2018-03-01 18:32:52,025 Info:Linear Algebra: Aggregate statistics:
PID27408 2018-03-01 18:32:52,025 Info:Linear Algebra: Krylov: WCT time 1882.77
PID27408 2018-03-01 18:32:52,026 Info:Linear Algebra: Lingen CPU time 586.27, WCT time 163.55
PID27408 2018-03-01 18:32:52,027 Info:Linear Algebra: Mksol: WCT time 1043.0
PID27408 2018-03-01 18:32:52,027 Info:Quadratic Characters: Total cpu/real time for characters: 16.38/7.23538
PID27408 2018-03-01 18:32:52,028 Info:Square Root: Total cpu/real time for sqrt: 419.33/145.825
PID27408 2018-03-01 18:32:52,204 Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 43307.6/12677.9

So
Poly Select: 1475 sec (24 min 35 sec)
Sieving: 28356 CPUsec (~about 2h clock time)
Filtering: ~486 sec (~8 min)
LA: 1883 + 164 (586 CPUsec) + 1043 sec (~52 clock min)
SQR: 419 CPUsec (146 sec wall clock)
Total: 43,307 CPUsec / 12,678 sec WCT

Or about 3.5h real time, not bad for such a tiny board with outdated processor (quadcore Cortex-A9) and memory (LPDDR2-880).

[EdH]

Originally Posted by henryzz

Is the client downloading the siever from the server? Can you stop that behaviour?

I had tried figuring out how to use --bindir= and had it wrong. I think I have it figured out now. Thanks!

[henryzz]

Originally Posted by VictordeHolland

CADO-NFS works on my ARM board (Odroid-U2)!
Compiles with just a simple 'make' command with GCC 6.3
So kudo's to the people making the make file!

Tested it with RSA-100

PID27408 2018-03-01 18:32:52,010 Info:Square Root: finished
PID27408 2018-03-01 18:32:52,011 Info:Square Root: Factors: 40094690950920881030683735292761468389214899724061 37975227936943673922808872755445627854565536638199
PID27408 2018-03-01 18:32:52,012 Info:Square Root: Total cpu/real time for sqrt: 419.33/145.825
PID27408 2018-03-01 18:32:52,014 Info:Polynomial Selection (size optimized): Total time: 1138.84
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Aggregate statistics:
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Total time: 338.18
PID27408 2018-03-01 18:32:52,016 Info:Polynomial Selection (root optimized): Rootsieve time: 336.08
PID27408 2018-03-01 18:32:52,017 Info:Generate Factor Base: Total cpu/real time for makefb: 5.19/1.45861
PID27408 2018-03-01 18:32:52,017 Info:Generate Free Relations: Total cpu/real time for freerel: 145.56/40.3167
PID27408 2018-03-01 18:32:52,020 Info:Lattice Sieving: Total CPU time: 28356.3s
PID27408 2018-03-01 18:32:52,020 Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 16.03/43.1087
PID27408 2018-03-01 18:32:52,021 Info:Filtering - Duplicate Removal, splitting pass: Aggregate statistics:
PID27408 2018-03-01 18:32:52,021 Info:Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 42.7s
PID27408 2018-03-01 18:32:52,022 Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 80.83/58.5983
PID27408 2018-03-01 18:32:52,023 Info:Filtering - Singleton removal: Total cpu/real time for purge: 53.38/32.7059
PID27408 2018-03-01 18:32:52,023 Info:Filtering - Merging: Total cpu/real time for merge: 292.17/277.597
PID27408 2018-03-01 18:32:52,024 Info:Filtering - Merging: Total cpu/real time for replay: 25.63/30.9723
PID27408 2018-03-01 18:32:52,024 Info:Linear Algebra: Total cpu/real time for bwc: 12419.8/0.000614882
PID27408 2018-03-01 18:32:52,025 Info:Linear Algebra: Aggregate statistics:
PID27408 2018-03-01 18:32:52,025 Info:Linear Algebra: Krylov: WCT time 1882.77
PID27408 2018-03-01 18:32:52,026 Info:Linear Algebra: Lingen CPU time 586.27, WCT time 163.55
PID27408 2018-03-01 18:32:52,027 Info:Linear Algebra: Mksol: WCT time 1043.0
PID27408 2018-03-01 18:32:52,027 Info:Quadratic Characters: Total cpu/real time for characters: 16.38/7.23538
PID27408 2018-03-01 18:32:52,028 Info:Square Root: Total cpu/real time for sqrt: 419.33/145.825
PID27408 2018-03-01 18:32:52,204 Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 43307.6/12677.9

So
Poly Select: 1475 sec (24 min 35 sec)
Sieving: 28356 CPUsec (~about 2h clock time)
Filtering: ~486 sec (~8 min)
LA: 1883 + 164 (586 CPUsec) + 1043 sec (~52 clock min)
SQR: 419 CPUsec (146 sec wall clock)
Total: 43,307 CPUsec / 12,678 sec WCT

Or about 3.5h real time, not bad for such a tiny board with outdated processor (quadcore Cortex-A9) and memory (LPDDR2-880).

How does that compare with the pure c version of the ggnfs siever?

[EdH]

Originally Posted by EdH

I had tried figuring out how to use --bindir= and had it wrong. I think I have it figured out now. Thanks!

This was also the issue with my ancient Debian Core 2 machine. It was trying to run the Ubuntu i7 binaries. It seems fine now that it's running its own...

It looks like I'm nearing a point where I'll run a test with RSA-130.

Thanks for all the help everyone.