CADO NFS

[EdH]

The C155 is in LA:

Sending workunit c155_sieving_27572000-27574000 to client
Adding workunit c155_sieving_27592000-27594000 to database
Reached target of 137934925 relations, now have 137935755

[VBCurtis]

Excellent! I'm trying to pick lim's such that sieving completes somewhere between the midpoint of the lim's and the higher lim; for the C155 params I set, that would be 25 to 33M. 27 lands nicely in that range. :) My GGNFS run of a similar input size sieved to 28M, but msieve needs fewer relations to build a matrix so I think this still indicates CADO yield is superior to GGNFS. Your run needed 23MQ for 137M raw relations, a yield of almost exactly 6.0.

As input size grows, the sieve range will grow faster than the lim's grow; this is consistent with choices on the 14e queue on nfs@home, and accepts a small decrease in yield in trade for hopes of a smaller matrix. CADO sievers appear to perform much better at relatively large Q than GGNFS, so I wonder if we ought to test even-smaller lim's on the 150-170 digit range.

[henryzz]

Originally Posted by VictordeHolland

Does anybody got CADO-NFS working under windoz?
On Ubuntu it is super easy with "git clone" and "make" and it will get all the packages it needs.

I compiled GMP, ECM and mlucas on Windows using Mingw, but this is a totally different beast. Even with the instructions included in the git repo :(. But I'm still a newbie with compiling....

I believe natively on windows isn't possible. It runs easily on the Linux subsystem on Windows 10

[EdH]

And, the final results:

Info:Polynomial Selection (size optimized): Aggregate statistics:
Info:Polynomial Selection (size optimized): potential collisions: 150312
Info:Polynomial Selection (size optimized): raw lognorm (nr/min/av/max/std): 152195/46.670/56.086/63.060/1.112
Info:Polynomial Selection (size optimized): optimized lognorm (nr/min/av/max/std): 152195/45.120/50.015/56.830/1.370
Info:Polynomial Selection (size optimized): Total time: 303408
Info:Polynomial Selection (root optimized): Aggregate statistics:
Info:Polynomial Selection (root optimized): Total time: 12391.1
Info:Polynomial Selection (root optimized): Rootsieve time: 12389.6
Info:Generate Factor Base: Total cpu/real time for makefb: 38.09/7.7912
Info:Generate Free Relations: Total cpu/real time for freerel: 2481.92/322.779
Info:Lattice Sieving: Aggregate statistics:
Info:Lattice Sieving: Total number of relations: 137935755
Info:Lattice Sieving: Average J: 7702.62 for 1391605 special-q, max bucket fill: 0.618744
Info:Lattice Sieving: Total CPU time: 5.15512e+06s
Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 768.46/444.318
Info:Filtering - Duplicate Removal, splitting pass: Aggregate statistics:
Info:Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 444.1s
Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 2034.56/409.591
Info:Filtering - Singleton removal: Total cpu/real time for purge: 888.21/283.444
Info:Filtering - Merging: Total cpu/real time for merge: 1090.49/979.903
Info:Filtering - Merging: Total cpu/real time for replay: 143.67/120.943
Info:Linear Algebra: Total cpu/real time for bwc: 440875/0.000195265
Info:Linear Algebra: Aggregate statistics:
Info:Linear Algebra: Krylov: WCT time 36874.43
Info:Linear Algebra: Lingen CPU time 1438.56, WCT time 219.83
Info:Linear Algebra: Mksol: WCT time 19838.11
Info:Quadratic Characters: Total cpu/real time for characters: 173.52/48.1477
Info:Square Root: Total cpu/real time for sqrt: 10010.9/1382.47
Info:HTTP server: Shutting down HTTP server
Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 5.92942e+06/97592.1
Info:root: Cleaning up computation data in /tmp/cado.xp9ko4wu
106603488380168454820927220360012878679207958575989291522270608237193062808643 102639592829741105772054196573991675900716567808038066803341933521790711307779

During the sieving, I lost three i7 clients for an undetermined time. I restarted all three.

The core count was still around 170, but I have shifted some machines around and actually removed some dual core low end ones from the mix. So, the "mix" was slightly different from the previous run(s).

[VBCurtis]

Looks like CADO is stretching its lead: C150 took 3.71M cpu-sec, while C155 took 5.93M cpu-sec. That's well under a doubling of time. My prior results showed a "double every 6 digits" scaling from C100 to C132; comparing these two times suggests that scaling is consistent up here in "meaningful work" territory.

I've yet to see someone dispute that msieve-GGNFS scales very close to doubling-every-5-digits, so it looks like even with the slow LA phase CADO may be the package of choice at GNFS150+!

If you get a chance to run a similar-sized number (say, within a factor of 2) with msieve/GGNFS, I'm quite interested to see the wall-clock comparison.

[EdH]

If things go as planned, I'll start the same C155 in the morning using the same machines.

[LaurV]

Originally Posted by VBCurtis

If you get a chance to run a similar-sized number (say, within a factor of 2) with msieve/GGNFS

Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).

[EdH]

Originally Posted by EdH

If things go as planned, I'll start the same C155 in the morning using the same machines.

I had a troublesome start. I'll have to try again tomorrow.

Originally Posted by LaurV

Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).

Curtis might be able to calculate some details, but to get a comparison wall clock time, I plan to use the same machines with the same composite to run my msieve/ggnfs scripts and compare the wall times directly.

But, thanks and Curtis may have a use for the logs.

[VBCurtis]

Originally Posted by LaurV

Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).

Definitely! Just need wall-clock time for poly select, sieving, and LA; and the CPU specs for the machine used.
Ed's testing is sticking to inputs a multiple a 5 digits, so data you have for C145, 150,155,... will be quite helpful. I'll be doing CADO runs on these sizes on single machines, so I can compare my results to yours to get a sense of the relative speed of the two packages.

[EdH]

Well, the polyselect and sieving time came in just under the sleep time for my slumber set of machines. I think that was longer than cado, but I didn't check. The LA has an ETA about six hours from now. It was expecting 7h18m at the start.

I was noticing that there are directions for running cado-nfs to LA, programming cado-nfs to stop there, perform a couple steps and transfer everything to msieve for the LA. As soon as I understand that process a bit more and can automate it, I will try that method with this composite and see if it would save even more.

More to follow in the morning...

[EdH]

C155 msieve/ggnfs results are in:

Started process at day0 08:20:04 with a five minute poly select deadline:

Tue May  8 08:20:34 2018  Msieve v. 1.54 (SVN 1015)
Tue May  8 08:20:34 2018  random seeds: 2f149115 be482431
Tue May  8 08:20:34 2018  factoring 10941738641570527421809707322040357612003732945449205990913842131476349984288934784717997257891267332497625752899781833797076537244027146743531593354333897 (155 digits)
Tue May  8 08:20:35 2018  searching for 15-digit factors
Tue May  8 08:20:35 2018  commencing number field sieve (155-digit input)
Tue May  8 08:20:35 2018  commencing number field sieve polynomial selection
Tue May  8 08:20:35 2018  polynomial degree: 5
Tue May  8 08:20:35 2018  max stage 1 norm: 8.07e+23
Tue May  8 08:20:35 2018  max stage 2 norm: 4.96e+21
Tue May  8 08:20:35 2018  min E-value: 2.38e-12
Tue May  8 08:20:35 2018  poly select deadline: 300
Tue May  8 08:20:35 2018  time limit set to 0.08 CPU-hours
Tue May  8 08:20:35 2018  expecting poly E from 2.51e-12 to > 2.89e-12
Tue May  8 08:20:35 2018  searching leading coefficients from 1 to 3000
Tue May  8 08:34:05 2018  polynomial selection complete
Tue May  8 08:34:05 2018  R0: -3908239679572398037902061722952
Tue May  8 08:34:05 2018  R1: 4396266218171701
Tue May  8 08:34:05 2018  A0: -3110911159201674198111127589701474714298715
Tue May  8 08:34:05 2018  A1: 32056958887872842584995494485398422
Tue May  8 08:34:05 2018  A2: 591851843466883645077818327
Tue May  8 08:34:05 2018  A3: -4151561972490124674
Tue May  8 08:34:05 2018  A4: -21071420652
Tue May  8 08:34:05 2018  A5: 12
Tue May  8 08:34:05 2018  skew 194465259.33, size 3.996e-15, alpha -7.245, combined = 2.484e-12 rroots = 5
Tue May  8 08:34:05 2018  elapsed time 00:13:31

Sieving began at day0 08:35:05 across 36 machines (~170 cores):

Tue May 08 08:35:05 2018 -> factmsieve.py (v0.86)
Tue May 08 08:35:05 2018 -> This is client 1 of 36
Tue May 08 08:35:05 2018 -> Running on 4 Cores with 2 hyper-threads per Core
Tue May 08 08:35:05 2018 -> Working with NAME = comp
Tue May 08 08:35:05 2018 -> Selected lattice siever: gnfs-lasieve4I14e
Tue May 08 08:35:05 2018 -> Creating param file to detect parameter changes...
Tue May 08 08:35:05 2018 -> Running setup ...
Tue May 08 08:35:05 2018 -> Estimated minimum relations needed: 6.03237e+07
Tue May 08 08:35:05 2018 -> cleaning up before a restart
Tue May 08 08:35:05 2018 -> Running lattice siever ...
Tue May 08 08:35:05 2018 -> entering sieving loop
Tue May 08 08:35:05 2018 -> making sieve job for q = 13550000 in 13550000 .. 13562500 as file comp.job.T0

Linear Algebra began at day0 22:31:16:

Tue May  8 22:31:16 2018  commencing linear algebra
Tue May  8 22:31:17 2018  read 3995271 cycles
Tue May  8 22:31:21 2018  cycles contain 11109792 unique relations
Tue May  8 22:32:25 2018  read 11109792 relations
Tue May  8 22:32:36 2018  using 20 quadratic characters above 4294917295
Tue May  8 22:33:24 2018  building initial matrix
Tue May  8 22:34:56 2018  memory use: 1526.8 MB
Tue May  8 22:34:57 2018  read 3995271 cycles
Tue May  8 22:34:58 2018  matrix is 3995094 x 3995271 (1210.4 MB) with weight 377883871 (94.58/col)
Tue May  8 22:34:58 2018  sparse part has weight 269359854 (67.42/col)
Tue May  8 22:35:26 2018  filtering completed in 2 passes
Tue May  8 22:35:27 2018  matrix is 3994383 x 3994560 (1210.4 MB) with weight 377860202 (94.59/col)
Tue May  8 22:35:27 2018  sparse part has weight 269354323 (67.43/col)
Tue May  8 22:35:45 2018  matrix starts at (0, 0)
Tue May  8 22:35:46 2018  matrix is 3994383 x 3994560 (1210.4 MB) with weight 377860202 (94.59/col)
Tue May  8 22:35:46 2018  sparse part has weight 269354323 (67.43/col)
Tue May  8 22:35:46 2018  saving the first 48 matrix rows for later
Tue May  8 22:35:47 2018  matrix includes 64 packed rows
Tue May  8 22:35:47 2018  matrix is 3994335 x 3994560 (1166.8 MB) with weight 302134637 (75.64/col)
Tue May  8 22:35:47 2018  sparse part has weight 265931977 (66.57/col)
Tue May  8 22:35:47 2018  using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Tue May  8 22:35:58 2018  commencing Lanczos iteration (8 threads)
Tue May  8 22:35:58 2018  memory use: 935.5 MB
Tue May  8 22:36:09 2018  linear algebra at 0.0%, ETA 7h18m

The factors were logged at day1 06:41:25:

Wed May  9 06:41:25 2018  p78 factor: 102639592829741105772054196573991675900716567808038066803341933521790711307779
Wed May  9 06:41:25 2018  p78 factor: 106603488380168454820927220360012878679207958575989291522270608237193062808643

Total wall time:

22:21:21

if my clock math is correct. I think cado-nfs came in at:

27:06:32