mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   CADO-NFS (https://www.mersenneforum.org/forumdisplay.php?f=170)
-   -   CADO NFS (https://www.mersenneforum.org/showthread.php?t=11948)

EdH 2018-05-06 02:33

The C155 is in LA:
[code]
Sending workunit c155_sieving_27572000-27574000 to client
Adding workunit c155_sieving_27592000-27594000 to database
Reached target of 137934925 relations, now have 137935755
[/code]

VBCurtis 2018-05-06 02:51

Excellent! I'm trying to pick lim's such that sieving completes somewhere between the midpoint of the lim's and the higher lim; for the C155 params I set, that would be 25 to 33M. 27 lands nicely in that range. :) My GGNFS run of a similar input size sieved to 28M, but msieve needs fewer relations to build a matrix so I think this still indicates CADO yield is superior to GGNFS. Your run needed 23MQ for 137M raw relations, a yield of almost exactly 6.0.

As input size grows, the sieve range will grow faster than the lim's grow; this is consistent with choices on the 14e queue on nfs@home, and accepts a small decrease in yield in trade for hopes of a smaller matrix. CADO sievers appear to perform much better at relatively large Q than GGNFS, so I wonder if we ought to test even-smaller lim's on the 150-170 digit range.

henryzz 2018-05-06 05:09

[QUOTE=VictordeHolland;487012]Does anybody got CADO-NFS working under windoz?
On Ubuntu it is super easy with "git clone" and "make" and it will get all the packages it needs.

I compiled GMP, ECM and mlucas on Windows using Mingw, but this is a totally different beast. Even with the instructions included in the git repo :(. But I'm still a newbie with compiling....[/QUOTE]

I believe natively on windows isn't possible. It runs easily on the Linux subsystem on Windows 10

EdH 2018-05-06 17:39

And, the final results:
[code]
Info:Polynomial Selection (size optimized): Aggregate statistics:
Info:Polynomial Selection (size optimized): potential collisions: 150312
Info:Polynomial Selection (size optimized): raw lognorm (nr/min/av/max/std): 152195/46.670/56.086/63.060/1.112
Info:Polynomial Selection (size optimized): optimized lognorm (nr/min/av/max/std): 152195/45.120/50.015/56.830/1.370
Info:Polynomial Selection (size optimized): Total time: 303408
Info:Polynomial Selection (root optimized): Aggregate statistics:
Info:Polynomial Selection (root optimized): Total time: 12391.1
Info:Polynomial Selection (root optimized): Rootsieve time: 12389.6
Info:Generate Factor Base: Total cpu/real time for makefb: 38.09/7.7912
Info:Generate Free Relations: Total cpu/real time for freerel: 2481.92/322.779
Info:Lattice Sieving: Aggregate statistics:
Info:Lattice Sieving: Total number of relations: 137935755
Info:Lattice Sieving: Average J: 7702.62 for 1391605 special-q, max bucket fill: 0.618744
Info:Lattice Sieving: Total CPU time: 5.15512e+06s
Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 768.46/444.318
Info:Filtering - Duplicate Removal, splitting pass: Aggregate statistics:
Info:Filtering - Duplicate Removal, splitting pass: CPU time for dup1: 444.1s
Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 2034.56/409.591
Info:Filtering - Singleton removal: Total cpu/real time for purge: 888.21/283.444
Info:Filtering - Merging: Total cpu/real time for merge: 1090.49/979.903
Info:Filtering - Merging: Total cpu/real time for replay: 143.67/120.943
Info:Linear Algebra: Total cpu/real time for bwc: 440875/0.000195265
Info:Linear Algebra: Aggregate statistics:
Info:Linear Algebra: Krylov: WCT time 36874.43
Info:Linear Algebra: Lingen CPU time 1438.56, WCT time 219.83
Info:Linear Algebra: Mksol: WCT time 19838.11
Info:Quadratic Characters: Total cpu/real time for characters: 173.52/48.1477
Info:Square Root: Total cpu/real time for sqrt: 10010.9/1382.47
Info:HTTP server: Shutting down HTTP server
Info:Complete Factorization: Total cpu/elapsed time for entire factorization: 5.92942e+06/97592.1
Info:root: Cleaning up computation data in /tmp/cado.xp9ko4wu
106603488380168454820927220360012878679207958575989291522270608237193062808643 102639592829741105772054196573991675900716567808038066803341933521790711307779
[/code]During the sieving, I lost three i7 clients for an undetermined time. I restarted all three.

The core count was still around 170, but I have shifted some machines around and actually removed some dual core low end ones from the mix. So, the "mix" was slightly different from the previous run(s).

VBCurtis 2018-05-06 18:17

Looks like CADO is stretching its lead: C150 took 3.71M cpu-sec, while C155 took 5.93M cpu-sec. That's well under a doubling of time. My prior results showed a "double every 6 digits" scaling from C100 to C132; comparing these two times suggests that scaling is consistent up here in "meaningful work" territory.

I've yet to see someone dispute that msieve-GGNFS scales very close to doubling-every-5-digits, so it looks like even with the slow LA phase CADO may be the package of choice at GNFS150+!

If you get a chance to run a similar-sized number (say, within a factor of 2) with msieve/GGNFS, I'm quite interested to see the wall-clock comparison.

EdH 2018-05-07 01:12

If things go as planned, I'll start the same C155 in the morning using the same machines.

LaurV 2018-05-07 04:12

[QUOTE=VBCurtis;487091]If you get a chance to run a similar-sized number (say, within a factor of 2) with msieve/GGNFS [/QUOTE]
Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).

EdH 2018-05-07 15:48

[QUOTE=EdH;487104]If things go as planned, I'll start the same C155 in the morning using the same machines.[/QUOTE]I had a troublesome start. I'll have to try again tomorrow.

[QUOTE=LaurV;487110]Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).[/QUOTE]Curtis might be able to calculate some details, but to get a comparison wall clock time, I plan to use the same machines with the same composite to run my msieve/ggnfs scripts and compare the wall times directly.

But, thanks and Curtis may have a use for the logs.

VBCurtis 2018-05-07 16:54

[QUOTE=LaurV;487110]Does yafu counts? (we run our aliquot sequences in this range and may have logs, and yafu is using msieve/ggnfs, albeit not the last version).[/QUOTE]

Definitely! Just need wall-clock time for poly select, sieving, and LA; and the CPU specs for the machine used.
Ed's testing is sticking to inputs a multiple a 5 digits, so data you have for C145, 150,155,... will be quite helpful. I'll be doing CADO runs on these sizes on single machines, so I can compare my results to yours to get a sense of the relative speed of the two packages.

EdH 2018-05-09 04:19

Well, the polyselect and sieving time came in just under the sleep time for my slumber set of machines. I think that was longer than cado, but I didn't check. The LA has an ETA about six hours from now. It was expecting 7h18m at the start.

I was noticing that there are directions for running cado-nfs to LA, programming cado-nfs to stop there, perform a couple steps and transfer everything to msieve for the LA. As soon as I understand that process a bit more and can automate it, I will try that method with this composite and see if it would save even more.

More to follow in the morning...

EdH 2018-05-09 14:58

C155 msieve/ggnfs results are in:

Started process at day0 08:20:04 with a five minute poly select deadline:
[code]
Tue May 8 08:20:34 2018 Msieve v. 1.54 (SVN 1015)
Tue May 8 08:20:34 2018 random seeds: 2f149115 be482431
Tue May 8 08:20:34 2018 factoring 10941738641570527421809707322040357612003732945449205990913842131476349984288934784717997257891267332497625752899781833797076537244027146743531593354333897 (155 digits)
Tue May 8 08:20:35 2018 searching for 15-digit factors
Tue May 8 08:20:35 2018 commencing number field sieve (155-digit input)
Tue May 8 08:20:35 2018 commencing number field sieve polynomial selection
Tue May 8 08:20:35 2018 polynomial degree: 5
Tue May 8 08:20:35 2018 max stage 1 norm: 8.07e+23
Tue May 8 08:20:35 2018 max stage 2 norm: 4.96e+21
Tue May 8 08:20:35 2018 min E-value: 2.38e-12
Tue May 8 08:20:35 2018 poly select deadline: 300
Tue May 8 08:20:35 2018 time limit set to 0.08 CPU-hours
Tue May 8 08:20:35 2018 expecting poly E from 2.51e-12 to > 2.89e-12
Tue May 8 08:20:35 2018 searching leading coefficients from 1 to 3000
Tue May 8 08:34:05 2018 polynomial selection complete
Tue May 8 08:34:05 2018 R0: -3908239679572398037902061722952
Tue May 8 08:34:05 2018 R1: 4396266218171701
Tue May 8 08:34:05 2018 A0: -3110911159201674198111127589701474714298715
Tue May 8 08:34:05 2018 A1: 32056958887872842584995494485398422
Tue May 8 08:34:05 2018 A2: 591851843466883645077818327
Tue May 8 08:34:05 2018 A3: -4151561972490124674
Tue May 8 08:34:05 2018 A4: -21071420652
Tue May 8 08:34:05 2018 A5: 12
Tue May 8 08:34:05 2018 skew 194465259.33, size 3.996e-15, alpha -7.245, combined = 2.484e-12 rroots = 5
Tue May 8 08:34:05 2018 elapsed time 00:13:31
[/code]Sieving began at day0 08:35:05 across 36 machines (~170 cores):
[code]Tue May 08 08:35:05 2018 -> factmsieve.py (v0.86)
Tue May 08 08:35:05 2018 -> This is client 1 of 36
Tue May 08 08:35:05 2018 -> Running on 4 Cores with 2 hyper-threads per Core
Tue May 08 08:35:05 2018 -> Working with NAME = comp
Tue May 08 08:35:05 2018 -> Selected lattice siever: gnfs-lasieve4I14e
Tue May 08 08:35:05 2018 -> Creating param file to detect parameter changes...
Tue May 08 08:35:05 2018 -> Running setup ...
Tue May 08 08:35:05 2018 -> Estimated minimum relations needed: 6.03237e+07
Tue May 08 08:35:05 2018 -> cleaning up before a restart
Tue May 08 08:35:05 2018 -> Running lattice siever ...
Tue May 08 08:35:05 2018 -> entering sieving loop
Tue May 08 08:35:05 2018 -> making sieve job for q = 13550000 in 13550000 .. 13562500 as file comp.job.T0
[/code]Linear Algebra began at day0 22:31:16:
[code]
Tue May 8 22:31:16 2018 commencing linear algebra
Tue May 8 22:31:17 2018 read 3995271 cycles
Tue May 8 22:31:21 2018 cycles contain 11109792 unique relations
Tue May 8 22:32:25 2018 read 11109792 relations
Tue May 8 22:32:36 2018 using 20 quadratic characters above 4294917295
Tue May 8 22:33:24 2018 building initial matrix
Tue May 8 22:34:56 2018 memory use: 1526.8 MB
Tue May 8 22:34:57 2018 read 3995271 cycles
Tue May 8 22:34:58 2018 matrix is 3995094 x 3995271 (1210.4 MB) with weight 377883871 (94.58/col)
Tue May 8 22:34:58 2018 sparse part has weight 269359854 (67.42/col)
Tue May 8 22:35:26 2018 filtering completed in 2 passes
Tue May 8 22:35:27 2018 matrix is 3994383 x 3994560 (1210.4 MB) with weight 377860202 (94.59/col)
Tue May 8 22:35:27 2018 sparse part has weight 269354323 (67.43/col)
Tue May 8 22:35:45 2018 matrix starts at (0, 0)
Tue May 8 22:35:46 2018 matrix is 3994383 x 3994560 (1210.4 MB) with weight 377860202 (94.59/col)
Tue May 8 22:35:46 2018 sparse part has weight 269354323 (67.43/col)
Tue May 8 22:35:46 2018 saving the first 48 matrix rows for later
Tue May 8 22:35:47 2018 matrix includes 64 packed rows
Tue May 8 22:35:47 2018 matrix is 3994335 x 3994560 (1166.8 MB) with weight 302134637 (75.64/col)
Tue May 8 22:35:47 2018 sparse part has weight 265931977 (66.57/col)
Tue May 8 22:35:47 2018 using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Tue May 8 22:35:58 2018 commencing Lanczos iteration (8 threads)
Tue May 8 22:35:58 2018 memory use: 935.5 MB
Tue May 8 22:36:09 2018 linear algebra at 0.0%, ETA 7h18m
[/code]The factors were logged at day1 06:41:25:
[code]
Wed May 9 06:41:25 2018 p78 factor: 102639592829741105772054196573991675900716567808038066803341933521790711307779
Wed May 9 06:41:25 2018 p78 factor: 106603488380168454820927220360012878679207958575989291522270608237193062808643
[/code]Total wall time:
[code]
22:21:21
[/code]if my clock math is correct. I think cado-nfs came in at:
[code]
27:06:32
[/code]


All times are UTC. The time now is 19:12.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.