Some CADO-NFS Work At Around 175-180 Decimal Digits - Page 4

charybdis · 2020-04-16, 21:23

Quote:

Originally Posted by VBCurtis

Hrmmmmm..... I've been using the screen display "total sieving time" that pops up at the end of the sieve phase, and matches the end-of-factorization summary printed to screen. I can't say I delve into the log file hardly ever.

Ed's timing falls into my curve-fitting of time vs input size from the other jobs I listed, so I am quite confused. I appreciate you pointing out the inconsistency in the log report!

Did you also use the "total sieving time" to get the times for some of the other jobs you mentioned? If so that might explain the discrepancy.

Here are a couple of sample lines from an old completed job, as an example:

Code:

Info:Lattice Sieving: Total time: 403109s
...
Info:Complete Factorization / Discrete logarithm: Total cpu/elapsed time for entire factorization: 862194/16096.8

The second one seems to give the correct CPU time, whereas the first is just client-seconds; the sieving for this job took just over 2 hours wall clock time on ~110 cores, which is consistent with ~800k CPU-seconds for sieving plus a little extra for polyselect + postprocessing.

EdH · 2020-04-16, 21:48

Quote:

Originally Posted by VBCurtis

Hrmmmmm..... I've been using the screen display "total sieving time" that pops up at the end of the sieve phase, and matches the end-of-factorization summary printed to screen. I can't say I delve into the log file hardly ever.

Ed's timing falls into my curve-fitting of time vs input size from the other jobs I listed, so I am quite confused. I appreciate you pointing out the inconsistency in the log report!

Here are the significant last lines from the screen run to 270M:

Code:

Info:Lattice Sieving: Reached target of 270000000 relations, now have 270013333
Info:Lattice Sieving: Aggregate statistics:
Info:Lattice Sieving: Total number of relations: 270013333
Info:Lattice Sieving: Average J: 16023.9 for 5345903 special-q, max bucket fill -bkmult 1.0,1s:1.416720
Info:Lattice Sieving: Total time: 2.19772e+07s
Info:Filtering - Duplicate Removal, splitting pass: Stopping at duplicates1

EdH · 2020-04-16, 21:51

t_d=100 will build a matrix:

Code:

Thu Apr 16 12:15:32 2020  commencing relation filtering
Thu Apr 16 12:15:32 2020  setting target matrix density to 100.0
Thu Apr 16 12:15:32 2020  estimated available RAM is 15926.6 MB
Thu Apr 16 12:15:32 2020  commencing duplicate removal, pass 1
Thu Apr 16 12:15:34 2020  error -1 reading relation 189590
. . .
Thu Apr 16 12:49:54 2020  error -1 reading relation 267865222
Thu Apr 16 12:50:24 2020  found 88770665 hash collisions in 271761268 relations
Thu Apr 16 12:50:56 2020  added 122298 free relations
Thu Apr 16 12:50:56 2020  commencing duplicate removal, pass 2
Thu Apr 16 12:57:26 2020  found 125086447 duplicates and 146797119 unique relations
Thu Apr 16 12:57:26 2020  memory use: 1449.5 MB
Thu Apr 16 12:57:26 2020  reading ideals above 139919360
Thu Apr 16 12:57:26 2020  commencing singleton removal, initial pass
Thu Apr 16 13:13:00 2020  memory use: 3012.0 MB
Thu Apr 16 13:13:00 2020  reading all ideals from disk
Thu Apr 16 13:13:19 2020  memory use: 2357.7 MB
Thu Apr 16 13:13:24 2020  commencing in-memory singleton removal
Thu Apr 16 13:13:28 2020  begin with 146797119 relations and 142617272 unique ideals
Thu Apr 16 13:14:09 2020  reduce to 56506171 relations and 38799310 ideals in 18 passes
Thu Apr 16 13:14:09 2020  max relations containing the same ideal: 21
Thu Apr 16 13:14:12 2020  reading ideals above 720000
Thu Apr 16 13:14:12 2020  commencing singleton removal, initial pass
Thu Apr 16 13:24:49 2020  memory use: 1506.0 MB
Thu Apr 16 13:24:49 2020  reading all ideals from disk
Thu Apr 16 13:25:08 2020  memory use: 2241.3 MB
Thu Apr 16 13:25:14 2020  keeping 54258109 ideals with weight <= 200, target excess is 313347
Thu Apr 16 13:25:20 2020  commencing in-memory singleton removal
Thu Apr 16 13:25:25 2020  begin with 56506171 relations and 54258109 unique ideals
Thu Apr 16 13:26:29 2020  reduce to 56030216 relations and 53781539 ideals in 13 passes
Thu Apr 16 13:26:29 2020  max relations containing the same ideal: 200
Thu Apr 16 13:26:55 2020  removing 3684525 relations and 3284525 ideals in 400000 cliques
Thu Apr 16 13:26:57 2020  commencing in-memory singleton removal
Thu Apr 16 13:27:01 2020  begin with 52345691 relations and 53781539 unique ideals
Thu Apr 16 13:27:47 2020  reduce to 52174610 relations and 50324306 ideals in 10 passes
Thu Apr 16 13:27:47 2020  max relations containing the same ideal: 197
Thu Apr 16 13:28:11 2020  removing 2772195 relations and 2372195 ideals in 400000 cliques
Thu Apr 16 13:28:13 2020  commencing in-memory singleton removal
Thu Apr 16 13:28:17 2020  begin with 49402415 relations and 50324306 unique ideals
Thu Apr 16 13:28:55 2020  reduce to 49291975 relations and 47840809 ideals in 9 passes
Thu Apr 16 13:28:55 2020  max relations containing the same ideal: 190
Thu Apr 16 13:29:19 2020  removing 2488158 relations and 2088158 ideals in 400000 cliques
Thu Apr 16 13:29:20 2020  commencing in-memory singleton removal
Thu Apr 16 13:29:24 2020  begin with 46803817 relations and 47840809 unique ideals
Thu Apr 16 13:30:00 2020  reduce to 46708746 relations and 45656840 ideals in 9 passes
Thu Apr 16 13:30:00 2020  max relations containing the same ideal: 185
Thu Apr 16 13:30:22 2020  removing 2334687 relations and 1934687 ideals in 400000 cliques
Thu Apr 16 13:30:23 2020  commencing in-memory singleton removal
Thu Apr 16 13:30:27 2020  begin with 44374059 relations and 45656840 unique ideals
Thu Apr 16 13:30:57 2020  reduce to 44283467 relations and 43630836 ideals in 8 passes
Thu Apr 16 13:30:57 2020  max relations containing the same ideal: 182
Thu Apr 16 13:31:18 2020  removing 1701806 relations and 1412658 ideals in 289148 cliques
Thu Apr 16 13:31:19 2020  commencing in-memory singleton removal
Thu Apr 16 13:31:23 2020  begin with 42581661 relations and 43630836 unique ideals
Thu Apr 16 13:31:52 2020  reduce to 42532132 relations and 42168363 ideals in 8 passes
Thu Apr 16 13:31:52 2020  max relations containing the same ideal: 176
Thu Apr 16 13:32:19 2020  relations with 0 large ideals: 1038
Thu Apr 16 13:32:19 2020  relations with 1 large ideals: 1550
Thu Apr 16 13:32:19 2020  relations with 2 large ideals: 21593
Thu Apr 16 13:32:19 2020  relations with 3 large ideals: 198057
Thu Apr 16 13:32:19 2020  relations with 4 large ideals: 1072004
Thu Apr 16 13:32:19 2020  relations with 5 large ideals: 3623143
Thu Apr 16 13:32:19 2020  relations with 6 large ideals: 7969344
Thu Apr 16 13:32:19 2020  relations with 7+ large ideals: 29645403
Thu Apr 16 13:32:19 2020  commencing 2-way merge
Thu Apr 16 13:32:48 2020  reduce to 25695897 relation sets and 25332128 unique ideals
Thu Apr 16 13:32:48 2020  commencing full merge
Thu Apr 16 13:41:06 2020  memory use: 3081.6 MB
Thu Apr 16 13:41:09 2020  found 12276374 cycles, need 12244328
Thu Apr 16 13:41:12 2020  weight of 12244328 cycles is about 1224870286 (100.04/cycle)
Thu Apr 16 13:41:12 2020  distribution of cycle lengths:
Thu Apr 16 13:41:12 2020  1 relations: 1128304
Thu Apr 16 13:41:12 2020  2 relations: 1130284
Thu Apr 16 13:41:12 2020  3 relations: 1190346
Thu Apr 16 13:41:12 2020  4 relations: 1117203
Thu Apr 16 13:41:12 2020  5 relations: 1062965
Thu Apr 16 13:41:12 2020  6 relations: 969179
Thu Apr 16 13:41:12 2020  7 relations: 868868
Thu Apr 16 13:41:12 2020  8 relations: 766062
Thu Apr 16 13:41:12 2020  9 relations: 679349
Thu Apr 16 13:41:12 2020  10+ relations: 3331768
Thu Apr 16 13:41:12 2020  heaviest cycle: 28 relations
Thu Apr 16 13:41:15 2020  commencing cycle optimization
Thu Apr 16 13:41:35 2020  start with 86442588 relations
Thu Apr 16 13:43:59 2020  pruned 2629604 relations
Thu Apr 16 13:44:00 2020  memory use: 2639.1 MB
Thu Apr 16 13:44:00 2020  distribution of cycle lengths:
Thu Apr 16 13:44:00 2020  1 relations: 1128304
Thu Apr 16 13:44:00 2020  2 relations: 1158969
Thu Apr 16 13:44:00 2020  3 relations: 1235456
Thu Apr 16 13:44:00 2020  4 relations: 1152269
Thu Apr 16 13:44:00 2020  5 relations: 1097163
Thu Apr 16 13:44:00 2020  6 relations: 991876
Thu Apr 16 13:44:00 2020  7 relations: 886776
Thu Apr 16 13:44:00 2020  8 relations: 775313
Thu Apr 16 13:44:00 2020  9 relations: 683433
Thu Apr 16 13:44:00 2020  10+ relations: 3134769
Thu Apr 16 13:44:00 2020  heaviest cycle: 27 relations
Thu Apr 16 13:44:19 2020  RelProcTime: 5327
Thu Apr 16 13:44:24 2020  
Thu Apr 16 13:44:24 2020  commencing linear algebra
Thu Apr 16 13:44:25 2020  read 12244328 cycles
Thu Apr 16 13:44:47 2020  cycles contain 42246024 unique relations
Thu Apr 16 13:51:48 2020  read 42246024 relations
Thu Apr 16 13:52:54 2020  using 20 quadratic characters above 4294917295
Thu Apr 16 13:55:56 2020  building initial matrix
Thu Apr 16 14:03:01 2020  memory use: 5922.5 MB
Thu Apr 16 14:03:06 2020  read 12244328 cycles
Thu Apr 16 14:03:09 2020  matrix is 12244151 x 12244328 (5011.5 MB) with weight 1544672270 (126.15/col)
Thu Apr 16 14:03:09 2020  sparse part has weight 1166814329 (95.29/col)
Thu Apr 16 14:05:56 2020  filtering completed in 2 passes
Thu Apr 16 14:05:59 2020  matrix is 12243059 x 12243236 (5011.5 MB) with weight 1544627458 (126.16/col)
Thu Apr 16 14:05:59 2020  sparse part has weight 1166804691 (95.30/col)
Thu Apr 16 14:07:01 2020  matrix starts at (0, 0)
Thu Apr 16 14:07:04 2020  matrix is 12243059 x 12243236 (5011.5 MB) with weight 1544627458 (126.16/col)
Thu Apr 16 14:07:04 2020  sparse part has weight 1166804691 (95.30/col)
Thu Apr 16 14:07:04 2020  saving the first 48 matrix rows for later
Thu Apr 16 14:07:06 2020  matrix includes 64 packed rows
Thu Apr 16 14:07:08 2020  matrix is 12243011 x 12243236 (4887.3 MB) with weight 1292507191 (105.57/col)
Thu Apr 16 14:07:08 2020  sparse part has weight 1158749915 (94.64/col)
Thu Apr 16 14:07:08 2020  using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Thu Apr 16 14:08:03 2020  commencing Lanczos iteration (8 threads)
Thu Apr 16 14:08:03 2020  memory use: 4077.8 MB
Thu Apr 16 14:08:55 2020  linear algebra at 0.0%, ETA 112h 0m

VBCurtis · 2020-04-17, 00:33

12.2M at TD 100 seems pretty good; so maybe just 10-15M more relations will get you TD 120 and 20% faster matrix time.

EdH · 2020-04-17, 16:41

t_d=120 success:

Code:

Fri Apr 17 10:03:58 2020  Msieve v. 1.54 (SVN 1018)
Fri Apr 17 10:03:58 2020  random seeds: 947503a5 a978d24a
Fri Apr 17 10:03:58 2020  factoring 76552370139504036674890813564032281493867343366619508594816489005834882856199128873928842970710045044111574726594936894404957063604759585302342441093226844531070349677623657609 (176 digits)
Fri Apr 17 10:03:59 2020  searching for 15-digit factors
Fri Apr 17 10:03:59 2020  commencing number field sieve (176-digit input)
Fri Apr 17 10:03:59 2020  R0: -10749206376460432970317818596117873
Fri Apr 17 10:03:59 2020  R1: 4023609444811856477743
Fri Apr 17 10:03:59 2020  A0: 91389778824609164214454779424151524400880
Fri Apr 17 10:03:59 2020  A1: 16573333756774205759678902993899502
Fri Apr 17 10:03:59 2020  A2: -8753197000583595457254903663
Fri Apr 17 10:03:59 2020  A3: -1186820920867031701728
Fri Apr 17 10:03:59 2020  A4: 77519198521772
Fri Apr 17 10:03:59 2020  A5: 533400
Fri Apr 17 10:03:59 2020  skew 1.00, size 3.822e-17, alpha -6.645, combined = 8.280e-16 rroots = 5
Fri Apr 17 10:03:59 2020  
Fri Apr 17 10:03:59 2020  commencing relation filtering
Fri Apr 17 10:03:59 2020  setting target matrix density to 120.0
Fri Apr 17 10:03:59 2020  estimated available RAM is 15926.6 MB
Fri Apr 17 10:03:59 2020  commencing duplicate removal, pass 1
Fri Apr 17 10:04:01 2020  error -1 reading relation 160728
. . .
Fri Apr 17 10:42:16 2020  error -1 reading relation 294036055
Fri Apr 17 10:42:19 2020  found 96785463 hash collisions in 294359053 relations
Fri Apr 17 10:42:51 2020  added 122298 free relations
Fri Apr 17 10:42:51 2020  commencing duplicate removal, pass 2
Fri Apr 17 10:49:52 2020  found 136725894 duplicates and 157755457 unique relations
Fri Apr 17 10:49:52 2020  memory use: 2387.0 MB
Fri Apr 17 10:49:52 2020  reading ideals above 139919360
Fri Apr 17 10:49:52 2020  commencing singleton removal, initial pass
Fri Apr 17 11:06:57 2020  memory use: 3012.0 MB
Fri Apr 17 11:06:57 2020  reading all ideals from disk
Fri Apr 17 11:07:13 2020  memory use: 2535.2 MB
Fri Apr 17 11:07:18 2020  commencing in-memory singleton removal
Fri Apr 17 11:07:22 2020  begin with 157755457 relations and 148007529 unique ideals
Fri Apr 17 11:08:04 2020  reduce to 67900523 relations and 45789557 ideals in 15 passes
Fri Apr 17 11:08:04 2020  max relations containing the same ideal: 22
Fri Apr 17 11:08:08 2020  reading ideals above 720000
Fri Apr 17 11:08:08 2020  commencing singleton removal, initial pass
Fri Apr 17 11:20:26 2020  memory use: 1506.0 MB
Fri Apr 17 11:20:26 2020  reading all ideals from disk
Fri Apr 17 11:20:48 2020  memory use: 2698.7 MB
Fri Apr 17 11:20:55 2020  keeping 61205493 ideals with weight <= 200, target excess is 380214
Fri Apr 17 11:21:02 2020  commencing in-memory singleton removal
Fri Apr 17 11:21:09 2020  begin with 67900523 relations and 61205493 unique ideals
Fri Apr 17 11:22:25 2020  reduce to 67676809 relations and 60981638 ideals in 13 passes
Fri Apr 17 11:22:25 2020  max relations containing the same ideal: 200
Fri Apr 17 11:22:56 2020  removing 6407085 relations and 5407085 ideals in 1000000 cliques
Fri Apr 17 11:22:59 2020  commencing in-memory singleton removal
Fri Apr 17 11:23:04 2020  begin with 61269724 relations and 60981638 unique ideals
Fri Apr 17 11:23:52 2020  reduce to 60870327 relations and 55168052 ideals in 9 passes
Fri Apr 17 11:23:52 2020  max relations containing the same ideal: 194
Fri Apr 17 11:24:20 2020  removing 4932237 relations and 3932237 ideals in 1000000 cliques
Fri Apr 17 11:24:22 2020  commencing in-memory singleton removal
Fri Apr 17 11:24:27 2020  begin with 55938090 relations and 55168052 unique ideals
Fri Apr 17 11:25:05 2020  reduce to 55656169 relations and 50949518 ideals in 8 passes
Fri Apr 17 11:25:05 2020  max relations containing the same ideal: 182
Fri Apr 17 11:25:31 2020  removing 4493365 relations and 3493365 ideals in 1000000 cliques
Fri Apr 17 11:25:33 2020  commencing in-memory singleton removal
Fri Apr 17 11:25:37 2020  begin with 51162804 relations and 50949518 unique ideals
Fri Apr 17 11:26:12 2020  reduce to 50901236 relations and 47190406 ideals in 8 passes
Fri Apr 17 11:26:12 2020  max relations containing the same ideal: 175
Fri Apr 17 11:26:36 2020  removing 4277502 relations and 3277502 ideals in 1000000 cliques
Fri Apr 17 11:26:37 2020  commencing in-memory singleton removal
Fri Apr 17 11:26:42 2020  begin with 46623734 relations and 47190406 unique ideals
Fri Apr 17 11:27:17 2020  reduce to 46363498 relations and 43648404 ideals in 9 passes
Fri Apr 17 11:27:17 2020  max relations containing the same ideal: 163
Fri Apr 17 11:27:39 2020  removing 4160361 relations and 3160361 ideals in 1000000 cliques
Fri Apr 17 11:27:40 2020  commencing in-memory singleton removal
Fri Apr 17 11:27:44 2020  begin with 42203137 relations and 43648404 unique ideals
Fri Apr 17 11:28:09 2020  reduce to 41929982 relations and 40209959 ideals in 7 passes
Fri Apr 17 11:28:09 2020  max relations containing the same ideal: 152
Fri Apr 17 11:28:28 2020  removing 4106362 relations and 3106362 ideals in 1000000 cliques
Fri Apr 17 11:28:30 2020  commencing in-memory singleton removal
Fri Apr 17 11:28:33 2020  begin with 37823620 relations and 40209959 unique ideals
Fri Apr 17 11:28:55 2020  reduce to 37524651 relations and 36798691 ideals in 7 passes
Fri Apr 17 11:28:55 2020  max relations containing the same ideal: 144
Fri Apr 17 11:29:13 2020  removing 1493656 relations and 1208745 ideals in 284911 cliques
Fri Apr 17 11:29:14 2020  commencing in-memory singleton removal
Fri Apr 17 11:29:17 2020  begin with 36030995 relations and 36798691 unique ideals
Fri Apr 17 11:29:38 2020  reduce to 35991753 relations and 35550443 ideals in 7 passes
Fri Apr 17 11:29:38 2020  max relations containing the same ideal: 141
Fri Apr 17 11:30:01 2020  relations with 0 large ideals: 1293
Fri Apr 17 11:30:01 2020  relations with 1 large ideals: 2792
Fri Apr 17 11:30:01 2020  relations with 2 large ideals: 37024
Fri Apr 17 11:30:01 2020  relations with 3 large ideals: 297491
Fri Apr 17 11:30:01 2020  relations with 4 large ideals: 1403043
Fri Apr 17 11:30:01 2020  relations with 5 large ideals: 4111486
Fri Apr 17 11:30:01 2020  relations with 6 large ideals: 7823119
Fri Apr 17 11:30:01 2020  relations with 7+ large ideals: 22315505
Fri Apr 17 11:30:01 2020  commencing 2-way merge
Fri Apr 17 11:30:24 2020  reduce to 22470918 relation sets and 22029608 unique ideals
Fri Apr 17 11:30:24 2020  commencing full merge
Fri Apr 17 11:38:40 2020  memory use: 2811.5 MB
Fri Apr 17 11:38:42 2020  found 10221905 cycles, need 10199808
Fri Apr 17 11:38:45 2020  weight of 10199808 cycles is about 1224250293 (120.03/cycle)
Fri Apr 17 11:38:45 2020  distribution of cycle lengths:
Fri Apr 17 11:38:45 2020  1 relations: 584468
Fri Apr 17 11:38:45 2020  2 relations: 687298
Fri Apr 17 11:38:45 2020  3 relations: 768063
Fri Apr 17 11:38:45 2020  4 relations: 784213
Fri Apr 17 11:38:45 2020  5 relations: 792232
Fri Apr 17 11:38:45 2020  6 relations: 768076
Fri Apr 17 11:38:45 2020  7 relations: 737052
Fri Apr 17 11:38:45 2020  8 relations: 696318
Fri Apr 17 11:38:45 2020  9 relations: 646480
Fri Apr 17 11:38:45 2020  10+ relations: 3735608
Fri Apr 17 11:38:45 2020  heaviest cycle: 28 relations
Fri Apr 17 11:38:47 2020  commencing cycle optimization
Fri Apr 17 11:39:06 2020  start with 85373695 relations
Fri Apr 17 11:42:01 2020  pruned 3541155 relations
Fri Apr 17 11:42:01 2020  memory use: 2367.5 MB
Fri Apr 17 11:42:01 2020  distribution of cycle lengths:
Fri Apr 17 11:42:01 2020  1 relations: 584468
Fri Apr 17 11:42:01 2020  2 relations: 705356
Fri Apr 17 11:42:01 2020  3 relations: 800146
Fri Apr 17 11:42:01 2020  4 relations: 815349
Fri Apr 17 11:42:01 2020  5 relations: 828312
Fri Apr 17 11:42:01 2020  6 relations: 800399
Fri Apr 17 11:42:01 2020  7 relations: 769761
Fri Apr 17 11:42:01 2020  8 relations: 722256
Fri Apr 17 11:42:01 2020  9 relations: 668701
Fri Apr 17 11:42:01 2020  10+ relations: 3505060
Fri Apr 17 11:42:01 2020  heaviest cycle: 28 relations
Fri Apr 17 11:42:21 2020  RelProcTime: 5902
Fri Apr 17 11:42:25 2020  
Fri Apr 17 11:42:25 2020  commencing linear algebra
Fri Apr 17 11:42:26 2020  read 10199808 cycles
Fri Apr 17 11:42:46 2020  cycles contain 35733787 unique relations
Fri Apr 17 11:49:55 2020  read 35733787 relations
Fri Apr 17 11:50:57 2020  using 20 quadratic characters above 4294917295
Fri Apr 17 11:53:31 2020  building initial matrix
Fri Apr 17 12:00:17 2020  memory use: 4960.8 MB
Fri Apr 17 12:00:22 2020  read 10199808 cycles
Fri Apr 17 12:00:25 2020  matrix is 10199631 x 10199808 (4836.1 MB) with weight 1480894507 (145.19/col)
Fri Apr 17 12:00:25 2020  sparse part has weight 1145354678 (112.29/col)
Fri Apr 17 12:02:39 2020  filtering completed in 2 passes
Fri Apr 17 12:02:42 2020  matrix is 10199506 x 10199683 (4836.1 MB) with weight 1480888728 (145.19/col)
Fri Apr 17 12:02:42 2020  sparse part has weight 1145353286 (112.29/col)
Fri Apr 17 12:03:55 2020  matrix starts at (0, 0)
Fri Apr 17 12:03:57 2020  matrix is 10199506 x 10199683 (4836.1 MB) with weight 1480888728 (145.19/col)
Fri Apr 17 12:03:57 2020  sparse part has weight 1145353286 (112.29/col)
Fri Apr 17 12:03:57 2020  saving the first 48 matrix rows for later
Fri Apr 17 12:03:59 2020  matrix includes 64 packed rows
Fri Apr 17 12:04:01 2020  matrix is 10199458 x 10199683 (4728.6 MB) with weight 1264348240 (123.96/col)
Fri Apr 17 12:04:01 2020  sparse part has weight 1137575108 (111.53/col)
Fri Apr 17 12:04:01 2020  using block size 8192 and superblock size 786432 for processor cache size 8192 kB
Fri Apr 17 12:04:51 2020  commencing Lanczos iteration (8 threads)
Fri Apr 17 12:04:51 2020  memory use: 3960.1 MB
Fri Apr 17 12:05:37 2020  linear algebra at 0.0%, ETA 82h 9m
Fri Apr 17 12:05:52 2020  checkpointing every 130000 dimensions

About 30 hours saved in LA, with less than 24 extra in sieving. But, as was pointed out elsewhere, the LA could have been running on a sole machine, with the sieving running for the next project. I'll have to think about this for a bit. It just might still be better, at least at this size, to go for the first viable matrix and move the rest of the farm to the next composite. Then again, I'm not nearly that organized.

VBCurtis · 2020-04-17, 18:08

Two data points that these params lead to ~12M matrices even without tons of extra sieving; at that size, I agree that adding more sieving is a waste of power.

Looks like 260M relations is enough to get to a matrix, and Ed's job has one of the highest duplicate rates I've ever seen- almost 45%! Pending further testing, let's chalk that up to an unlucky poly, and figure his 270M isn't usually necessary. Charybdis' other filtering runs show us that 5M relations of extra sieving takes half a million dimensions off the matrix; that's a tradeoff I happily take. I suppose I can put a note in the params file that jobs run on a single machine should use 250M rels_wanted.

Now that we have that setting fixed, we can tweak lambdas and ncurves etc to try to slice off another 10-20% of sieve time. Ed's log snippet shows Q=93M at the end of sieving; what we don't have is obvious indication that I=15 is faster or slower than A=28. That's not so bad, as it means they're likely fairly close in speed.

Loosening lambda settings should help on A=28, since increasing yield will reduce the Q-range and thus reduce the slow-sieving Q's. I'll wait for data from Charybdis' second run (32/32LP, I think?) before I go mucking with lambdas. Perhaps this weekend I'll try some test-sieving, even.

EdH · 2020-04-18, 14:53

I told CADO-NFS to go ahead and start LA. It currently shows krylov finishing on the 21st:

Code:

Info:Linear Algebra: krylov: N=51000 ; ETA (N=288000): Tue Apr 21 13:15:12 2020 [1.134 s/iter]

vs. msieve LA completion around 1:00 a.m. on Apr 21:

Code:

linear algebra completed 2752960 of 10199683 dimensions (27.0%, ETA 61h11m)

Of course, the rest of the CADO-NFS steps will take quite a bit longer, while the msieve square root step will take a few extra minutes.

I tried to harvest some additional data from the CADO-NFS log, but gedit isn't being cooperative. I did extract a bunch of time data:

Code:

PID2635 2020-04-07 12:01:27,455 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID2635 2020-04-07 12:12:09,925 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID29604 2020-04-08 22:52:05,674 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID29604 2020-04-08 22:52:05,676 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID10019 2020-04-09 22:48:52,134 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID10019 2020-04-09 22:48:52,137 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID12298 2020-04-12 22:30:38,662 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID12298 2020-04-12 22:30:38,664 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID22249 2020-04-13 19:59:08,517 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID22249 2020-04-13 19:59:08,518 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID23248 2020-04-13 22:47:27,140 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID23248 2020-04-13 22:47:27,150 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID15335 2020-04-15 22:45:00,195 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID15335 2020-04-15 22:45:00,196 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID21322 2020-04-16 12:08:52,799 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID21322 2020-04-16 12:08:52,800 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID25635 2020-04-16 22:52:09,492 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID25635 2020-04-16 22:52:09,494 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID31904 2020-04-17 12:46:17,245 Info:Generate Factor Base: Total cpu/real time for makefb: 158.69/31.8918
PID31904 2020-04-17 12:46:17,252 Info:Generate Free Relations: Total cpu/real time for freerel: 4950.49/642.391
PID31904 2020-04-17 14:42:34,649 Info:Filtering - Duplicate Removal, splitting pass: Total cpu/real time for dup1: 2112.98/6974.47
PID31904 2020-04-17 15:07:33,352 Info:Filtering - Duplicate Removal, removal pass: Total cpu/real time for dup2: 5388.95/1496.74
PID31904 2020-04-17 15:18:41,063 Info:Filtering - Singleton removal: Total cpu/real time for purge: 2189.08/664.827
PID31904 2020-04-17 16:45:35,564 Info:Filtering - Merging: Total cpu/real time for merge: 5613.2/4885.08
PID31904 2020-04-17 16:45:35,564 Info:Filtering - Merging: Total cpu/real time for replay: 365.12/328.686

Are there some other specifics you'd like me to find?

EdH · 2020-04-18, 14:56

Quote:

Originally Posted by VBCurtis

Two data points that these params lead to ~12M matrices even without tons of extra sieving; at that size, I agree that adding more sieving is a waste of power.

Looks like 260M relations is enough to get to a matrix, and Ed's job has one of the highest duplicate rates I've ever seen- almost 45%! Pending further testing, let's chalk that up to an unlucky poly, and figure his 270M isn't usually necessary. Charybdis' other filtering runs show us that 5M relations of extra sieving takes half a million dimensions off the matrix; that's a tradeoff I happily take. I suppose I can put a note in the params file that jobs run on a single machine should use 250M rels_wanted.

Now that we have that setting fixed, we can tweak lambdas and ncurves etc to try to slice off another 10-20% of sieve time. Ed's log snippet shows Q=93M at the end of sieving; what we don't have is obvious indication that I=15 is faster or slower than A=28. That's not so bad, as it means they're likely fairly close in speed.

Loosening lambda settings should help on A=28, since increasing yield will reduce the Q-range and thus reduce the slow-sieving Q's. I'll wait for data from Charybdis' second run (32/32LP, I think?) before I go mucking with lambdas. Perhaps this weekend I'll try some test-sieving, even.

I suppose I'm nearly ready to tackle another 17x sometime soon. If you'd like to suggest some changes to "my" params, I'd welcome giving them a shot.

EdH · 2020-04-18, 15:04

I am considering breaking some of these posts into a separate thread with a new, more accurate title. Suggestions are welcome for a title and for whether to keep the new thread in my blog area or move it into factoring. I'm thinking that loading the posts into the CADO-NFS thread would not be good.

Perhaps it's time for a CADO-NFS sub-forum?

VBCurtis · 2020-04-18, 17:02

(edited to remove CADO-requesting, since you already asked xyzzy)
Sure, I'll tweak a few settings for your next c17x. I'll post the new params here later today.

EdH · 2020-04-18, 19:52

Quote:

Originally Posted by VBCurtis

. . .
Sure, I'll tweak a few settings for your next c17x. I'll post the new params here later today.

Thanks. I've reserved 5+2,415 (12586...71 <177 dd>) for this next run.

2020-04-18, 17:02	#43
VBCurtis "Curtis" Feb 2005 Riverside, CA 5814₁₀ Posts	(edited to remove CADO-requesting, since you already asked xyzzy) Sure, I'll tweak a few settings for your next c17x. I'll post the new params here later today. Last fiddled with by VBCurtis on 2020-04-18 at 17:05

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Integers congruent to last two decimal digits mod 23	enzocreti	enzocreti	1	2020-03-03 18:38
Twin Primes with 128 Decimal Digits	tuckerkao	Miscellaneous Math	2	2020-02-16 06:23
Playing with decimal representation	Nick	Puzzles	9	2013-02-13 17:17
Decimal Value of Mersenne Prime	vsuite	GPU Computing	11	2011-02-02 04:47
Decimal Places	Corbyguy	Software	3	2008-06-09 18:09

2020-04-17, 00:33	#37
VBCurtis "Curtis" Feb 2005 Riverside, CA 2×3²×17×19 Posts	12.2M at TD 100 seems pretty good; so maybe just 10-15M more relations will get you TD 120 and 20% faster matrix time.

2020-04-17, 18:08	#39
VBCurtis "Curtis" Feb 2005 Riverside, CA 13266₈ Posts	Two data points that these params lead to ~12M matrices even without tons of extra sieving; at that size, I agree that adding more sieving is a waste of power. Looks like 260M relations is enough to get to a matrix, and Ed's job has one of the highest duplicate rates I've ever seen- almost 45%! Pending further testing, let's chalk that up to an unlucky poly, and figure his 270M isn't usually necessary. Charybdis' other filtering runs show us that 5M relations of extra sieving takes half a million dimensions off the matrix; that's a tradeoff I happily take. I suppose I can put a note in the params file that jobs run on a single machine should use 250M rels_wanted. Now that we have that setting fixed, we can tweak lambdas and ncurves etc to try to slice off another 10-20% of sieve time. Ed's log snippet shows Q=93M at the end of sieving; what we don't have is obvious indication that I=15 is faster or slower than A=28. That's not so bad, as it means they're likely fairly close in speed. Loosening lambda settings should help on A=28, since increasing yield will reduce the Q-range and thus reduce the slow-sieving Q's. I'll wait for data from Charybdis' second run (32/32LP, I think?) before I go mucking with lambdas. Perhaps this weekend I'll try some test-sieving, even.

2020-04-18, 15:04	#42
EdH "Ed Hall" Dec 2009 Adirondack Mtns 3×7×263 Posts	I am considering breaking some of these posts into a separate thread with a new, more accurate title. Suggestions are welcome for a title and for whether to keep the new thread in my blog area or move it into factoring. I'm thinking that loading the posts into the CADO-NFS thread would not be good. Perhaps it's time for a CADO-NFS sub-forum?