Some CADO-NFS Work At Around 175-180 Decimal Digits - Page 12

charybdis · 2020-09-02, 23:36

Quote:

Originally Posted by VBCurtis

Sorry for the delay, been busy with some data-gathering for nfs@home queue planning.

A params.C185 file should have the usual 25-30% increase in lim's, and we should test 32/32 against the current setting.

If we stay with 31/32, I'd add another 20-30M relations wanted. 32/32 should be 30% higher than that to start with.

Poly select should be about double the C180 file- say, 60% increase in admax and 25% increase in P.

This is what I've got for c180:

Code:

###########################################################################
# Polynomial selection
###########################################################################

tasks.polyselect.degree = 5
tasks.polyselect.P = 2500000
tasks.polyselect.admin = 10080
tasks.polyselect.admax = 22e5
tasks.polyselect.adrange = 1680
tasks.polyselect.incr = 210
tasks.polyselect.nq = 15625
tasks.polyselect.nrkeep = 96
tasks.polyselect.ropteffort = 35

###########################################################################
# Sieve
###########################################################################

tasks.I = 15
tasks.qmin = 20000000
tasks.lim0 = 95000000
tasks.lim1 = 135000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 90
tasks.sieve.lambda0 = 2.07
# tasks.sieve.lambda1 = 3.01 ?? would match what we've done with lambda0
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 13
tasks.sieve.rels_wanted = 300000000 # for a single machine; I've been aiming for around 320M
tasks.sieve.qrange = 5000

The polyselect parameters won't be optimal, but at least they produce decent polys.
The lims probably aren't optimal either. Optimising them would probably require running the same number lots of times - easy enough at c120, but a bit of an issue at c180...

Quote:

Edit: I'd also raise qmin to 25M or 30M. The most recent CADO-factorization paper mentions that controlling the qmax/qmin ratio helps to control the duplicate rate; so as our jobs get tougher and sieve up to larger Q's, qmin should rise as well. If I understood what they said properly (a weak assumption), a ratio of 7 is a decent target, and duplicate-rates get poor once the ratio exceeds 10. We saw that back when I suggested qmin of 500k, and their paper agrees with the data you gathered. We expect Q-max of 175-200M, I think?

Thanks for sharing this! A ratio of 7 does indeed line up well with what I found. I'll try (edit: changed a bit to reflect Curtis's draft c185.params)

Code:

tasks.I = 15
tasks.qmin = 30000000
tasks.lim0 = 125000000
tasks.lim1 = 175000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 90
tasks.sieve.lambda0 = 2.07
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 13

for the first c184, and we'll see if you're right about needing an extra 20M-30M relations. The next number can be the trial run for 32/32.

charybdis · 2020-09-12, 18:53

First successful attempt at a matrix for the c184 from 4+3_466:

Code:

Sat Sep 12 17:35:10 2020  Msieve v. 1.54 (SVN 1030M)
Sat Sep 12 17:35:10 2020  random seeds: 24ad7ccb b689eb64
Sat Sep 12 17:35:10 2020  factoring 1288420870966541327457423401697563128187770527956826832585494726405215165841980470637608224215821394194152554733348168114523551339058369701611634537845108590062264466133701035309124657 (184 digits)
Sat Sep 12 17:35:11 2020  searching for 15-digit factors
Sat Sep 12 17:35:11 2020  commencing number field sieve (184-digit input)
Sat Sep 12 17:35:11 2020  R0: -400690796617504803266434417211713435
Sat Sep 12 17:35:11 2020  R1: 17433125019345016821281
Sat Sep 12 17:35:11 2020  A0: 974638980914360897045449737219395727892611072
Sat Sep 12 17:35:11 2020  A1: 4219797827063945184925953662650167666
Sat Sep 12 17:35:11 2020  A2: -494543283474842279204870729055
Sat Sep 12 17:35:11 2020  A3: -10129443552045375616276
Sat Sep 12 17:35:11 2020  A4: 34163940584578
Sat Sep 12 17:35:11 2020  A5: 124740
Sat Sep 12 17:35:11 2020  skew 101124293.72, size 6.600e-18, alpha -7.549, combined = 5.544e-14 rroots = 3
Sat Sep 12 17:35:11 2020  
Sat Sep 12 17:35:11 2020  commencing relation filtering
Sat Sep 12 17:35:11 2020  setting target matrix density to 100.0
Sat Sep 12 17:35:11 2020  estimated available RAM is 15845.8 MB
Sat Sep 12 17:35:11 2020  commencing duplicate removal, pass 1
Sat Sep 12 18:05:22 2020  found 83600412 hash collisions in 300946431 relations
Sat Sep 12 18:05:43 2020  commencing duplicate removal, pass 2
Sat Sep 12 18:11:32 2020  found 98891655 duplicates and 202054776 unique relations
Sat Sep 12 18:11:32 2020  memory use: 2387.0 MB
Sat Sep 12 18:11:33 2020  reading ideals above 183959552
Sat Sep 12 18:11:33 2020  commencing singleton removal, initial pass
Sat Sep 12 18:27:10 2020  memory use: 5512.0 MB
Sat Sep 12 18:27:11 2020  reading all ideals from disk
Sat Sep 12 18:27:32 2020  memory use: 3530.8 MB
Sat Sep 12 18:27:37 2020  commencing in-memory singleton removal
Sat Sep 12 18:27:42 2020  begin with 202054776 relations and 195951001 unique ideals
Sat Sep 12 18:28:34 2020  reduce to 91808076 relations and 69726710 ideals in 18 passes
Sat Sep 12 18:28:34 2020  max relations containing the same ideal: 30
Sat Sep 12 18:28:40 2020  reading ideals above 720000
Sat Sep 12 18:28:40 2020  commencing singleton removal, initial pass
Sat Sep 12 18:40:14 2020  memory use: 2756.0 MB
Sat Sep 12 18:40:14 2020  reading all ideals from disk
Sat Sep 12 18:40:38 2020  memory use: 3788.1 MB
Sat Sep 12 18:40:45 2020  keeping 89701206 ideals with weight <= 200, target excess is 486271
Sat Sep 12 18:40:53 2020  commencing in-memory singleton removal
Sat Sep 12 18:40:59 2020  begin with 91808076 relations and 89701206 unique ideals
Sat Sep 12 18:42:20 2020  reduce to 91576225 relations and 89469280 ideals in 14 passes
Sat Sep 12 18:42:20 2020  max relations containing the same ideal: 200
Sat Sep 12 18:42:54 2020  removing 4408357 relations and 4008357 ideals in 400000 cliques
Sat Sep 12 18:42:56 2020  commencing in-memory singleton removal
Sat Sep 12 18:43:02 2020  begin with 87167868 relations and 89469280 unique ideals
Sat Sep 12 18:43:52 2020  reduce to 86998225 relations and 85290168 ideals in 9 passes
Sat Sep 12 18:43:52 2020  max relations containing the same ideal: 199
Sat Sep 12 18:44:24 2020  removing 3296653 relations and 2896653 ideals in 400000 cliques
Sat Sep 12 18:44:25 2020  commencing in-memory singleton removal
Sat Sep 12 18:44:31 2020  begin with 83701572 relations and 85290168 unique ideals
Sat Sep 12 18:45:14 2020  reduce to 83597784 relations and 82289128 ideals in 8 passes
Sat Sep 12 18:45:14 2020  max relations containing the same ideal: 195
Sat Sep 12 18:45:45 2020  removing 2943680 relations and 2543680 ideals in 400000 cliques
Sat Sep 12 18:45:46 2020  commencing in-memory singleton removal
Sat Sep 12 18:45:51 2020  begin with 80654104 relations and 82289128 unique ideals
Sat Sep 12 18:46:33 2020  reduce to 80566167 relations and 79657103 ideals in 8 passes
Sat Sep 12 18:46:33 2020  max relations containing the same ideal: 192
Sat Sep 12 18:47:02 2020  removing 2416391 relations and 2071402 ideals in 344989 cliques
Sat Sep 12 18:47:04 2020  commencing in-memory singleton removal
Sat Sep 12 18:47:09 2020  begin with 78149776 relations and 79657103 unique ideals
Sat Sep 12 18:47:49 2020  reduce to 78087981 relations and 77523656 ideals in 8 passes
Sat Sep 12 18:47:49 2020  max relations containing the same ideal: 189
Sat Sep 12 18:48:28 2020  relations with 0 large ideals: 1755
Sat Sep 12 18:48:28 2020  relations with 1 large ideals: 1308
Sat Sep 12 18:48:28 2020  relations with 2 large ideals: 26394
Sat Sep 12 18:48:28 2020  relations with 3 large ideals: 279678
Sat Sep 12 18:48:28 2020  relations with 4 large ideals: 1664509
Sat Sep 12 18:48:28 2020  relations with 5 large ideals: 6065505
Sat Sep 12 18:48:28 2020  relations with 6 large ideals: 14102865
Sat Sep 12 18:48:28 2020  relations with 7+ large ideals: 55945967
Sat Sep 12 18:48:28 2020  commencing 2-way merge
Sat Sep 12 18:49:10 2020  reduce to 47771699 relation sets and 47207374 unique ideals
Sat Sep 12 18:49:10 2020  commencing full merge
Sat Sep 12 19:01:29 2020  memory use: 5643.5 MB
Sat Sep 12 19:01:33 2020  found 22554044 cycles, need 22541574
Sat Sep 12 19:01:39 2020  weight of 22541574 cycles is about 2254542524 (100.02/cycle)
Sat Sep 12 19:01:39 2020  distribution of cycle lengths:
Sat Sep 12 19:01:39 2020  1 relations: 2505630
Sat Sep 12 19:01:39 2020  2 relations: 2281308
Sat Sep 12 19:01:39 2020  3 relations: 2240340
Sat Sep 12 19:01:39 2020  4 relations: 2050168
Sat Sep 12 19:01:39 2020  5 relations: 1873938
Sat Sep 12 19:01:39 2020  6 relations: 1689463
Sat Sep 12 19:01:39 2020  7 relations: 1480064
Sat Sep 12 19:01:39 2020  8 relations: 1281871
Sat Sep 12 19:01:39 2020  9 relations: 1129919
Sat Sep 12 19:01:39 2020  10+ relations: 6008873
Sat Sep 12 19:01:39 2020  heaviest cycle: 28 relations
Sat Sep 12 19:01:42 2020  commencing cycle optimization
Sat Sep 12 19:02:11 2020  start with 157972026 relations
Sat Sep 12 19:05:44 2020  pruned 4774693 relations
Sat Sep 12 19:05:45 2020  memory use: 4817.7 MB
Sat Sep 12 19:05:45 2020  distribution of cycle lengths:
Sat Sep 12 19:05:45 2020  1 relations: 2505630
Sat Sep 12 19:05:45 2020  2 relations: 2342283
Sat Sep 12 19:05:45 2020  3 relations: 2330295
Sat Sep 12 19:05:45 2020  4 relations: 2110706
Sat Sep 12 19:05:45 2020  5 relations: 1931570
Sat Sep 12 19:05:45 2020  6 relations: 1721214
Sat Sep 12 19:05:45 2020  7 relations: 1502020
Sat Sep 12 19:05:45 2020  8 relations: 1291081
Sat Sep 12 19:05:45 2020  9 relations: 1129255
Sat Sep 12 19:05:45 2020  10+ relations: 5677520
Sat Sep 12 19:05:45 2020  heaviest cycle: 28 relations
Sat Sep 12 19:06:23 2020  RelProcTime: 5472
Sat Sep 12 19:06:30 2020  
Sat Sep 12 19:06:30 2020  commencing linear algebra
Sat Sep 12 19:06:31 2020  read 22541574 cycles
Sat Sep 12 19:07:06 2020  cycles contain 77557788 unique relations
Sat Sep 12 19:14:45 2020  read 77557788 relations
Sat Sep 12 19:16:28 2020  using 20 quadratic characters above 4294917295
Sat Sep 12 19:21:25 2020  building initial matrix
Sat Sep 12 19:33:28 2020  memory use: 10853.9 MB
Sat Sep 12 19:34:23 2020  read 22541574 cycles
Sat Sep 12 19:34:26 2020  matrix is 22541397 x 22541574 (9232.7 MB) with weight 2833980271 (125.72/col)
Sat Sep 12 19:34:26 2020  sparse part has weight 2149800401 (95.37/col)
Sat Sep 12 19:37:38 2020  filtering completed in 2 passes
Sat Sep 12 19:37:42 2020  matrix is 22540213 x 22540390 (9232.6 MB) with weight 2833930299 (125.73/col)
Sat Sep 12 19:37:42 2020  sparse part has weight 2149790384 (95.38/col)
Sat Sep 12 19:40:48 2020  matrix starts at (0, 0)
Sat Sep 12 19:40:51 2020  matrix is 22540213 x 22540390 (9232.6 MB) with weight 2833930299 (125.73/col)
Sat Sep 12 19:40:51 2020  sparse part has weight 2149790384 (95.38/col)
Sat Sep 12 19:40:51 2020  saving the first 48 matrix rows for later
Sat Sep 12 19:40:53 2020  matrix includes 64 packed rows
Sat Sep 12 19:40:56 2020  matrix is 22540165 x 22540390 (9014.3 MB) with weight 2387432529 (105.92/col)
Sat Sep 12 19:40:56 2020  sparse part has weight 2137643846 (94.84/col)
Sat Sep 12 19:40:56 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Sat Sep 12 19:42:01 2020  commencing Lanczos iteration (6 threads)
Sat Sep 12 19:42:01 2020  memory use: 8657.6 MB
Sat Sep 12 19:43:06 2020  linear algebra at 0.0%, ETA 255h42m
Sat Sep 12 19:43:26 2020  checkpointing every 90000 dimensions

Increasing the lims doesn't seem to have changed the number of required relations all that much, but the matrix has got bigger.
It also looks like the higher lims give a bit of a speedup; I'll provide some more data once I have a more manageable matrix.

charybdis · 2020-09-13, 16:21

Sieving Q from 30M to 201M took 109.9M CPU-seconds, and produced:

Code:

Sun Sep 13 13:55:54 2020  commencing relation filtering
Sun Sep 13 13:55:54 2020  setting target matrix density to 110.0
Sun Sep 13 13:55:54 2020  estimated available RAM is 15845.8 MB
Sun Sep 13 13:55:54 2020  commencing duplicate removal, pass 1
Sun Sep 13 14:28:58 2020  found 92069979 hash collisions in 327933354 relations
Sun Sep 13 14:29:20 2020  commencing duplicate removal, pass 2
Sun Sep 13 14:35:46 2020  found 109446093 duplicates and 218487261 unique relations
Sun Sep 13 14:35:46 2020  memory use: 2387.0 MB
Sun Sep 13 14:35:46 2020  reading ideals above 200998912
Sun Sep 13 14:35:46 2020  commencing singleton removal, initial pass
Sun Sep 13 14:52:41 2020  memory use: 5512.0 MB
Sun Sep 13 14:52:42 2020  reading all ideals from disk
Sun Sep 13 14:53:09 2020  memory use: 3752.4 MB
Sun Sep 13 14:53:14 2020  commencing in-memory singleton removal
Sun Sep 13 14:53:20 2020  begin with 218487261 relations and 200826425 unique ideals
...
Sun Sep 13 16:09:52 2020  matrix is 18797898 x 18798123 (8118.4 MB) with weight 2166084557 (115.23/col)
Sun Sep 13 16:09:52 2020  sparse part has weight 1940221126 (103.21/col)
Sun Sep 13 16:09:52 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Sun Sep 13 16:10:46 2020  commencing Lanczos iteration (6 threads)
Sun Sep 13 16:10:46 2020  memory use: 7721.7 MB
Sun Sep 13 16:11:38 2020  linear algebra at 0.0%, ETA 171h51m

Higher lims don't seem to require more unique relations to build a matrix, though they do increase the duplication rate because more of the sieving is below lim1. The matrix hasn't got much larger either.
Taking into account the poly scores, speedup relative to the lower lims is about 5%.

I'm going to do the c183 from 4-3_443 next, as Sean's test-sieving showed that GNFS ought to be slightly faster than SNFS. I'll use lpb 32/32, mfb 60/90.

charybdis · 2020-09-24, 17:40

Decided to try an early filtering run to get an idea of how many relations would be needed to build a matrix, and unexpectedly found I already had enough:

Code:

Thu Sep 24 13:58:53 2020  Msieve v. 1.54 (SVN 1030M)
Thu Sep 24 13:58:53 2020  random seeds: ef4162b7 451c1899
Thu Sep 24 13:58:53 2020  factoring 804578163904697763240436614199214815395940531796301633839037380172577453559153269748944116985171617459275488471139711654222671571416371470668646221173373030303086767051992380630291851 (183 digits)
Thu Sep 24 13:58:54 2020  searching for 15-digit factors
Thu Sep 24 13:58:54 2020  commencing number field sieve (183-digit input)
Thu Sep 24 13:58:54 2020  R0: -271295764750414187331730229412933346
Thu Sep 24 13:58:54 2020  R1: 1815599719426695021034001
Thu Sep 24 13:58:54 2020  A0: -157759777296013619586250253003236919241978
Thu Sep 24 13:58:54 2020  A1: 208393498976386503111847358887987939
Thu Sep 24 13:58:54 2020  A2: 1903348446157907742464167536
Thu Sep 24 13:58:54 2020  A3: -28655357853458133620365
Thu Sep 24 13:58:54 2020  A4: -145343027262384
Thu Sep 24 13:58:54 2020  A5: 59126760
Thu Sep 24 13:58:54 2020  skew 5738604.63, size 5.111e-18, alpha -6.698, combined = 4.768e-14 rroots = 5
Thu Sep 24 13:58:54 2020  
Thu Sep 24 13:58:54 2020  commencing relation filtering
Thu Sep 24 13:58:54 2020  setting target matrix density to 100.0
Thu Sep 24 13:58:54 2020  estimated available RAM is 15845.8 MB
Thu Sep 24 13:58:54 2020  commencing duplicate removal, pass 1
...
Thu Sep 24 14:37:47 2020  found 96535007 hash collisions in 371534711 relations
Thu Sep 24 14:38:09 2020  added 121654 free relations
Thu Sep 24 14:38:09 2020  commencing duplicate removal, pass 2
Thu Sep 24 14:46:01 2020  found 108284827 duplicates and 263371538 unique relations
Thu Sep 24 14:46:01 2020  memory use: 2387.0 MB
Thu Sep 24 14:46:01 2020  reading ideals above 186974208
Thu Sep 24 14:46:01 2020  commencing singleton removal, initial pass
Thu Sep 24 15:06:32 2020  memory use: 6024.0 MB
Thu Sep 24 15:06:32 2020  reading all ideals from disk
Thu Sep 24 15:07:11 2020  memory use: 4787.5 MB
Thu Sep 24 15:07:19 2020  commencing in-memory singleton removal
Thu Sep 24 15:07:26 2020  begin with 263371538 relations and 255561661 unique ideals
Thu Sep 24 15:08:37 2020  reduce to 123121291 relations and 95945973 ideals in 17 passes
Thu Sep 24 15:08:37 2020  max relations containing the same ideal: 35
Thu Sep 24 15:08:46 2020  reading ideals above 720000
Thu Sep 24 15:08:46 2020  commencing singleton removal, initial pass
Thu Sep 24 15:24:11 2020  memory use: 3012.0 MB
Thu Sep 24 15:24:12 2020  reading all ideals from disk
Thu Sep 24 15:24:56 2020  memory use: 5110.3 MB
Thu Sep 24 15:25:06 2020  keeping 116085117 ideals with weight <= 200, target excess is 643178
Thu Sep 24 15:25:17 2020  commencing in-memory singleton removal
Thu Sep 24 15:25:25 2020  begin with 123121291 relations and 116085117 unique ideals
Thu Sep 24 15:27:04 2020  reduce to 123023063 relations and 115986878 ideals in 13 passes
Thu Sep 24 15:27:04 2020  max relations containing the same ideal: 200
Thu Sep 24 15:27:50 2020  removing 8828550 relations and 7828550 ideals in 1000000 cliques
Thu Sep 24 15:27:52 2020  commencing in-memory singleton removal
Thu Sep 24 15:28:00 2020  begin with 114194513 relations and 115986878 unique ideals
Thu Sep 24 15:29:18 2020  reduce to 113693675 relations and 107651902 ideals in 11 passes
Thu Sep 24 15:29:18 2020  max relations containing the same ideal: 198
Thu Sep 24 15:30:00 2020  removing 6670193 relations and 5670193 ideals in 1000000 cliques
Thu Sep 24 15:30:03 2020  commencing in-memory singleton removal
Thu Sep 24 15:30:10 2020  begin with 107023482 relations and 107651902 unique ideals
Thu Sep 24 15:31:10 2020  reduce to 106698207 relations and 101653297 ideals in 9 passes
Thu Sep 24 15:31:10 2020  max relations containing the same ideal: 193
Thu Sep 24 15:31:49 2020  removing 6014282 relations and 5014282 ideals in 1000000 cliques
Thu Sep 24 15:31:52 2020  commencing in-memory singleton removal
Thu Sep 24 15:31:58 2020  begin with 100683925 relations and 101653297 unique ideals
Thu Sep 24 15:32:55 2020  reduce to 100396164 relations and 96348603 ideals in 9 passes
Thu Sep 24 15:32:55 2020  max relations containing the same ideal: 184
Thu Sep 24 15:33:33 2020  removing 5675031 relations and 4675031 ideals in 1000000 cliques
Thu Sep 24 15:33:35 2020  commencing in-memory singleton removal
Thu Sep 24 15:33:41 2020  begin with 94721133 relations and 96348603 unique ideals
Thu Sep 24 15:34:28 2020  reduce to 94444992 relations and 91394891 ideals in 8 passes
Thu Sep 24 15:34:28 2020  max relations containing the same ideal: 180
Thu Sep 24 15:35:03 2020  removing 5474215 relations and 4474215 ideals in 1000000 cliques
Thu Sep 24 15:35:05 2020  commencing in-memory singleton removal
Thu Sep 24 15:35:11 2020  begin with 88970777 relations and 91394891 unique ideals
Thu Sep 24 15:36:01 2020  reduce to 88694416 relations and 86641692 ideals in 9 passes
Thu Sep 24 15:36:01 2020  max relations containing the same ideal: 171
Thu Sep 24 15:36:35 2020  removing 5350364 relations and 4350365 ideals in 1000000 cliques
Thu Sep 24 15:36:37 2020  commencing in-memory singleton removal
Thu Sep 24 15:36:42 2020  begin with 83344052 relations and 86641692 unique ideals
Thu Sep 24 15:37:28 2020  reduce to 83062640 relations and 82007106 ideals in 9 passes
Thu Sep 24 15:37:28 2020  max relations containing the same ideal: 165
Thu Sep 24 15:37:59 2020  removing 2035095 relations and 1725648 ideals in 309447 cliques
Thu Sep 24 15:38:01 2020  commencing in-memory singleton removal
Thu Sep 24 15:38:06 2020  begin with 81027545 relations and 82007106 unique ideals
Thu Sep 24 15:38:36 2020  reduce to 80987675 relations and 80241438 ideals in 6 passes
Thu Sep 24 15:38:36 2020  max relations containing the same ideal: 163
Thu Sep 24 15:39:17 2020  relations with 0 large ideals: 2387
Thu Sep 24 15:39:17 2020  relations with 1 large ideals: 2644
Thu Sep 24 15:39:17 2020  relations with 2 large ideals: 48967
Thu Sep 24 15:39:17 2020  relations with 3 large ideals: 476525
Thu Sep 24 15:39:17 2020  relations with 4 large ideals: 2541197
Thu Sep 24 15:39:17 2020  relations with 5 large ideals: 8234652
Thu Sep 24 15:39:17 2020  relations with 6 large ideals: 16947934
Thu Sep 24 15:39:17 2020  relations with 7+ large ideals: 52733369
Thu Sep 24 15:39:17 2020  commencing 2-way merge
Thu Sep 24 15:40:00 2020  reduce to 49697082 relation sets and 48950844 unique ideals
Thu Sep 24 15:40:00 2020  commencing full merge
Thu Sep 24 15:51:57 2020  memory use: 5812.0 MB
Thu Sep 24 15:52:01 2020  found 23504201 cycles, need 23427044
Thu Sep 24 15:52:07 2020  weight of 23427044 cycles is about 2342739929 (100.00/cycle)
Thu Sep 24 15:52:07 2020  distribution of cycle lengths:
Thu Sep 24 15:52:07 2020  1 relations: 2500861
Thu Sep 24 15:52:07 2020  2 relations: 2225344
Thu Sep 24 15:52:07 2020  3 relations: 2213648
Thu Sep 24 15:52:07 2020  4 relations: 2063640
Thu Sep 24 15:52:07 2020  5 relations: 1936571
Thu Sep 24 15:52:07 2020  6 relations: 1794380
Thu Sep 24 15:52:07 2020  7 relations: 1614061
Thu Sep 24 15:52:07 2020  8 relations: 1433750
Thu Sep 24 15:52:07 2020  9 relations: 1293116
Thu Sep 24 15:52:07 2020  10+ relations: 6351673
Thu Sep 24 15:52:07 2020  heaviest cycle: 27 relations
Thu Sep 24 15:52:11 2020  commencing cycle optimization
Thu Sep 24 15:52:40 2020  start with 162941027 relations
Thu Sep 24 15:56:11 2020  pruned 4844744 relations
Thu Sep 24 15:56:12 2020  memory use: 4991.4 MB
Thu Sep 24 15:56:12 2020  distribution of cycle lengths:
Thu Sep 24 15:56:12 2020  1 relations: 2500861
Thu Sep 24 15:56:12 2020  2 relations: 2281779
Thu Sep 24 15:56:12 2020  3 relations: 2297419
Thu Sep 24 15:56:12 2020  4 relations: 2125272
Thu Sep 24 15:56:12 2020  5 relations: 1998705
Thu Sep 24 15:56:12 2020  6 relations: 1832292
Thu Sep 24 15:56:12 2020  7 relations: 1647060
Thu Sep 24 15:56:12 2020  8 relations: 1454221
Thu Sep 24 15:56:12 2020  9 relations: 1305171
Thu Sep 24 15:56:12 2020  10+ relations: 5984264
Thu Sep 24 15:56:12 2020  heaviest cycle: 27 relations
Thu Sep 24 15:56:51 2020  RelProcTime: 7077
Thu Sep 24 15:56:59 2020  
Thu Sep 24 15:56:59 2020  commencing linear algebra
Thu Sep 24 15:57:01 2020  read 23427044 cycles
Thu Sep 24 15:57:36 2020  cycles contain 80452255 unique relations
Thu Sep 24 16:06:39 2020  read 80452255 relations
Thu Sep 24 16:08:27 2020  using 20 quadratic characters above 4294917295
Thu Sep 24 16:13:36 2020  building initial matrix
Thu Sep 24 16:26:16 2020  memory use: 11291.7 MB
Thu Sep 24 16:27:47 2020  read 23427044 cycles
Thu Sep 24 16:27:50 2020  matrix is 23426866 x 23427044 (9592.6 MB) with weight 2929149922 (125.03/col)
Thu Sep 24 16:27:50 2020  sparse part has weight 2233524910 (95.34/col)
Thu Sep 24 16:31:09 2020  filtering completed in 2 passes
Thu Sep 24 16:31:13 2020  matrix is 23423858 x 23424036 (9592.4 MB) with weight 2929033628 (125.04/col)
Thu Sep 24 16:31:13 2020  sparse part has weight 2233502594 (95.35/col)
Thu Sep 24 16:34:27 2020  matrix starts at (0, 0)
Thu Sep 24 16:34:30 2020  matrix is 23423858 x 23424036 (9592.4 MB) with weight 2929033628 (125.04/col)
Thu Sep 24 16:34:30 2020  sparse part has weight 2233502594 (95.35/col)
Thu Sep 24 16:34:30 2020  saving the first 48 matrix rows for later
Thu Sep 24 16:34:33 2020  matrix includes 64 packed rows
Thu Sep 24 16:34:36 2020  matrix is 23423810 x 23424036 (9296.0 MB) with weight 2454035299 (104.77/col)
Thu Sep 24 16:34:36 2020  sparse part has weight 2202649119 (94.03/col)
Thu Sep 24 16:34:36 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Thu Sep 24 16:35:39 2020  commencing Lanczos iteration (6 threads)
Thu Sep 24 16:35:40 2020  memory use: 8970.1 MB
Thu Sep 24 16:36:45 2020  linear algebra at 0.0%, ETA 265h12m
Thu Sep 24 16:37:06 2020  checkpointing every 90000 dimensions

I was caught out by the fact that the duplication rate is substantially lower than it was on the 31/32 runs; Curtis, you were right about 32/32 needing 30% more *unique* relations than 31/32.

I can't immediately see why raising the lpb on the rational side while sieving on the algebraic side ought to have anything to do with the duplication rate, but I'm not a number theorist so I'd appreciate it if someone could explain this.

It's looking like 32/32 is a win at this size. As usual I'll have more details once I've got the matrix size down.

VBCurtis · 2020-09-24, 21:56

Quote:

Originally Posted by charybdis

I was caught out by the fact that the duplication rate is substantially lower than it was on the 31/32 runs; Curtis, you were right about 32/32 needing 30% more *unique* relations than 31/32.

I can't immediately see why raising the lpb on the rational side while sieving on the algebraic side ought to have anything to do with the duplication rate, but I'm not a number theorist so I'd appreciate it if someone could explain this.

It's looking like 32/32 is a win at this size. As usual I'll have more details once I've got the matrix size down.

There's a strong chance it's just a lucky polynomial with better-than-avg duplication rate. I could see a small improvement in unique-to-raw ratio if this run needed a smaller Q range (i.e. much better yield), but otherwise I wager it's just luck.

I'm afraid the tradeoff of faster sieving vs larger matrix gets worse if we go above 32LP; but I also believe 32/33 is fastest at C193 (I helped someone off the forum run ~10 jobs at that size, we did a bunch of work on parameters). On this job, I hope that tradeoff proves worth it.

charybdis · 2020-09-25, 17:22

110.4M CPU-seconds of sieving, from 30M to 207M, gave:

Code:

Fri Sep 25 13:16:07 2020  commencing relation filtering
Fri Sep 25 13:16:07 2020  setting target matrix density to 110.0
...
Fri Sep 25 13:59:11 2020  found 108139037 hash collisions in 408503325 relations
Fri Sep 25 13:59:32 2020  commencing duplicate removal, pass 2
Fri Sep 25 14:08:03 2020  found 121841205 duplicates and 286662120 unique relations
Fri Sep 25 14:08:03 2020  memory use: 2387.0 MB
Fri Sep 25 14:08:04 2020  reading ideals above 207028224
Fri Sep 25 14:08:04 2020  commencing singleton removal, initial pass
Fri Sep 25 14:30:25 2020  memory use: 6024.0 MB
Fri Sep 25 14:30:26 2020  reading all ideals from disk
Fri Sep 25 14:31:11 2020  memory use: 5114.7 MB
Fri Sep 25 14:31:19 2020  commencing in-memory singleton removal
Fri Sep 25 14:31:27 2020  begin with 286662120 relations and 262594881 unique ideals
...
Fri Sep 25 16:00:53 2020  matrix is 19772369 x 19772594 (8473.1 MB) with weight 2251213167 (113.86/col)
Fri Sep 25 16:00:53 2020  sparse part has weight 2023439071 (102.34/col)
Fri Sep 25 16:00:53 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Fri Sep 25 16:01:50 2020  commencing Lanczos iteration (6 threads)
Fri Sep 25 16:01:50 2020  memory use: 8099.3 MB
Fri Sep 25 16:02:44 2020  linear algebra at 0.0%, ETA 188h45m

Quote:

Originally Posted by VBCurtis

There's a strong chance it's just a lucky polynomial with better-than-avg duplication rate. I could see a small improvement in unique-to-raw ratio if this run needed a smaller Q range (i.e. much better yield), but otherwise I wager it's just luck.

The polynomial score actually isn't great: 4.768e-14 for a large c183, compared to 5.544e-14 for the c184 I ran previously. It did get an unusually large boost of around 3% from the re-scoring that CADO runs at the end of polyselect, which took it up from 4th to 1st in the list of best polys - though even adding 3% to 4.768e-14 only gets you up to 4.911e-14.

If we assume that this poly is indeed ~13% worse than the c184 poly, then 32/32 is a clear win. But if the low duplication rate wasn't picked up by the poly score, then this might not be a fair comparison, so I think another run with identical parameters is in order.

charybdis · 2020-10-07, 01:47

c184 32/32 run: sieving 30M-205M took 112M CPU-seconds and gave

Code:

Tue Oct  6 23:43:20 2020  Msieve v. 1.54 (SVN 1030M)
Tue Oct  6 23:43:20 2020  random seeds: 7a19fd23 910545fb
Tue Oct  6 23:43:20 2020  factoring 1926769016895629095108654427662805571730828819453815038321063771589859906145101426691203081858286347255498278743308679252628417992530572751984914065156690326565318892677376743855798161 (184 digits)
Tue Oct  6 23:43:20 2020  searching for 15-digit factors
Tue Oct  6 23:43:20 2020  commencing number field sieve (184-digit input)
Tue Oct  6 23:43:20 2020  R0: -232800310647131682476873772810105422
Tue Oct  6 23:43:20 2020  R1: 90317080737385565963549
Tue Oct  6 23:43:20 2020  A0: 2439943766309753461417671509774240145609855
Tue Oct  6 23:43:20 2020  A1: 3109448214836060753130916105182937413
Tue Oct  6 23:43:20 2020  A2: -51506391362834289409336135888
Tue Oct  6 23:43:20 2020  A3: -20569441357157925834233
Tue Oct  6 23:43:20 2020  A4: 80188144206473
Tue Oct  6 23:43:20 2020  A5: 2817780
Tue Oct  6 23:43:20 2020  skew 20877130.61, size 5.540e-18, alpha -6.572, combined = 4.934e-14 rroots = 5
Tue Oct  6 23:43:20 2020  
Tue Oct  6 23:43:20 2020  commencing relation filtering
Tue Oct  6 23:43:20 2020  setting target matrix density to 110.0
Tue Oct  6 23:43:20 2020  estimated available RAM is 15845.8 MB
Tue Oct  6 23:43:20 2020  commencing duplicate removal, pass 1
Wed Oct  7 00:24:51 2020  found 114260066 hash collisions in 416382602 relations
Wed Oct  7 00:25:13 2020  commencing duplicate removal, pass 2
Wed Oct  7 00:33:42 2020  found 132138239 duplicates and 284244363 unique relations
Wed Oct  7 00:33:42 2020  memory use: 2387.0 MB
Wed Oct  7 00:33:42 2020  reading ideals above 205193216
Wed Oct  7 00:33:42 2020  commencing singleton removal, initial pass
Wed Oct  7 00:55:32 2020  memory use: 6024.0 MB
Wed Oct  7 00:55:33 2020  reading all ideals from disk
Wed Oct  7 00:56:13 2020  memory use: 5093.3 MB
Wed Oct  7 00:56:21 2020  commencing in-memory singleton removal
Wed Oct  7 00:56:29 2020  begin with 284244363 relations and 262016506 unique ideals
...
Wed Oct  7 02:25:05 2020  matrix is 20333308 x 20333533 (8742.9 MB) with weight 2327221335 (114.45/col)
Wed Oct  7 02:25:05 2020  sparse part has weight 2088575957 (102.72/col)
Wed Oct  7 02:25:05 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Wed Oct  7 02:26:04 2020  commencing Lanczos iteration (6 threads)
Wed Oct  7 02:26:04 2020  memory use: 8351.8 MB
Wed Oct  7 02:27:01 2020  linear algebra at 0.0%, ETA 204h37m

Duplication rate is more normal after the anomalous poly on the previous job, but 32/32 still looks like a win: compared to the last 31/32 job, sieving ran just a couple percent slower despite the poly scoring 12% worse. Even if we insist on a bit of oversieving with 32/32 to compensate for the larger matrices, that should only add a few more percent to the sieving time.

I suppose there's a chance 32/32 is already better at c180, so I ought to test that at some point. For now, though, the mfb bounds could do with some experimentation.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Integers congruent to last two decimal digits mod 23	enzocreti	enzocreti	1	2020-03-03 18:38
Twin Primes with 128 Decimal Digits	tuckerkao	Miscellaneous Math	2	2020-02-16 06:23
Playing with decimal representation	Nick	Puzzles	9	2013-02-13 17:17
Decimal Value of Mersenne Prime	vsuite	GPU Computing	11	2011-02-02 04:47
Decimal Places	Corbyguy	Software	3	2008-06-09 18:09