CADO-NFS Data Harvesting

[VBCurtis]

Not really; you'll have a slightly higher duplicate rate, meaning you may need an extra 1-3M raw relations to get the matrix size you expect. Sieve time will barely change.

[EdH]

Here's the c165:

N = 309... <165 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 523377
Polynomial Selection (root optimized): Total time: 35527
Lattice Sieving: Total time: 4.96142e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 175009615
Found 122644730 unique, 53361072 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 7.58493957e-13

Although I have another c165 available, I'm going to move back to working on the easier Aliquot sequences list for a little bit, since the number of sequences is diminishing and I want to try to make sure it doesn't get too low.

[VBCurtis]

If I've got the scaling of poly score vs sieve time, looks like 88 is quicker.

[EdH]

Originally Posted by VBCurtis

If I've got the scaling of poly score vs sieve time, looks like 88 is quicker.

Not at all empirical, but I had the feeling this was more difficult. I encountered a CADO-NFS filtering crash (possibly the strategy 2 trouble you mentioned?) and Msieve LA came back with over 32.5 hours ETA with 40 threads. I should probably save a copy of the Msieve log just for later perusal. (I wonder if I should see if a Colab instance would give me a good GPU and what it might show for ETA. . .)

ETA: I'm planning to try the Colab session, but the compression and upload times, as well as Colab inialization, are going to take quite a while, so I'm not sure there's an advantage.

ETA2: After nearly an hour and nowhere near completion, I gave up on my msieve.dat.mat.tbz upload to Google Drive for a Colab test. It had already taken over a half-hour to copy and compress it.

[EdH]

Here's a c162 I just finished:

N = 314... <162 digits>
tasks.I = 14
tasks.lim0 = 45000000
tasks.lim1 = 70000000
tasks.lpb0 = 31
tasks.lpb1 = 32
tasks.qmin = 17000000
tasks.sieve.lambda0 = 1.84
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 62
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 10000
Polynomial Selection (size optimized): Total time: 494305
Polynomial Selection (root optimized): Total time: 28582
Lattice Sieving: Total time: 3.96443e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 209104683
Found 146771345 unique, 45989579 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.11487621e-12

I'm guessing I should adjust some of these values to be closer to the params for the previous c165?

[VBCurtis]

Looks like 7-8% slower than the scaling one would expect for a C162 given the timing data from the C165 (that is, using the "double effort every 5.5 digits" heuristic).

I'd expect 3.72 megaseconds based on the 4.96 megasecond time for C165, calculated by inverting the ratio of the poly scores; the run was 3.96 megaseconds, 8% slower than target.

This may indicate 3 large primes is valuable on the C160 file, or perhaps some other parameter is not ideal. I'd try the lims and mfbs from the C165 run, with 31/31LP and 5% lower target relations (because the job is smaller/easier). I'd also use the q-min from the C165.

Ignoring the changes made for 3LP, I've been scaling the lim's to double every 10 digits. I'm not sure what is best for 3LP cases, but C165 used 60M/40M; perhaps 50M for lim0 makes sense on C160 file.

[EdH]

Thanks! I've possibly got a c160 (615...) coming up. I think I've got the changes made to the params.* I'll post what turns up.

* I've got a high rels_wanted, but Msieve is going to break in when a matrix is possible.

[EdH]

Here's the latest:

N = 615... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 25
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 493434
Polynomial Selection (root optimized): Total time: 27103.8
Lattice Sieving: Total time: 3.40375e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 169962886
Found 119992365 unique, 35401629 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.64176010e-12

[VBCurtis]

Hrmmm.... poly score nearly 50% better than the last one, but only 15% less sieve time. That's bad- especially when raw relations needed went from 209M to 169M.
Aha! I forgot: Breaking 3LP cofactors uses a small ncurves value. Leaving ncurves1 at 25 may have caused this slow job. I think the CADO default is to use ncurves of 9 or 10 on the 3LP side (20-25 on the 2LP side is still good).
So, when using mfb1 > 2 * lpb1 which is how we do 3LP on that side, drop ncurves1 from 25 to 10ish.

[EdH]

I'm running a sequence that is currently at 169 digits on its way down. It has a good chance of providing some more 16x composites. I'll drop ncurves1 to 10 for the next one, which could actually be any of the three param files (160, 165, 170). I'm somewhat set for 160 and 165, but what would you suggest for 170 if I start with the 165 params?

[EdH]

N = 344... <163 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 529204
Polynomial Selection (root optimized): Total time: 30833.4
Lattice Sieving: Total time: 4.95571e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 185882690
Found 125042962 unique, 43355067 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 9.45211518e-13