CADO-NFS Data Harvesting

[VBCurtis]

Weird; two digits easier than the C165 from a few posts ago, with the same params, yet needed 10M more raw relations, more unique relations, and took the same amount of time as a poly that scored 20%+ worse.

[EdH]

Originally Posted by VBCurtis

Weird; two digits easier than the C165 from a few posts ago, with the same params, yet needed 10M more raw relations, more unique relations, and took the same amount of time as a poly that scored 20%+ worse.

Is all of this skewed because of my use of Msieve telling CADO-NFS when to stop sieving?

Edit: Msieve runs filtering to test for a matrix success and then waits a short time and tests again.

[VBCurtis]

No, I think your way is actually more accurate, since it measures when a matrix can be built of a certain density rather than using user-set guesses for number of relations to find.

[EdH]

One more:

N = 164... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 480487
Polynomial Selection (root optimized): Total time: 27393
Lattice Sieving: Total time: 3.16854e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 166474932
Found 113244194 unique, 35828316 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.62682689e-12

[EdH]

And, a c172:

N = 237... <172 digits>
tasks.I = 14
tasks.lim0 = 65000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 985587
Polynomial Selection (root optimized): Total time: 10003
Lattice Sieving: Total time: 1.28816e+07s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 240648323
Found 151223351 unique, 81169538 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 3.16608561e-13

[EdH]

Here's a c173:

N = 107... <173 digits>
tasks.A = 28
tasks.lim0 = 65000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.adjust_strategy = 2
tasks.sieve.mfb0 = 59
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 19
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 994417
Polynomial Selection (root optimized): Total time: 9945.86
Lattice Sieving: Total time: 1.28671e+07s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 233696828
Found 145036125 unique, 73955642 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 2.69267673e-13

What changes, other than increasing qmin would be helpful? A suggestion for qmin=15M has been made. What about strategy = 2? Is that good here? It seems we removed that in the past.

[charybdis]

Originally Posted by EdH

What changes, other than increasing qmin would be helpful? A suggestion for qmin=15M has been made. What about strategy = 2? Is that good here? It seems we removed that in the past.

Strategy 2 is pretty much essential when using even values of A. The performance benefit is over 10% if I remember correctly. As already discussed in this thread, it also tends to be a few percent faster at this size even for odd values of A (i.e. when I is used instead of A).

Presumably strategy 2 was removed because of the occasional CADO filtering crashes. But with your script setup that shouldn't be an issue, as long as CADO filtering doesn't run until msieve can build a matrix.

Apart from increasing qmin, the other thing to try for small c17x would be raising lpb1 to 32, and correspondingly mfb1 to 92.

[EdH]

Thanks! I've been trying to adjust rels_wanted to make sure it exceeds Msieve matrix build relations needed, so I'll stick with strategy 2. Part of this is so the CADO filtering isn't fighting for resources while Msieve filtering is testing for a matrix. I'll adjust qmin to 15M, and raise lpb1 and mfb1 for my next c17x run, whenever I do another one.

I do run CADO filtering briefly at the end to gather the data for this thread. At that point, if it crashes, it doesn't bother my scripts, but I might miss some of the values in these posts. I can look at that later if it does occur.

[EdH]

Here's another c160:

N = 126... <160 digits>
tasks.I = 14
tasks.lim0 = 50000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 7000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 20
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 488894
Polynomial Selection (root optimized): Total time: 25992.8
Lattice Sieving: Total time: 3.49294e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 172663455
Found 118538364 unique, 41243958 duplicate, and 0 bad relations.
cownoise Best MurphyE for polynomial is 1.50177701e-12

I have since, increased qmin for both c160 and c165 params. I should have another c165 tomorrow.

BTW, I have been harvesting all my runs for a while now. If you have some specific sizes you'd be interested in, let me know and I'll post them.

[charybdis]

Comparing with earlier in the thread, it looks to me like 3LP is slower at c160.

[EdH]

Here's a c165:

N = 322... <165 digits>
tasks.I = 14
tasks.lim0 = 60000000
tasks.lim1 = 40000000
tasks.lpb0 = 31
tasks.lpb1 = 31
tasks.qmin = 12000000
tasks.sieve.lambda0 = 1.83
tasks.sieve.mfb0 = 58
tasks.sieve.mfb1 = 89
tasks.sieve.ncurves0 = 18
tasks.sieve.ncurves1 = 10
tasks.sieve.qrange = 5000
Polynomial Selection (size optimized): Total time: 513630
Polynomial Selection (root optimized): Total time: 34447.3
Lattice Sieving: Total time: 5.58991e+06s (all clients used 4 threads)
Lattice Sieving: Total number of relations: 174610477
Found 128257371 unique, 38152963 duplicate, and 0 bad relations.
 cownoise Best MurphyE for polynomial is 7.81486672e-13

How does this compare?

I had less duplication with 12M, but I can't say how time turned out. It seems to be a bit longer than a c163, as expected. You think I should increase qmin some more and/or take mfb1 down a bit?