Some CADO-NFS Work At Around 175-180 Decimal Digits

[EdH]

I found and ran the poly for the 168 digit cofactor for 6+5,370 and got the following from cownoise:

4266086.58004      5.23508635e-13

Not sure how that compares. . .

[VBCurtis]

The records are kept in this thread, in the msieve forum:
https://mersenneforum.org/showthread...610#post539610

5.32 is the record for C168, so you were 2% shy.

[VBCurtis]

Originally Posted by EdH

Could there have been any possible advantage to my having 30+ totally separate machines searching (maybe close to 200 threads), vs. a few machines with many threads?

@VBCurtis: Are you taking into account duplication ratios? Is that something to even consider?

Both poly select and sieving are totally deterministic in CADO, so the manner in which the work is completed should have no effect.

We are noting dup ratios by way of noting the number of unique relations, which is the only count that matters for filtering.

I appreciate Charybdis noting his poly scores- that may indeed explain the speed difference! However, if it's a tie the lower LP should be used to save storage space and potentially produce smaller matrices. Perhaps C180 params should be tested with 32/32, though. Sigh, so many options!

[EdH]

Originally Posted by VBCurtis

Both poly select and sieving are totally deterministic in CADO, so the manner in which the work is completed should have no effect.

We are noting dup ratios by way of noting the number of unique relations, which is the only count that matters for filtering.

I appreciate Charybdis noting his poly scores- that may indeed explain the speed difference! However, if it's a tie the lower LP should be used to save storage space and potentially produce smaller matrices. Perhaps C180 params should be tested with 32/32, though. Sigh, so many options!

The unique relations was what I was wondering about when I "read" relations.

Unfortunately, my compiled logs don't seem to have the actual polynomials, but they do list the Murphy_E scores as computed by CADO-NFS for the chosen poly's. What I don't understand is that my score for 5+2,415 is totally different from charybdis':

5+2_415    177 (5721...) 1.440e-13

as opposed to:

Info:Polynomial Selection (root optimized): 
Finished, best polynomial has Murphy_E = 1.239e-07

[EdH]

Originally Posted by VBCurtis

The records are kept in this thread, in the msieve forum:
https://mersenneforum.org/showthread...610#post539610

5.32 is the record for C168, so you were 2% shy.

Thanks! I see I'm now listed! Thanks swellman! I suppose I'll now have to pay more attention to my polynomials.

2% shy - and I thought is was a poor poly. Didn't I have huge duplication for that one? Maybe it was the one before - darn memory - it's only great for some things.

[VBCurtis]

The poly score evaluation has no way to know how many relations will be dups; so you had a strong score on that one, but it was unlucky in that it found lots of duplicate relations so it didn't perform as well as the score would indicate.

CADO uses a different score-calculation method, one that they believe better forecasts poly performance, than the traditional Murphy E-score. cownoise finds the traditional score, which uses a fixed test area to determine score. CADO uses the actual lim's and sieve area (I or A value) and large-primes to estimate performance, so the CADO score depends on your parameter choice while the traditional Murphy E-score does not.

We use the traditional scores to compare for obvious reasons, but within a single factorization with pre-set params I think CADO is more accurately evaluating which poly will sieve best among those found during poly select.

[VBCurtis]

Originally Posted by VBCurtis

Run 4: mfb1 88, no lambda. 171000 rels/hr, ETA 62 days 5 hr. Yield 5.12
Run 5: mfb1 90, no lambda. 174500 rels/hr, ETA 59 days 15 hr. Yield 5.21

Same as run 5, except A=28 rather than I=15.
Run 6: mfb1 90, A=28. 220800 rels/hr, ETA 47 days 5 hr. Yield 3.45.

So, yield is back to the original parameters on I=15, by using 3LP instead; ETA went from 14 July to 10 June, 12 weeks down to 7! (Not really, since the target relations is both too low and the same for all settings)
Testing mfb1=92 next, then I'll mess with ncurves. Also, CADO default params switch which lim is bigger at this size, perhaps because 3LP works well with smaller lim, so I'll try that also. That requires a new run from scratch, since factor bases will change.

[swellman]

By happy chance, a 182 dd composite just fell out of ECM of the kosta project after Yoyo@Home found a p67. Specifically C182_M19_k94:

 26521232090195873108384905824300492852413283081683568418163219479089273132380406501680155963531361683795706304607082425988301635509432877463621844114521741860720947862338201013214619

You guys may want to consider it if you explore the C180-185 parameters. The SNFS poly looked ok, but a decent GNFS poly should beat it. [/shameless plug]

# 524287 ^ 47 + 1 
# MurphyE Score: 1.291e-14 anorm: 1.169e49 rnorm: 9.577e52. (Since rnorm is larger, sieve with the "-r" parameter.)
# SNFS difficulty is 274.539 which is approximately equivalent to GNFS difficulty 183. (Since n has 182 digits, it's recommended to use either SNFS or GNFS.)
# (some extra msieve library info) size: 4.922e-14 alpha: 1.228 rroots: 0
n: 26521232090195873108384905824300492852413283081683568418163219479089273132380406501680155963531361683795706304607082425988301635509432877463621844114521741860720947862338201013214619
skew: 12.16597
type: snfs
c6: 1
c5: 0
c4: 0
c3: 0
c2: 0
c1: 0
c0: 524287
Y1: 1
Y0: -5708903659119442793759136591282812149479505921
rlambda: 2.6
alambda: 2.6
lpbr: 31
lpba: 31
mfbr: 62
mfba: 62
alim: 134000000
rlim: 134000000

[charybdis]

As promised, here's some data from my third c177 run, with I=15 and 31/32LP:

Fri Apr 24 23:34:21 2020  commencing relation filtering
Fri Apr 24 23:34:21 2020  setting target matrix density to 110.0
Fri Apr 24 23:34:21 2020  estimated available RAM is 15845.4 MB
Fri Apr 24 23:34:21 2020  commencing duplicate removal, pass 1
Sat Apr 25 00:03:27 2020  found 98515288 hash collisions in 295205985 relations
Sat Apr 25 00:03:49 2020  commencing duplicate removal, pass 2
Sat Apr 25 00:09:33 2020  found 139833153 duplicates and 155372832 unique relations
...
Sat Apr 25 01:08:08 2020  matrix is 11425351 x 11425576 (4878.0 MB) with weight 1295565782 (113.39/col)
Sat Apr 25 01:08:08 2020  sparse part has weight 1164492361 (101.92/col)
Sat Apr 25 01:08:08 2020  using block size 8192 and superblock size 884736 for processor cache size 9216 kB
Sat Apr 25 01:08:39 2020  commencing Lanczos iteration (6 threads)
Sat Apr 25 01:08:39 2020  memory use: 4605.1 MB
Sat Apr 25 01:09:04 2020  linear algebra at 0.0%, ETA 50h16m

The very high duplication rate appears to be a feature of I=15 sieving with these parameters, given that Ed's first run had ~158M unique from ~294M relations. Nevertheless, I=15 seems to be 10-15% faster than A=28 at this size, judging by the CPU-time to collect ~155M unique relations (58.6M vs 67.4M CPU-seconds). The polynomial scores for these two numbers were very similar as can be seen from my last post.

Edit: Curtis, if I do a c178 next is there any parameter-testing-over-a-whole-job that you'd like me to do?

[VBCurtis]

Originally Posted by swellman

You guys may want to consider it if you explore the C180-185 parameters. The SNFS poly looked ok, but a decent GNFS poly should beat it. [/shameless plug]

# 524287 ^ 47 + 1 
# MurphyE Score: 1.291e-14 anorm: 1.169e49 rnorm: 9.577e52. (Since rnorm is larger, sieve with the "-r" parameter.)
# SNFS difficulty is 274.539 which is approximately equivalent to GNFS difficulty 183. (Since n has 182 digits, it's recommended to use either SNFS or GNFS.)

I moved Sean's post from the CADO-params thread to this one about c180ish'es.

The record poly score for a C182 is 5 times bigger than this listed SNFS poly score; I imagine GNFS will be faster even accounting for the "deg 6 / SNFS scores don't translate perfectly to deg 5" issue.

[VBCurtis]

Originally Posted by charybdis

Edit: Curtis, if I do a c178 next is there any parameter-testing-over-a-whole-job that you'd like me to do?

That's odd- all else equal, a larger siever usually produces fewer duplicate relations than a smaller siever. When I go from I=14 to I=15 on the same sized input, I reduce about 8% from rels_wanted to account for this.

Let's try the 3LP settings for your C178: MFB1 = 90, ncurves1=13, A=28, rels_wanted=320M. EDIT: Ditch the lambda1 line entirely.

I haven't yet tested new lim's; I'm using 100/140M for lim0 and lim1. I'm running mfb1=92 right now, with a lim-swap to 140/100 coming next. If you can wait a couple hours, I'll have a good idea if that should be faster.

If I grasp 3LP correctly, sieving will be faster but the matrix will be bigger. Then again, 320M is a wild guess, and we can trade some sieve time for a smaller matrix.