Pascal's OPN roadblock files

[ryanp]

Originally Posted by VBCurtis

I wouldn't go up a siever size at large Q; just let it run.
On the GNFS-207 we team-sieved in 2019 with CADO, I ran 34/35LP. We dropped to I=15 after about half the expected number of relations were sieved, and the duplicate rate went well above expected. We ended up with 3300M raw relations for 1800M unique, and a matrix around 70M ensued.

Edit: your 33/36 should need a single-digit percentage more relations than my 34/35 job did.

I imagine you could get a bad matrix (like density 70-80 on msieve) now-ish with 1600M uniques, and 1900-2000M should get you a reasonable matrix.

Do you happen to know the max q that CADO's "las" binary will support? (this is somewhat new territory for me...)

[VBCurtis]

The CADO group has reported jobs with Q over 10G. It'll run a while.
It may be that Q is limited to 32-bit numbers unless one of the big-number compile flags is set; obviously we've set the one that allows LP above 32, but the one that allows more than 2^32 relations may also be needed to allow Q > 2^32. I selected these 33/36 bounds because going bigger LP would possibly need more than 2^32 raw relations and thus need that flag set.

Anyway, you're nowhere near such bounds.

[charybdis]

The siever runs just fine with Q > 2^32. There is a flag "SUPPORT_LARGE_Q" which is needed to support Q-lattice basis coordinates larger than 2^31. For this job the basis coordinates are more like 2^17 so it won't be a problem.

The authors made a comment in the code to the effect that they don't really know exactly when SUPPORT_LARGE_Q becomes necessary. They suggest it's possible the coordinates could get too large when Q > 2^30, but I expect this would be rare unless the skew is enormous. If one of the basis coordinates for a particular Q is too big, the siever prints

# Warning, special-q basis is too skewed, skipping this special-q. Define SUPPORT_LARGE_Q to proceed anyway.

Ryan and Curtis, do you by any chance happen to still have the relations from any c200+ GNFS jobs? It would be interesting to see if this warning ever appeared.

[lavalamp]

Progress on the t2200 file:

All composites larger than 2^1018 have had 224 curves @ B1=50e3, B2=13.7e6

For the ~13500 composites less than 2^1018 which can be run on GPU, I'm running 1152 curves per candidate @ B1=3e6, B2=14e9. So far 6250 composites have completed stage 1 and 4000 have also completed stage 2. It will be a couple of months before this finishes.

I've attached an updated file of the now 629 unique factors found so far.

A couple of highlights:

p50, 15623559626630713111877621077337100532126980341173
group order: 2^3 * 23 * 97 * 8,837 * 21,407 * 35,731 * 169,777 * 396,703 * 669,847 * 1,807,189 * 1,588,404,343

p53, 15179119343320317741925064897365462655253427486861001
group order: 2^2 * 3 * 7 * 11 * 13^2 * 19 * 59 * 83 * 677 * 1,049 * 8,999 * 15,277 * 34,313 * 237,163 * 331,171 * 1,089,653 * 3,643,859

[ryanp]

Originally Posted by charybdis

Ryan and Curtis, do you by any chance happen to still have the relations from any c200+ GNFS jobs? It would be interesting to see if this warning ever appeared.

Nope, sorry, I've long trashed those to make room for newer jobs...

[ryanp]

found 535727162 hash collisions in 1974476487 relations
added 1 free relations
commencing duplicate removal, pass 2
found 9 duplicates and 1974476479 unique relations
memory use: 16280.0 MB
reading ideals above 1475805184
commencing singleton removal, initial pass

Fingers crossed. This is with TD=120, which may be overly optimistic...

[VBCurtis]

How optimistic was it?

[ryanp]

Originally Posted by VBCurtis

How optimistic was it?

msieve seemed to get wedged in the "full merge" stage, so I'm trying again now, this time with 2.09B uniques and target_density=110. Running now.

[ryanp]

In case anyone is enough of an msieve fortune teller, here's how the current run is progressing so far:

found 576046017 hash collisions in 2069585215 relations
added 1 free relations
commencing duplicate removal, pass 2
found 10 duplicates and 2069585206 unique relations
memory use: 16280.0 MB
reading ideals above 1551826944
commencing singleton removal, initial pass
memory use: 41024.0 MB
reading all ideals from disk
memory use: 39164.1 MB
commencing in-memory singleton removal
begin with 2069585205 relations and 2015728165 unique ideals
reduce to 940662702 relations and 730460437 ideals in 20 passes
max relations containing the same ideal: 35
reading ideals above 720000
commencing singleton removal, initial pass
memory use: 21024.0 MB
reading all ideals from disk
memory use: 44497.0 MB
keeping 879692958 ideals with weight <= 200, target excess is 5062658
commencing in-memory singleton removal
begin with 940662704 relations and 879692958 unique ideals
reduce to 939764688 relations and 878794787 ideals in 16 passes
max relations containing the same ideal: 200
removing 9227310 relations and 9073191 ideals in 2000000 cliques
commencing in-memory singleton removal
begin with 930537378 relations and 878794787 unique ideals
reduce to 925704051 relations and 866728266 ideals in 19 passes
max relations containing the same ideal: 200
removing 7338135 relations and 6830981 ideals in 2000000 cliques
commencing in-memory singleton removal
begin with 918365916 relations and 866728266 unique ideals
reduce to 914423445 relations and 857443713 ideals in 16 passes
max relations containing the same ideal: 200
removing 6760210 relations and 6153345 ideals in 2000000 cliques
commencing in-memory singleton removal
begin with 907663235 relations and 857443713 unique ideals
reduce to 903941424 relations and 848958168 ideals in 17 passes
max relations containing the same ideal: 200
removing 6478726 relations and 5837191 ideals in 2000000 cliques
commencing in-memory singleton removal
begin with 897462698 relations and 848958168 unique ideals

[charybdis]

Originally Posted by ryanp

In case anyone is enough of an msieve fortune teller, here's how the current run is progressing so far:

That looks promising; still can't rule out msieve getting stuck in merge for an age before spitting out "too few cycles, matrix probably cannot build", but if that happens then a lower TD or more relations should do it.

If you do get a matrix I expect it will be well over 100M.

[ryanp]

Originally Posted by charybdis

That looks promising; still can't rule out msieve getting stuck in merge for an age before spitting out "too few cycles, matrix probably cannot build", but if that happens then a lower TD or more relations should do it.

If you do get a matrix I expect it will be well over 100M.

Wedged again for about 12 hours so far:

Sun Jan 10 19:56:49 2021  commencing in-memory singleton removal
Sun Jan 10 19:58:41 2021  begin with 621912609 relations and 624555581 unique ideals
Sun Jan 10 20:33:55 2021  reduce to 616867386 relations and 611731201 ideals in 17 passes
Sun Jan 10 20:33:55 2021  max relations containing the same ideal: 167
Sun Jan 10 20:37:11 2021  relations with 0 large ideals: 289519
Sun Jan 10 20:37:11 2021  relations with 1 large ideals: 190512
Sun Jan 10 20:37:11 2021  relations with 2 large ideals: 1663725
Sun Jan 10 20:37:11 2021  relations with 3 large ideals: 9813804
Sun Jan 10 20:37:11 2021  relations with 4 large ideals: 36668291
Sun Jan 10 20:37:11 2021  relations with 5 large ideals: 89881466
Sun Jan 10 20:37:11 2021  relations with 6 large ideals: 145991448
Sun Jan 10 20:37:11 2021  relations with 7+ large ideals: 332368621
Sun Jan 10 20:37:11 2021  commencing 2-way merge

I'm going to give it a bit more time, then trying adding more relations and repeating.