CADO NFS

[jyb]

Originally Posted by EdH

I hadn't really given it much thought as I was tied up for the Thanksgiving time frame, but this current Homogeneous Cunningham Number (5-2,395) is kicking my . . . I knew it would be a bit rough as a quartic, but it has been much more troublesome than I expected. After ten plus days, it is still running krylov. It stepped itself up to over 150M relations prior to that, and msieve wouldn't even build a matrix with those:

Wed Nov 27 15:42:07 2019  found 43148403 hash collisions in 150481636 relations
Wed Nov 27 15:42:17 2019  added 3657731 free relations
Wed Nov 27 15:42:17 2019  commencing duplicate removal, pass 2
Wed Nov 27 15:45:40 2019  found 58197613 duplicates and 95941754 unique relations
Wed Nov 27 15:45:40 2019  memory use: 852.8 MB
Wed Nov 27 15:45:40 2019  reading ideals above 104267776
Wed Nov 27 15:45:40 2019  commencing singleton removal, initial pass
Wed Nov 27 15:54:06 2019  memory use: 2756.0 MB
Wed Nov 27 15:54:06 2019  reading all ideals from disk
Wed Nov 27 15:54:07 2019  memory use: 1781.1 MB
Wed Nov 27 15:54:11 2019  commencing in-memory singleton removal
Wed Nov 27 15:54:14 2019  begin with 95941754 relations and 89951500 unique ideals
Wed Nov 27 15:54:53 2019  reduce to 48565925 relations and 36703205 ideals in 19 passes
Wed Nov 27 15:54:53 2019  max relations containing the same ideal: 28
Wed Nov 27 15:54:55 2019  reading ideals above 720000
Wed Nov 27 15:54:55 2019  commencing singleton removal, initial pass
Wed Nov 27 16:01:22 2019  memory use: 1378.0 MB
Wed Nov 27 16:01:22 2019  reading all ideals from disk
Wed Nov 27 16:01:23 2019  memory use: 1846.4 MB
Wed Nov 27 16:01:28 2019  keeping 48438181 ideals with weight <= 200, target excess is 250010
Wed Nov 27 16:01:32 2019  commencing in-memory singleton removal
Wed Nov 27 16:01:37 2019  begin with 48565925 relations and 48438181 unique ideals
Wed Nov 27 16:02:34 2019  reduce to 48463653 relations and 48333630 ideals in 14 passes
Wed Nov 27 16:02:34 2019  max relations containing the same ideal: 200
Wed Nov 27 16:02:38 2019  filtering wants 1000000 more relations

Did I just have bad luck, or was the polynomial a truly poor one?

n: 15511640641470902861412193110950053902622414575896230622010872491557176010272780951376514004554647568057529677786967709412454141325321707454094419046730623511926783273119854156313082841
skew: 1.0000
c4: 1
c3: 1
c2: 1
c1: 1
c0: 1
Y1: -604462909807314587353088
Y0: 16543612251060553497428173841399257071316242218017578125

I can't speak to the particulars of using CADO-NFS vs. msieve, but a quartic of difficulty 221 is really just hard. Based on prior experience (using ggnfs/msieve), it's about as hard as a sextic of difficulty 245 or 250. I.e. pushing the limit of what one person can do on personal hardware.

[EdH]

Originally Posted by jyb

I can't speak to the particulars of using CADO-NFS vs. msieve, but a quartic of difficulty 221 is really just hard. Based on prior experience (using ggnfs/msieve), it's about as hard as a sextic of difficulty 245 or 250. I.e. pushing the limit of what one person can do on personal hardware.

If this isn't out of the ordinary, then I'll continue with these quartic forced HCNs, since I'm capable. I may need to revisit a couple things, though. If I could have gotten msieve to do the LA, I could already be sieving the next one. As my setup currently stands, I have a lot of idle machines while this one finishes.

[chris2be8]

Looking at the msieve output it would probably be able to build a matrix with 10% more relations (or possibly a few % less).

But I've not done quite that large a quartic. If I had to I'd probably try test sieving the octic against the quartic (assuming it has a reasonable octic).

Chris

[EdH]

Originally Posted by chris2be8

Looking at the msieve output it would probably be able to build a matrix with 10% more relations (or possibly a few % less).

But I've not done quite that large a quartic. If I had to I'd probably try test sieving the octic against the quartic (assuming it has a reasonable octic).

Chris

Thanks,

I'll probably leave the quartics for someone else, for now, after the current two finish.

[EdH]

After more than two weeks, a bit of disappointment. (I know, two weeks is but a minor bit compared to some projects.):

Info:Quadratic Characters: Starting
Info:Quadratic Characters: Total cpu/real time for characters: 489.84/145.22
Info:Square Root: Starting
Info:Square Root: Creating file of (a,b) values
Warning:Command: Process with PID 29213 finished with return code -9
Error:Square Root: Program run on server failed with exit code -9
Error:Square Root: Command line was: /home/math90/Math/cado-nfs/build/math90/sqrt/sqrt -poly /tmp/cadofactor/snfsTest.poly -prefix /tmp/cadofactor/snfsTest.dep.gz -purged /tmp/cadofactor/snfsTest.purged.gz -index /tmp/cadofactor/snfsTest.index.gz -ker /tmp/cadofactor/snfsTest.kernel -dep 0 -t 8 -side0 -side1 -gcd > /tmp/cadofactor/snfsTest.sqrt.stdout.2 2> /tmp/cadofactor/snfsTest.sqrt.stderr.2
Error:Square Root: Stderr output (last 10 lines only) follow (stored in file /tmp/cadofactor/snfsTest.sqrt.stderr.2):
Error:Square Root:     Rat(0): read 24052344 (a,b) pairs, including 1378888 free
Error:Square Root:     Rat(1): read 24000000 pairs in 4232.32s, size 578M (peak 18720M)
Error:Square Root:     Rat(2): read 24000000 pairs in 4233.25s, size 578M (peak 18720M)
Error:Square Root:     Rat(1): read 24066996 (a,b) pairs, including 1378412 free
Error:Square Root:     Rat(2): read 24061340 (a,b) pairs, including 1378834 free
Error:Square Root:     Rat(6): size of product = 4856295942 bits (peak 24415M)
Error:Square Root:     Rat(6): starting rational square root at 4400.28s
Error:Square Root:     Rat(5): size of product = 4856688979 bits (peak 25743M)
Error:Square Root:     Rat(5): starting rational square root at 4462.11s
Error:Square Root:     
Traceback (most recent call last):
  File "./cado-nfs.py", line 122, in <module>
    factors = factorjob.run()
  File "./scripts/cadofactor/cadotask.py", line 5885, in run
    last_status, last_task = self.run_next_task()
  File "./scripts/cadofactor/cadotask.py", line 5977, in run_next_task
    return [task.run(), task.title]
  File "./scripts/cadofactor/cadotask.py", line 4871, in run
    raise Exception("Program failed")
Exception: Program failed

Can this be solved with a large swap file?

[VBCurtis]

Lame!
If swap doesn't cut it, you can upload the entire work directory to the server we used for 2330L, and complete the job there. PM me for details, if you don't finish it yourself.

[EdH]

Originally Posted by VBCurtis

Lame!
If swap doesn't cut it, you can upload the entire work directory to the server we used for 2330L, and complete the job there. PM me for details, if you don't finish it yourself.

Thanks! I'm going to "experiment" here first, because I've got another one right behind this one, that will need processing. Will msieve do the root for this, even though it refused to build a matrix?

[VBCurtis]

I haven't tried that; I'm interested to know if it works, and if it does which files you had to copy/rename to feed to msieve.

[EdH]

Originally Posted by VBCurtis

I haven't tried that; I'm interested to know if it works, and if it does which files you had to copy/rename to feed to msieve.

Well, I got out of trying the msieve approach. A large swapfile did the trick.

[SethTro]

Originally Posted by wombatman

I was able to get the linear algebra to re-run by deleting the bwc folder under the /tmp/ work directory. Just posting this in case someone else runs into the same issue (or I do again...)

I'm trying to fix what I think is a bug in mksol ETA, (in that eta seems to increase directly in proportion with iterations). I'd love to be able to rerun just that step.

deleting the bwc directory had no impact.

Does anyone know how to rerun just a mksol? or if the ETA bug has been fixed by someone else?

Thanks.

[SethTro]

By also running

sqlite3 /tmp/cado.jtdbnixu/c75.db "delete from linalg"

I was able to get mksol to rerun.