Running Msieve - Page 20

FactorEyes · 2008-02-09, 20:33

If you're going to perform post-processing with msieve, then procrels is really only useful as a sequence of samples to see where you are with your sieving, right?

Is there a way of performing this test on your raw relations, using some means other than procrels? Doesn't "msieve -v -nc1" tell you you're a few relations short of a matrix (

) if you show up without enough? Or is procrels much quicker as a meter of progress?

If you have not yet used msieve for NFS post-processing, then you're in for a treat.

bsquared · 2008-02-09, 20:45

Quote:

Originally Posted by FactorEyes

If you're going to perform post-processing with msieve, then procrels is really only useful as a sequence of samples to see where you are with your sieving, right?

If you're using factLat.pl, which uses procrels, then procrels is useful to dump out the relations in the various rels.bin* files that are created.

For big factorizations, I don't use factLat but I've always used procrels and make_msieve.pl to get the files msieve needs. I'm sure there are other ways to do it as well.

Quote:

Originally Posted by FactorEyes

If you have not yet used msieve for NFS post-processing, then you're in for a treat.

Agreed!

xilman · 2008-02-10, 11:29

Quote:

Originally Posted by FactorEyes

If you have not yet used msieve for NFS post-processing, then you're in for a treat.

I hope to find out within a day or to. So far, I've been satisfied with the CWI post-processing suite.

Paul

Andi47 · 2008-02-10, 11:49

Quote:

Originally Posted by xilman

I hope to find out within a day or to. So far, I've been satisfied with the CWI post-processing suite.

Paul

Once you have got the relations in the following form, so that the lines look like this (I guess that's what you get when dumping the relations out with procrels):

Code:

-20924856703,65467:C757,9485B,4A4893,2761C91,1D,191,5C9,2F,2F,2:3fce4b69,4488703,13A6B,1AC0F,2A59DD,5B5E51,151F,3,3,3,3,7,3D,269,5,B,8583B29
1612784043,81091:f5d08ab,40d594f3,21979,97051,17,6B,139,2,2,2,2,2:259DF,65E25,4F9C13,5B882F,1779CF1,1B91,5,5,5,5,7,11,3D,43,6D,F1,199,3,3,3,8583B29
-26779529009,296087:DB55,1C10F3,13CB25D,140387F,B,E5,397,509,2,2,2,2:2901f587,6876fdfd,6079,6947,754C1,1A4EF9,1471,3,3,3,5,7,1F,3B,97,1F3,8583B29
-653208684,3377:21004c21,8CFCB,B471F,771E27,B,277,779:924ca57,11081,79165,B367B9,1587DED,160F,959,5,7,2F,3,3,3,2,2,2,2,2,8583B29

1.) Concatenate all files to one big file named msieve.dat and add "N <number to factorize>" as first line to msieve.dat.

2.) Create a msieve.fb file which looks like this (of course using your polynomial instead of mine)

Code:

N 230380135646168002240144238096238189782429580465812519176892278271650463794969643225877877269156894108094881082195219664775471894182470295616143804362949333632033489
R0 -7253635851193924156735160443739
R1  2391424041494417171
A0  547440910672314203689898814059115360
A1  33277562211750204806364306268284
A2  107677876784557388243547221
A3 -2612363701552248486716
A4 -3795305047120954
A5  11472718320

...with N is the number to factorize.

3.) create a worktodo.ini file which contains the number to factorize.

4.) copy these three files to your msieve directory.

5.) from the command line type "msieve -v -nc -t <number of threads>"

I hope this works.

xilman · 2008-02-15, 13:39

Quote:

Originally Posted by xilman

I hope to find out within a day or to. So far, I've been satisfied with the CWI post-processing suite.

Rather longer than a day or two, but the machine has been rather busy proving the primality of over 30 thousand integers in the range P13 through P250.

The linear algebra started about 35 minutes ago, plenty long enough for me to estimate that it's close to 2.2 times faster than the CWI implementation of blocked Lanczos. Assuming it finishes and that the sqrt phase is well-behaved it seems likely I'll be using msieve in future.

My thanks to everyone who helped.

Paul

fivemack · 2008-02-17, 09:48

I've now had two large jobs which have run the linear algebra without reported error but finding only trivial dependencies. These are both running on a machine of whose reliability I am confident. One is sieved just barely up to the point of making a matrix, the other is enormously oversieved.

I'm rerunning the 151-digit job with msieve-1.29 (from before the threading modifications), which will take another three days.

Code:

Wed Feb 13 21:03:30 2008  Msieve v. 1.33
Wed Feb 13 21:03:30 2008  random seeds: b4e9a7d6 1f21cb0b
Wed Feb 13 21:03:30 2008  factoring 6217523082758401556030460236402809630656994359423664008236740148265533840342530861439938963095458807452655685629643666716576581773404701260451369712951 (151 digits)
Wed Feb 13 21:03:32 2008  no P-1/P+1/ECM available, skipping
Wed Feb 13 21:03:32 2008  commencing number field sieve (151-digit input)
Wed Feb 13 21:03:32 2008  R0: -66862108170409256564279521485
Wed Feb 13 21:03:32 2008  R1:  323116668946615313
Wed Feb 13 21:03:32 2008  A0:  23907870192659903038552206588349732
Wed Feb 13 21:03:32 2008  A1:  703126393841859924006025666960
Wed Feb 13 21:03:32 2008  A2:  2767149157482587319179015
Wed Feb 13 21:03:32 2008  A3: -3838229760711724000
Wed Feb 13 21:03:32 2008  A4: -23001392760552
Wed Feb 13 21:03:32 2008  A5:  4652940
Wed Feb 13 21:03:33 2008  size score = 1.232009e-15, Murphy alpha = -6.418885, combined = 1.046749e-14
Wed Feb 13 21:08:30 2008  restarting with 48131793 relations
Wed Feb 13 21:08:30 2008  
Wed Feb 13 21:08:30 2008  commencing relation filtering
Wed Feb 13 21:08:30 2008  commencing duplicate removal, pass 1
Wed Feb 13 21:16:03 2008  found 8086405 hash collisions in 48131793 relations
Wed Feb 13 21:16:04 2008  commencing duplicate removal, pass 2
Wed Feb 13 21:21:11 2008  found 6162724 duplicates and 41969069 unique relations
Wed Feb 13 21:21:11 2008  memory use: 270.4 MB
Wed Feb 13 21:21:34 2008  ignoring smallest 2971685 rational and 2973397 algebraic ideals
Wed Feb 13 21:21:34 2008  filtering rational ideals above 49479680
Wed Feb 13 21:21:34 2008  filtering algebraic ideals above 49479680
Wed Feb 13 21:21:34 2008  need 10106639 more relations than ideals
Wed Feb 13 21:21:34 2008  commencing singleton removal, pass 1
Wed Feb 13 21:28:44 2008  relations with 0 large ideals: 1802844
Wed Feb 13 21:28:44 2008  relations with 1 large ideals: 8159953
Wed Feb 13 21:28:44 2008  relations with 2 large ideals: 14625261
Wed Feb 13 21:28:44 2008  relations with 3 large ideals: 12293453
Wed Feb 13 21:28:44 2008  relations with 4 large ideals: 4587390
Wed Feb 13 21:28:44 2008  relations with 5 large ideals: 500168
Wed Feb 13 21:28:44 2008  relations with 6 large ideals: 0
Wed Feb 13 21:28:44 2008  relations with 7+ large ideals: 0
Wed Feb 13 21:28:44 2008  41969069 relations and about 33002452 large ideals
Wed Feb 13 21:28:44 2008  commencing singleton removal, pass 2
Wed Feb 13 21:35:56 2008  found 11602604 singletons
Wed Feb 13 21:35:56 2008  current dataset: 30366465 relations and about 19926913 large ideals
Wed Feb 13 21:35:57 2008  commencing singleton removal, pass 3
Wed Feb 13 21:42:06 2008  found 2520768 singletons
Wed Feb 13 21:42:06 2008  current dataset: 27845697 relations and about 17318061 large ideals
Wed Feb 13 21:42:07 2008  commencing singleton removal, pass 4
Wed Feb 13 21:47:33 2008  found 566221 singletons
Wed Feb 13 21:47:33 2008  current dataset: 27279476 relations and about 16746794 large ideals
Wed Feb 13 21:47:33 2008  commencing singleton removal, pass 5
Wed Feb 13 21:52:07 2008  found 124874 singletons
Wed Feb 13 21:52:07 2008  current dataset: 27154602 relations and about 16621667 large ideals
Wed Feb 13 21:52:07 2008  commencing singleton removal, final pass
Wed Feb 13 21:57:52 2008  memory use: 418.9 MB
Wed Feb 13 21:58:07 2008  commencing in-memory singleton removal
Wed Feb 13 21:58:11 2008  begin with 27154602 relations and 20271707 unique ideals
Wed Feb 13 21:58:57 2008  reduce to 20611471 relations and 13406838 ideals in 16 passes
Wed Feb 13 21:58:57 2008  max relations containing the same ideal: 34
Wed Feb 13 21:59:03 2008  dataset has 21.2% excess relations
Wed Feb 13 21:59:23 2008  ignoring smallest 2691524 rational and 2693652 algebraic ideals
Wed Feb 13 21:59:23 2008  filtering rational ideals above 44531712
Wed Feb 13 21:59:23 2008  filtering algebraic ideals above 44531712
Wed Feb 13 21:59:23 2008  need 6484169 more relations than ideals
Wed Feb 13 21:59:23 2008  commencing singleton removal, final pass
Wed Feb 13 22:04:05 2008  memory use: 322.6 MB
Wed Feb 13 22:04:16 2008  commencing in-memory singleton removal
Wed Feb 13 22:04:19 2008  begin with 20611471 relations and 13955645 unique ideals
Wed Feb 13 22:04:51 2008  reduce to 20524886 relations and 13868999 ideals in 12 passes
Wed Feb 13 22:04:51 2008  max relations containing the same ideal: 34
Wed Feb 13 22:05:11 2008  removing 948017 relations and 862158 ideals in 85859 cliques
Wed Feb 13 22:05:11 2008  commencing in-memory singleton removal
Wed Feb 13 22:05:14 2008  begin with 19576869 relations and 13868999 unique ideals
Wed Feb 13 22:05:34 2008  reduce to 19552847 relations and 12982682 ideals in 8 passes
Wed Feb 13 22:05:34 2008  max relations containing the same ideal: 33
Wed Feb 13 22:05:49 2008  removing 687302 relations and 601443 ideals in 85859 cliques
Wed Feb 13 22:05:50 2008  commencing in-memory singleton removal
Wed Feb 13 22:05:53 2008  begin with 18865545 relations and 12982682 unique ideals
Wed Feb 13 22:06:09 2008  reduce to 18851820 relations and 12367443 ideals in 7 passes
Wed Feb 13 22:06:09 2008  max relations containing the same ideal: 32
Wed Feb 13 22:06:27 2008  dataset has 9.1% excess relations
Wed Feb 13 22:06:28 2008  relations with 0 large ideals: 1329807
Wed Feb 13 22:06:28 2008  relations with 1 large ideals: 4704885
Wed Feb 13 22:06:28 2008  relations with 2 large ideals: 6636860
Wed Feb 13 22:06:28 2008  relations with 3 large ideals: 4517177
Wed Feb 13 22:06:28 2008  relations with 4 large ideals: 1477055
Wed Feb 13 22:06:28 2008  relations with 5 large ideals: 186036
Wed Feb 13 22:06:28 2008  relations with 6 large ideals: 0
Wed Feb 13 22:06:28 2008  relations with 7+ large ideals: 0
Wed Feb 13 22:06:28 2008  commencing 2-way merge
Wed Feb 13 22:06:45 2008  reduce to 12026039 relation sets and 5541662 unique ideals
Wed Feb 13 22:06:45 2008  commencing full merge
Wed Feb 13 22:08:42 2008  memory use: 514.7 MB
Wed Feb 13 22:08:42 2008  found 6339499 cycles, need 5416876
Wed Feb 13 22:08:46 2008  weight of 5416876 cycles is about 342597479 (63.25/cycle)
Wed Feb 13 22:08:46 2008  distribution of cycle lengths:
Wed Feb 13 22:08:46 2008  1 relations: 1332169
Wed Feb 13 22:08:46 2008  2 relations: 759189
Wed Feb 13 22:08:46 2008  3 relations: 590631
Wed Feb 13 22:08:46 2008  4 relations: 481540
Wed Feb 13 22:08:46 2008  5 relations: 409674
Wed Feb 13 22:08:46 2008  6 relations: 349215
Wed Feb 13 22:08:46 2008  7 relations: 302740
Wed Feb 13 22:08:46 2008  8 relations: 261779
Wed Feb 13 22:08:46 2008  9 relations: 229409
Wed Feb 13 22:08:46 2008  10+ relations: 700530
Wed Feb 13 22:08:46 2008  heaviest cycle: 15 relations
Wed Feb 13 22:08:47 2008  commencing cycle optimization
Wed Feb 13 22:08:59 2008  start with 25045770 relations
Wed Feb 13 22:10:12 2008  pruned 1134955 relations
Wed Feb 13 22:10:12 2008  memory use: 793.9 MB
Wed Feb 13 22:10:12 2008  distribution of cycle lengths:
Wed Feb 13 22:10:12 2008  1 relations: 1332169
Wed Feb 13 22:10:12 2008  2 relations: 788850
Wed Feb 13 22:10:12 2008  3 relations: 625793
Wed Feb 13 22:10:12 2008  4 relations: 500893
Wed Feb 13 22:10:12 2008  5 relations: 429357
Wed Feb 13 22:10:12 2008  6 relations: 361645
Wed Feb 13 22:10:12 2008  7 relations: 313246
Wed Feb 13 22:10:12 2008  8 relations: 267379
Wed Feb 13 22:10:12 2008  9 relations: 230371
Wed Feb 13 22:10:12 2008  10+ relations: 567173
Wed Feb 13 22:10:12 2008  heaviest cycle: 15 relations
Wed Feb 13 22:10:22 2008  
Wed Feb 13 22:10:22 2008  commencing linear algebra
Wed Feb 13 22:10:36 2008  read 5416876 cycles
Wed Feb 13 22:11:13 2008  cycles contain 13464663 unique relations
Wed Feb 13 22:15:19 2008  read 13464663 relations
Wed Feb 13 22:15:58 2008  using 32 quadratic characters above 536870612
Wed Feb 13 22:18:13 2008  building initial matrix
Wed Feb 13 22:24:26 2008  memory use: 1879.5 MB
Wed Feb 13 22:24:30 2008  read 5416876 cycles
Wed Feb 13 22:24:44 2008  matrix is 5360072 x 5416876 (1462.9 MB) with weight 490510854 (90.55/col)
Wed Feb 13 22:24:44 2008  sparse part has weight 318480484 (58.79/col)
Wed Feb 13 22:28:35 2008  filtering completed in 3 passes
Wed Feb 13 22:28:37 2008  matrix is 5026035 x 5026229 (1360.6 MB) with weight 455825733 (90.69/col)
Wed Feb 13 22:28:37 2008  sparse part has weight 296365900 (58.96/col)
Wed Feb 13 22:30:05 2008  read 5026229 cycles
Wed Feb 13 22:30:18 2008  matrix is 5026035 x 5026229 (1360.6 MB) with weight 455825733 (90.69/col)
Wed Feb 13 22:30:18 2008  sparse part has weight 296365900 (58.96/col)
Wed Feb 13 22:30:18 2008  saving the first 48 matrix rows for later
Wed Feb 13 22:30:22 2008  matrix is 5025987 x 5026229 (1294.7 MB) with weight 349634052 (69.56/col)
Wed Feb 13 22:30:22 2008  sparse part has weight 279094804 (55.53/col)
Wed Feb 13 22:30:22 2008  matrix includes 128 packed rows
Wed Feb 13 22:30:22 2008  using block size 65536 for processor cache size 4096 kB
Wed Feb 13 22:30:52 2008  commencing Lanczos iteration (2 threads)
Wed Feb 13 22:30:52 2008  memory use: 1319.4 MB
Sun Feb 17 03:28:48 2008  lanczos halted after 79479 iterations (dim = 5025576)
Sun Feb 17 03:29:01 2008  lanczos error: only trivial dependencies found

and

Code:

Fri Feb  1 18:41:19 2008  Msieve v. 1.33
Fri Feb  1 18:41:19 2008  random seeds: cbb1d6bd 5b11044a
Fri Feb  1 18:41:19 2008  factoring 230380135646168002240144238096238189782429580465812519176892278271650463794969643225877877269156894108094881082195219664775471894182470295616143804362949333632033489 (165 digits)
Fri Feb  1 18:41:20 2008  no P-1/P+1/ECM available, skipping
Fri Feb  1 18:41:20 2008  commencing number field sieve (165-digit input)
Fri Feb  1 18:41:20 2008  R0: -7253635851193924156735160443739
Fri Feb  1 18:41:20 2008  R1:  2391424041494417171
Fri Feb  1 18:41:20 2008  A0:  547440910672314203689898814059115360
Fri Feb  1 18:41:20 2008  A1:  33277562211750204806364306268284
Fri Feb  1 18:41:20 2008  A2:  107677876784557388243547221
Fri Feb  1 18:41:20 2008  A3: -2612363701552248486716
Fri Feb  1 18:41:20 2008  A4: -3795305047120954
Fri Feb  1 18:41:20 2008  A5:  11472718320
Fri Feb  1 18:41:20 2008  size score = 4.400513e-17, Murphy alpha = -7.122525, combined = 4.727096e-16
Fri Feb  1 19:13:20 2008  restarting with 228057828 relations
Fri Feb  1 19:13:20 2008  
Fri Feb  1 19:13:20 2008  commencing relation filtering
[errors reading relation]
Fri Feb  1 20:01:36 2008  found 68940645 hash collisions in 227965000 relations
Fri Feb  1 20:01:36 2008  commencing duplicate removal, pass 2
Fri Feb  1 20:33:18 2008  found 43495997 duplicates and 184469003 unique relations
Fri Feb  1 20:33:18 2008  memory use: 1911.0 MB
Fri Feb  1 20:34:21 2008  ignoring smallest 7907833 rational and 7909147 algebraic ideals
Fri Feb  1 20:34:21 2008  filtering rational ideals above 139919360
Fri Feb  1 20:34:21 2008  filtering algebraic ideals above 139919360
Fri Feb  1 20:34:21 2008  need 26888866 more relations than ideals
Fri Feb  1 20:34:21 2008  commencing singleton removal, pass 1
Fri Feb  1 21:13:34 2008  relations with 0 large ideals: 7060370
Fri Feb  1 21:13:34 2008  relations with 1 large ideals: 34969288
Fri Feb  1 21:13:34 2008  relations with 2 large ideals: 67164031
Fri Feb  1 21:13:34 2008  relations with 3 large ideals: 57050919
Fri Feb  1 21:13:34 2008  relations with 4 large ideals: 18176795
Fri Feb  1 21:13:34 2008  relations with 5 large ideals: 47592
Fri Feb  1 21:13:34 2008  relations with 6 large ideals: 8
Fri Feb  1 21:13:34 2008  relations with 7+ large ideals: 0
Fri Feb  1 21:13:34 2008  184469003 relations and about 91212880 large ideals
Fri Feb  1 21:13:34 2008  commencing singleton removal, pass 2
Fri Feb  1 21:50:50 2008  found 17752850 singletons
Fri Feb  1 21:50:51 2008  current dataset: 166716153 relations and about 72783284 large ideals
Fri Feb  1 21:50:52 2008  commencing singleton removal, pass 3
Fri Feb  1 22:26:08 2008  relations with 0 large ideals: 7060370
Fri Feb  1 22:26:08 2008  relations with 1 large ideals: 33444844
Fri Feb  1 22:26:08 2008  relations with 2 large ideals: 61387319
Fri Feb  1 22:26:08 2008  relations with 3 large ideals: 49701411
Fri Feb  1 22:26:08 2008  relations with 4 large ideals: 15082229
Fri Feb  1 22:26:08 2008  relations with 5 large ideals: 39973
Fri Feb  1 22:26:08 2008  relations with 6 large ideals: 7
Fri Feb  1 22:26:08 2008  relations with 7+ large ideals: 0
Fri Feb  1 22:26:08 2008  166716153 relations and about 103590738 large ideals
Fri Feb  1 22:26:08 2008  commencing singleton removal, pass 4
Fri Feb  1 23:07:12 2008  found 24652189 singletons
Fri Feb  1 23:07:12 2008  current dataset: 142063964 relations and about 77434141 large ideals
Fri Feb  1 23:07:12 2008  commencing singleton removal, pass 5
Fri Feb  1 23:37:51 2008  relations with 0 large ideals: 7060370
Fri Feb  1 23:37:51 2008  relations with 1 large ideals: 31100390
Fri Feb  1 23:37:51 2008  relations with 2 large ideals: 53029094
Fri Feb  1 23:37:51 2008  relations with 3 large ideals: 39711529
Fri Feb  1 23:37:51 2008  relations with 4 large ideals: 11132819
Fri Feb  1 23:37:51 2008  relations with 5 large ideals: 29756
Fri Feb  1 23:37:51 2008  relations with 6 large ideals: 6
Fri Feb  1 23:37:51 2008  relations with 7+ large ideals: 0
Fri Feb  1 23:37:51 2008  142063964 relations and about 93534301 large ideals
Fri Feb  1 23:37:51 2008  commencing singleton removal, pass 6
Sat Feb  2 00:08:20 2008  found 17505340 singletons
Sat Feb  2 00:08:20 2008  current dataset: 124558624 relations and about 75155405 large ideals
Sat Feb  2 00:08:21 2008  commencing singleton removal, pass 7
Sat Feb  2 00:36:15 2008  found 3747152 singletons
Sat Feb  2 00:36:15 2008  current dataset: 120811472 relations and about 71361086 large ideals
Sat Feb  2 00:36:16 2008  commencing singleton removal, pass 8
Sat Feb  2 01:09:40 2008  found 711486 singletons
Sat Feb  2 01:09:40 2008  current dataset: 120099986 relations and about 70647739 large ideals
Sat Feb  2 01:09:41 2008  commencing singleton removal, pass 9
Sat Feb  2 01:37:50 2008  found 126821 singletons
Sat Feb  2 01:37:50 2008  current dataset: 119973165 relations and about 70520867 large ideals
Sat Feb  2 01:37:51 2008  commencing singleton removal, final pass
Sat Feb  2 02:09:45 2008  memory use: 1847.4 MB
Sat Feb  2 02:11:26 2008  commencing in-memory singleton removal
Sat Feb  2 02:11:39 2008  begin with 119973165 relations and 79045425 unique ideals
Sat Feb  2 02:14:18 2008  reduce to 112705469 relations and 71683219 ideals in 12 passes
Sat Feb  2 02:14:18 2008  max relations containing the same ideal: 69
Sat Feb  2 02:15:44 2008  removing 3362193 relations and 2962193 ideals in 400000 cliques
Sat Feb  2 02:15:47 2008  commencing in-memory singleton removal
Sat Feb  2 02:15:59 2008  begin with 109343276 relations and 71683219 unique ideals
Sat Feb  2 02:17:24 2008  reduce to 109296016 relations and 68673568 ideals in 7 passes
Sat Feb  2 02:17:24 2008  max relations containing the same ideal: 65
Sat Feb  2 02:18:28 2008  removing 2516523 relations and 2116523 ideals in 400000 cliques
Sat Feb  2 02:18:31 2008  commencing in-memory singleton removal
Sat Feb  2 02:18:42 2008  begin with 106779493 relations and 68673568 unique ideals
Sat Feb  2 02:19:53 2008  reduce to 106750234 relations and 66527684 ideals in 6 passes
Sat Feb  2 02:19:53 2008  max relations containing the same ideal: 63
Sat Feb  2 02:20:55 2008  removing 2242829 relations and 1842829 ideals in 400000 cliques
Sat Feb  2 02:20:57 2008  commencing in-memory singleton removal
Sat Feb  2 02:21:08 2008  begin with 104507405 relations and 66527684 unique ideals
Sat Feb  2 02:22:29 2008  reduce to 104483539 relations and 64660914 ideals in 7 passes
Sat Feb  2 02:22:29 2008  max relations containing the same ideal: 63
Sat Feb  2 02:23:29 2008  removing 2073230 relations and 1673230 ideals in 400000 cliques
Sat Feb  2 02:23:31 2008  commencing in-memory singleton removal
Sat Feb  2 02:23:42 2008  begin with 102410309 relations and 64660914 unique ideals
Sat Feb  2 02:24:49 2008  reduce to 102388657 relations and 62965953 ideals in 6 passes
Sat Feb  2 02:24:49 2008  max relations containing the same ideal: 61
Sat Feb  2 02:25:47 2008  removing 1960237 relations and 1560237 ideals in 400000 cliques
Sat Feb  2 02:25:49 2008  commencing in-memory singleton removal
Sat Feb  2 02:25:59 2008  begin with 100428420 relations and 62965953 unique ideals
Sat Feb  2 02:27:05 2008  reduce to 100409198 relations and 61386440 ideals in 6 passes
Sat Feb  2 02:27:05 2008  max relations containing the same ideal: 60
Sat Feb  2 02:28:01 2008  removing 1871974 relations and 1471974 ideals in 400000 cliques
Sat Feb  2 02:28:04 2008  commencing in-memory singleton removal
Sat Feb  2 02:28:14 2008  begin with 98537224 relations and 61386440 unique ideals
Sat Feb  2 02:29:17 2008  reduce to 98518901 relations and 59896081 ideals in 6 passes
Sat Feb  2 02:29:17 2008  max relations containing the same ideal: 59
Sat Feb  2 02:30:12 2008  removing 1800539 relations and 1400539 ideals in 400000 cliques
Sat Feb  2 02:30:14 2008  commencing in-memory singleton removal
Sat Feb  2 02:30:24 2008  begin with 96718362 relations and 59896081 unique ideals
Sat Feb  2 02:31:16 2008  reduce to 96700186 relations and 58477295 ideals in 5 passes
Sat Feb  2 02:31:16 2008  max relations containing the same ideal: 59
Sat Feb  2 02:32:09 2008  removing 1736333 relations and 1336333 ideals in 400000 cliques
Sat Feb  2 02:32:11 2008  commencing in-memory singleton removal
Sat Feb  2 02:32:21 2008  begin with 94963853 relations and 58477295 unique ideals
Sat Feb  2 02:33:11 2008  reduce to 94946533 relations and 57123572 ideals in 5 passes
Sat Feb  2 02:33:11 2008  max relations containing the same ideal: 59
Sat Feb  2 02:34:04 2008  removing 1701647 relations and 1301647 ideals in 400000 cliques
Sat Feb  2 02:34:06 2008  commencing in-memory singleton removal
Sat Feb  2 02:34:15 2008  begin with 93244886 relations and 57123572 unique ideals
Sat Feb  2 02:35:04 2008  reduce to 93229008 relations and 55805997 ideals in 5 passes
Sat Feb  2 02:35:04 2008  max relations containing the same ideal: 57
Sat Feb  2 02:35:55 2008  removing 1646287 relations and 1246287 ideals in 400000 cliques
Sat Feb  2 02:35:57 2008  commencing in-memory singleton removal
Sat Feb  2 02:36:06 2008  begin with 91582721 relations and 55805997 unique ideals
Sat Feb  2 02:36:54 2008  reduce to 91564682 relations and 54541598 ideals in 5 passes
Sat Feb  2 02:36:54 2008  max relations containing the same ideal: 56
Sat Feb  2 02:37:44 2008  removing 1630159 relations and 1230159 ideals in 400000 cliques
Sat Feb  2 02:37:46 2008  commencing in-memory singleton removal
Sat Feb  2 02:37:55 2008  begin with 89934523 relations and 54541598 unique ideals
Sat Feb  2 02:38:42 2008  reduce to 89920991 relations and 53297855 ideals in 5 passes
Sat Feb  2 02:38:42 2008  max relations containing the same ideal: 54
Sat Feb  2 02:39:30 2008  removing 1573058 relations and 1173058 ideals in 400000 cliques
Sat Feb  2 02:39:33 2008  commencing in-memory singleton removal
Sat Feb  2 02:39:41 2008  begin with 88347933 relations and 53297855 unique ideals
Sat Feb  2 02:40:27 2008  reduce to 88332948 relations and 52109743 ideals in 5 passes
Sat Feb  2 02:40:27 2008  max relations containing the same ideal: 54
Sat Feb  2 02:41:14 2008  removing 1563378 relations and 1163378 ideals in 400000 cliques
Sat Feb  2 02:41:16 2008  commencing in-memory singleton removal
Sat Feb  2 02:41:25 2008  begin with 86769570 relations and 52109743 unique ideals
Sat Feb  2 02:42:09 2008  reduce to 86755283 relations and 50932023 ideals in 5 passes
Sat Feb  2 02:42:09 2008  max relations containing the same ideal: 54
Sat Feb  2 02:42:55 2008  removing 1534319 relations and 1134319 ideals in 400000 cliques
Sat Feb  2 02:42:57 2008  commencing in-memory singleton removal
Sat Feb  2 02:43:05 2008  begin with 85220964 relations and 50932023 unique ideals
Sat Feb  2 02:43:49 2008  reduce to 85205994 relations and 49782672 ideals in 5 passes
Sat Feb  2 02:43:49 2008  max relations containing the same ideal: 54
Sat Feb  2 02:44:34 2008  removing 1503446 relations and 1103446 ideals in 400000 cliques
Sat Feb  2 02:44:36 2008  commencing in-memory singleton removal
Sat Feb  2 02:44:44 2008  begin with 83702548 relations and 49782672 unique ideals
Sat Feb  2 02:45:26 2008  reduce to 83686012 relations and 48662625 ideals in 5 passes
Sat Feb  2 02:45:26 2008  max relations containing the same ideal: 54
Sat Feb  2 02:46:10 2008  removing 1486477 relations and 1086477 ideals in 400000 cliques
Sat Feb  2 02:46:12 2008  commencing in-memory singleton removal
Sat Feb  2 02:46:20 2008  begin with 82199535 relations and 48662625 unique ideals
Sat Feb  2 02:47:02 2008  reduce to 82184256 relations and 47560798 ideals in 5 passes
Sat Feb  2 02:47:02 2008  max relations containing the same ideal: 53
Sat Feb  2 02:47:44 2008  removing 1458695 relations and 1058695 ideals in 400000 cliques
Sat Feb  2 02:47:46 2008  commencing in-memory singleton removal
Sat Feb  2 02:47:54 2008  begin with 80725561 relations and 47560798 unique ideals
Sat Feb  2 02:48:35 2008  reduce to 80711890 relations and 46488351 ideals in 5 passes
Sat Feb  2 02:48:35 2008  max relations containing the same ideal: 52
Sat Feb  2 02:49:16 2008  removing 1432791 relations and 1032791 ideals in 400000 cliques
Sat Feb  2 02:49:18 2008  commencing in-memory singleton removal
Sat Feb  2 02:49:26 2008  begin with 79279099 relations and 46488351 unique ideals
Sat Feb  2 02:50:05 2008  reduce to 79264864 relations and 45441264 ideals in 5 passes
Sat Feb  2 02:50:05 2008  max relations containing the same ideal: 51
Sat Feb  2 02:50:46 2008  removing 1406061 relations and 1006061 ideals in 400000 cliques
Sat Feb  2 02:50:48 2008  commencing in-memory singleton removal
Sat Feb  2 02:50:55 2008  begin with 77858803 relations and 45441264 unique ideals
Sat Feb  2 02:51:41 2008  reduce to 77842951 relations and 44419272 ideals in 6 passes
Sat Feb  2 02:51:41 2008  max relations containing the same ideal: 50
Sat Feb  2 02:52:21 2008  removing 1420018 relations and 1020018 ideals in 400000 cliques
Sat Feb  2 02:52:23 2008  commencing in-memory singleton removal
Sat Feb  2 02:52:30 2008  begin with 76422933 relations and 44419272 unique ideals
Sat Feb  2 02:53:00 2008  reduce to 76408633 relations and 43384876 ideals in 4 passes
Sat Feb  2 02:53:00 2008  max relations containing the same ideal: 49
Sat Feb  2 02:53:39 2008  removing 1385105 relations and 985105 ideals in 400000 cliques
Sat Feb  2 02:53:41 2008  commencing in-memory singleton removal
Sat Feb  2 02:53:48 2008  begin with 75023528 relations and 43384876 unique ideals
Sat Feb  2 02:54:25 2008  reduce to 75008306 relations and 42384466 ideals in 5 passes
Sat Feb  2 02:54:25 2008  max relations containing the same ideal: 47
Sat Feb  2 02:55:03 2008  removing 1386688 relations and 986688 ideals in 400000 cliques
Sat Feb  2 02:55:04 2008  commencing in-memory singleton removal
Sat Feb  2 02:55:11 2008  begin with 73621618 relations and 42384466 unique ideals
Sat Feb  2 02:55:47 2008  reduce to 73606900 relations and 41382984 ideals in 5 passes
Sat Feb  2 02:55:47 2008  max relations containing the same ideal: 47
Sat Feb  2 02:56:24 2008  removing 1367535 relations and 967535 ideals in 400000 cliques
Sat Feb  2 02:56:26 2008  commencing in-memory singleton removal
Sat Feb  2 02:56:32 2008  begin with 72239365 relations and 41382984 unique ideals
Sat Feb  2 02:57:01 2008  reduce to 72223753 relations and 40399755 ideals in 4 passes
Sat Feb  2 02:57:01 2008  max relations containing the same ideal: 46
Sat Feb  2 02:57:37 2008  removing 1351460 relations and 951460 ideals in 400000 cliques
Sat Feb  2 02:57:38 2008  commencing in-memory singleton removal
Sat Feb  2 02:57:45 2008  begin with 70872293 relations and 40399755 unique ideals
Sat Feb  2 02:58:19 2008  reduce to 70855135 relations and 39431032 ideals in 5 passes
Sat Feb  2 02:58:19 2008  max relations containing the same ideal: 45
Sat Feb  2 02:58:54 2008  removing 1340482 relations and 940482 ideals in 400000 cliques
Sat Feb  2 02:58:56 2008  commencing in-memory singleton removal
Sat Feb  2 02:59:02 2008  begin with 69514653 relations and 39431032 unique ideals
Sat Feb  2 02:59:36 2008  reduce to 69496474 relations and 38472235 ideals in 5 passes
Sat Feb  2 02:59:36 2008  max relations containing the same ideal: 45
Sat Feb  2 03:00:10 2008  removing 1328477 relations and 928477 ideals in 400000 cliques
Sat Feb  2 03:00:12 2008  commencing in-memory singleton removal
Sat Feb  2 03:00:18 2008  begin with 68167997 relations and 38472235 unique ideals
Sat Feb  2 03:00:51 2008  reduce to 68152003 relations and 37527626 ideals in 5 passes
Sat Feb  2 03:00:51 2008  max relations containing the same ideal: 45
Sat Feb  2 03:01:24 2008  removing 1306587 relations and 906587 ideals in 400000 cliques
Sat Feb  2 03:01:26 2008  commencing in-memory singleton removal
Sat Feb  2 03:01:32 2008  begin with 66845416 relations and 37527626 unique ideals
Sat Feb  2 03:02:04 2008  reduce to 66830237 relations and 36605742 ideals in 5 passes
Sat Feb  2 03:02:04 2008  max relations containing the same ideal: 45
Sat Feb  2 03:02:36 2008  removing 1298668 relations and 898668 ideals in 400000 cliques
Sat Feb  2 03:02:38 2008  commencing in-memory singleton removal
Sat Feb  2 03:02:44 2008  begin with 65531569 relations and 36605742 unique ideals
Sat Feb  2 03:03:15 2008  reduce to 65516826 relations and 35692229 ideals in 5 passes
Sat Feb  2 03:03:15 2008  max relations containing the same ideal: 45
Sat Feb  2 03:03:46 2008  removing 1285876 relations and 885876 ideals in 400000 cliques
Sat Feb  2 03:03:48 2008  commencing in-memory singleton removal
Sat Feb  2 03:03:53 2008  begin with 64230950 relations and 35692229 unique ideals
Sat Feb  2 03:04:30 2008  reduce to 64215889 relations and 34791183 ideals in 6 passes
Sat Feb  2 03:04:30 2008  max relations containing the same ideal: 44
Sat Feb  2 03:05:00 2008  removing 1276314 relations and 876314 ideals in 400000 cliques
Sat Feb  2 03:05:02 2008  commencing in-memory singleton removal
Sat Feb  2 03:05:07 2008  begin with 62939575 relations and 34791183 unique ideals
Sat Feb  2 03:05:31 2008  reduce to 62924457 relations and 33899638 ideals in 4 passes
Sat Feb  2 03:05:31 2008  max relations containing the same ideal: 44
Sat Feb  2 03:06:01 2008  removing 1266553 relations and 866553 ideals in 400000 cliques
Sat Feb  2 03:06:02 2008  commencing in-memory singleton removal
Sat Feb  2 03:06:08 2008  begin with 61657904 relations and 33899638 unique ideals
Sat Feb  2 03:06:36 2008  reduce to 61642021 relations and 33017093 ideals in 5 passes
Sat Feb  2 03:06:36 2008  max relations containing the same ideal: 44
Sat Feb  2 03:07:05 2008  removing 1269571 relations and 869571 ideals in 400000 cliques
Sat Feb  2 03:07:07 2008  commencing in-memory singleton removal
Sat Feb  2 03:07:12 2008  begin with 60372450 relations and 33017093 unique ideals
Sat Feb  2 03:07:45 2008  reduce to 60357820 relations and 32132801 ideals in 6 passes
Sat Feb  2 03:07:45 2008  max relations containing the same ideal: 41
Sat Feb  2 03:08:13 2008  removing 1264461 relations and 864461 ideals in 400000 cliques
Sat Feb  2 03:08:15 2008  commencing in-memory singleton removal
Sat Feb  2 03:08:20 2008  begin with 59093359 relations and 32132801 unique ideals
Sat Feb  2 03:08:47 2008  reduce to 59079551 relations and 31254448 ideals in 5 passes
Sat Feb  2 03:08:47 2008  max relations containing the same ideal: 41
Sat Feb  2 03:09:14 2008  removing 1246120 relations and 846120 ideals in 400000 cliques
Sat Feb  2 03:09:15 2008  commencing in-memory singleton removal
Sat Feb  2 03:09:20 2008  begin with 57833431 relations and 31254448 unique ideals
Sat Feb  2 03:09:47 2008  reduce to 57818623 relations and 30393423 ideals in 5 passes
Sat Feb  2 03:09:47 2008  max relations containing the same ideal: 41
Sat Feb  2 03:10:13 2008  removing 1249893 relations and 849893 ideals in 400000 cliques
Sat Feb  2 03:10:14 2008  commencing in-memory singleton removal
Sat Feb  2 03:10:19 2008  begin with 56568730 relations and 30393423 unique ideals
Sat Feb  2 03:10:45 2008  reduce to 56554433 relations and 29529107 ideals in 5 passes
Sat Feb  2 03:10:45 2008  max relations containing the same ideal: 41
Sat Feb  2 03:11:10 2008  removing 487390 relations and 350931 ideals in 136459 cliques
Sat Feb  2 03:11:11 2008  commencing in-memory singleton removal
Sat Feb  2 03:11:16 2008  begin with 56067043 relations and 29529107 unique ideals
Sat Feb  2 03:11:36 2008  reduce to 56064949 relations and 29176081 ideals in 4 passes
Sat Feb  2 03:11:36 2008  max relations containing the same ideal: 41
Sat Feb  2 03:11:51 2008  dataset has 70.0% excess relations
Sat Feb  2 03:12:48 2008  ignoring smallest 7159917 rational and 7160002 algebraic ideals
Sat Feb  2 03:12:48 2008  filtering rational ideals above 125927424
Sat Feb  2 03:12:48 2008  filtering algebraic ideals above 125927424
Sat Feb  2 03:12:48 2008  need 24199981 more relations than ideals
Sat Feb  2 03:12:48 2008  commencing singleton removal, final pass
Sat Feb  2 03:36:17 2008  memory use: 800.1 MB
Sat Feb  2 03:37:08 2008  commencing in-memory singleton removal
Sat Feb  2 03:37:13 2008  begin with 56064949 relations and 30649971 unique ideals
Sat Feb  2 03:37:42 2008  reduce to 55942514 relations and 30527459 ideals in 6 passes
Sat Feb  2 03:37:42 2008  max relations containing the same ideal: 44
Sat Feb  2 03:38:17 2008  removing 1250788 relations and 850788 ideals in 400000 cliques
Sat Feb  2 03:38:19 2008  commencing in-memory singleton removal
Sat Feb  2 03:38:23 2008  begin with 54691726 relations and 30527459 unique ideals
Sat Feb  2 03:38:46 2008  reduce to 54668862 relations and 29653415 ideals in 5 passes
Sat Feb  2 03:38:46 2008  max relations containing the same ideal: 43
Sat Feb  2 03:39:10 2008  removing 1242032 relations and 842032 ideals in 400000 cliques
Sat Feb  2 03:39:11 2008  commencing in-memory singleton removal
Sat Feb  2 03:39:15 2008  begin with 53426830 relations and 29653415 unique ideals
Sat Feb  2 03:39:34 2008  reduce to 53410290 relations and 28794675 ideals in 4 passes
Sat Feb  2 03:39:34 2008  max relations containing the same ideal: 42
Sat Feb  2 03:39:56 2008  removing 1232023 relations and 832023 ideals in 400000 cliques
Sat Feb  2 03:39:58 2008  commencing in-memory singleton removal
Sat Feb  2 03:40:02 2008  begin with 52178267 relations and 28794675 unique ideals
Sat Feb  2 03:40:24 2008  reduce to 52161123 relations and 27945366 ideals in 5 passes
Sat Feb  2 03:40:24 2008  max relations containing the same ideal: 41
Sat Feb  2 03:40:46 2008  removing 69595 relations and 53820 ideals in 15775 cliques
Sat Feb  2 03:40:47 2008  commencing in-memory singleton removal
Sat Feb  2 03:40:51 2008  begin with 52091528 relations and 27945366 unique ideals
Sat Feb  2 03:41:04 2008  reduce to 52091468 relations and 27891486 ideals in 3 passes
Sat Feb  2 03:41:04 2008  max relations containing the same ideal: 41
Sat Feb  2 03:41:17 2008  dataset has 53.0% excess relations
Sat Feb  2 03:42:07 2008  ignoring smallest 6407562 rational and 6407473 algebraic ideals
Sat Feb  2 03:42:07 2008  filtering rational ideals above 111935488
Sat Feb  2 03:42:07 2008  filtering algebraic ideals above 111935488
Sat Feb  2 03:42:07 2008  need 19359985 more relations than ideals
Sat Feb  2 03:42:07 2008  commencing singleton removal, final pass
Sat Feb  2 04:05:31 2008  memory use: 771.3 MB
Sat Feb  2 04:06:27 2008  commencing in-memory singleton removal
Sat Feb  2 04:06:31 2008  begin with 52091468 relations and 29376445 unique ideals
Sat Feb  2 04:06:55 2008  reduce to 51980098 relations and 29265023 ideals in 5 passes
Sat Feb  2 04:06:55 2008  max relations containing the same ideal: 41
Sat Feb  2 04:07:28 2008  removing 1280663 relations and 880663 ideals in 400000 cliques
Sat Feb  2 04:07:29 2008  commencing in-memory singleton removal
Sat Feb  2 04:07:33 2008  begin with 50699435 relations and 29265023 unique ideals
Sat Feb  2 04:07:55 2008  reduce to 50675789 relations and 28360376 ideals in 5 passes
Sat Feb  2 04:07:55 2008  max relations containing the same ideal: 40
Sat Feb  2 04:08:18 2008  removing 1234184 relations and 834184 ideals in 400000 cliques
Sat Feb  2 04:08:19 2008  commencing in-memory singleton removal
Sat Feb  2 04:08:23 2008  begin with 49441605 relations and 28360376 unique ideals
Sat Feb  2 04:08:44 2008  reduce to 49422269 relations and 27506635 ideals in 5 passes
Sat Feb  2 04:08:44 2008  max relations containing the same ideal: 40
Sat Feb  2 04:09:06 2008  removing 1221500 relations and 821500 ideals in 400000 cliques
Sat Feb  2 04:09:07 2008  commencing in-memory singleton removal
Sat Feb  2 04:09:11 2008  begin with 48200769 relations and 27506635 unique ideals
Sat Feb  2 04:09:32 2008  reduce to 48181209 relations and 26665385 ideals in 5 passes
Sat Feb  2 04:09:32 2008  max relations containing the same ideal: 40
Sat Feb  2 04:09:53 2008  removing 1215025 relations and 815025 ideals in 400000 cliques
Sat Feb  2 04:09:54 2008  commencing in-memory singleton removal
Sat Feb  2 04:09:58 2008  begin with 46966184 relations and 26665385 unique ideals
Sat Feb  2 04:10:17 2008  reduce to 46945582 relations and 25829526 ideals in 5 passes
Sat Feb  2 04:10:17 2008  max relations containing the same ideal: 39
Sat Feb  2 04:10:38 2008  removing 1208649 relations and 808649 ideals in 400000 cliques
Sat Feb  2 04:10:39 2008  commencing in-memory singleton removal
Sat Feb  2 04:10:43 2008  begin with 45736933 relations and 25829526 unique ideals
Sat Feb  2 04:11:02 2008  reduce to 45715487 relations and 24999189 ideals in 5 passes
Sat Feb  2 04:11:02 2008  max relations containing the same ideal: 39
Sat Feb  2 04:11:22 2008  removing 1200712 relations and 800712 ideals in 400000 cliques
Sat Feb  2 04:11:23 2008  commencing in-memory singleton removal
Sat Feb  2 04:11:26 2008  begin with 44514775 relations and 24999189 unique ideals
Sat Feb  2 04:11:41 2008  reduce to 44492252 relations and 24175675 ideals in 4 passes
Sat Feb  2 04:11:41 2008  max relations containing the same ideal: 38
Sat Feb  2 04:12:00 2008  removing 1183772 relations and 783772 ideals in 400000 cliques
Sat Feb  2 04:12:02 2008  commencing in-memory singleton removal
Sat Feb  2 04:12:05 2008  begin with 43308480 relations and 24175675 unique ideals
Sat Feb  2 04:12:19 2008  reduce to 43283180 relations and 23366236 ideals in 4 passes
Sat Feb  2 04:12:19 2008  max relations containing the same ideal: 38
Sat Feb  2 04:12:38 2008  removing 1206714 relations and 806714 ideals in 400000 cliques
Sat Feb  2 04:12:39 2008  commencing in-memory singleton removal
Sat Feb  2 04:12:42 2008  begin with 42076466 relations and 23366236 unique ideals
Sat Feb  2 04:12:56 2008  reduce to 42058239 relations and 22540970 ideals in 4 passes
Sat Feb  2 04:12:56 2008  max relations containing the same ideal: 37
Sat Feb  2 04:13:14 2008  removing 513254 relations and 355971 ideals in 157283 cliques
Sat Feb  2 04:13:15 2008  commencing in-memory singleton removal
Sat Feb  2 04:13:18 2008  begin with 41544985 relations and 22540970 unique ideals
Sat Feb  2 04:13:31 2008  reduce to 41541162 relations and 22181149 ideals in 4 passes
Sat Feb  2 04:13:31 2008  max relations containing the same ideal: 37
Sat Feb  2 04:13:43 2008  dataset has 22.4% excess relations
Sat Feb  2 04:14:27 2008  ignoring smallest 5649956 rational and 5649937 algebraic ideals
Sat Feb  2 04:14:27 2008  filtering rational ideals above 97943552
Sat Feb  2 04:14:27 2008  filtering algebraic ideals above 97943552
Sat Feb  2 04:14:27 2008  need 17082338 more relations than ideals
Sat Feb  2 04:14:27 2008  commencing singleton removal, final pass
Sat Feb  2 04:37:06 2008  memory use: 625.6 MB
Sat Feb  2 04:37:53 2008  commencing in-memory singleton removal
Sat Feb  2 04:37:57 2008  begin with 41541162 relations and 23670875 unique ideals
Sat Feb  2 04:38:18 2008  reduce to 41407975 relations and 23537597 ideals in 6 passes
Sat Feb  2 04:38:18 2008  max relations containing the same ideal: 37
Sat Feb  2 04:38:44 2008  removing 1201538 relations and 807518 ideals in 394020 cliques
Sat Feb  2 04:38:45 2008  commencing in-memory singleton removal
Sat Feb  2 04:38:49 2008  begin with 40206437 relations and 23537597 unique ideals
Sat Feb  2 04:39:06 2008  reduce to 40174424 relations and 22697365 ideals in 5 passes
Sat Feb  2 04:39:06 2008  max relations containing the same ideal: 37
Sat Feb  2 04:39:23 2008  removing 1172909 relations and 778889 ideals in 394020 cliques
Sat Feb  2 04:39:24 2008  commencing in-memory singleton removal
Sat Feb  2 04:39:27 2008  begin with 39001515 relations and 22697365 unique ideals
Sat Feb  2 04:39:43 2008  reduce to 38977233 relations and 21893800 ideals in 5 passes
Sat Feb  2 04:39:43 2008  max relations containing the same ideal: 36
Sat Feb  2 04:40:07 2008  dataset has 8.0% excess relations
Sat Feb  2 04:40:10 2008  relations with 0 large ideals: 2989715
Sat Feb  2 04:40:10 2008  relations with 1 large ideals: 11010656
Sat Feb  2 04:40:10 2008  relations with 2 large ideals: 14303313
Sat Feb  2 04:40:10 2008  relations with 3 large ideals: 8291833
Sat Feb  2 04:40:10 2008  relations with 4 large ideals: 2173298
Sat Feb  2 04:40:10 2008  relations with 5 large ideals: 208418
Sat Feb  2 04:40:10 2008  relations with 6 large ideals: 0
Sat Feb  2 04:40:10 2008  relations with 7+ large ideals: 0
Sat Feb  2 04:40:10 2008  commencing 2-way merge
Sat Feb  2 04:40:34 2008  reduce to 28878604 relation sets and 11795171 unique ideals
Sat Feb  2 04:40:34 2008  commencing full merge
Sat Feb  2 04:43:45 2008  memory use: 1177.2 MB
Sat Feb  2 04:43:45 2008  found 17171528 cycles, need 11399371
Sat Feb  2 04:43:54 2008  weight of 11399371 cycles is about 519435201 (45.57/cycle)
Sat Feb  2 04:43:54 2008  distribution of cycle lengths:
Sat Feb  2 04:43:54 2008  1 relations: 3006558
Sat Feb  2 04:43:54 2008  2 relations: 1935993
Sat Feb  2 04:43:54 2008  3 relations: 1751058
Sat Feb  2 04:43:54 2008  4 relations: 1591526
Sat Feb  2 04:43:54 2008  5 relations: 1423816
Sat Feb  2 04:43:54 2008  6 relations: 1156684
Sat Feb  2 04:43:54 2008  7 relations: 533590
Sat Feb  2 04:43:54 2008  8 relations: 146
Sat Feb  2 04:43:54 2008  9 relations: 0
Sat Feb  2 04:43:54 2008  10+ relations: 0
Sat Feb  2 04:43:54 2008  heaviest cycle: 8 relations
Sat Feb  2 04:43:56 2008  matrix not dense enough, retrying
Sat Feb  2 04:43:58 2008  dataset has 8.0% excess relations
Sat Feb  2 04:44:42 2008  ignoring smallest 4886474 rational and 4886641 algebraic ideals
Sat Feb  2 04:44:42 2008  filtering rational ideals above 83951616
Sat Feb  2 04:44:42 2008  filtering algebraic ideals above 83951616
Sat Feb  2 04:44:42 2008  need 17082338 more relations than ideals
Sat Feb  2 04:44:42 2008  commencing singleton removal, final pass
Sat Feb  2 05:06:09 2008  memory use: 616.0 MB
Sat Feb  2 05:06:54 2008  commencing in-memory singleton removal
Sat Feb  2 05:06:58 2008  begin with 38977233 relations and 23401261 unique ideals
Sat Feb  2 05:07:16 2008  reduce to 38864938 relations and 23288900 ideals in 5 passes
Sat Feb  2 05:07:16 2008  max relations containing the same ideal: 36
Sat Feb  2 05:07:26 2008  dataset has -1.5% excess relations
Sat Feb  2 05:07:29 2008  relations with 0 large ideals: 2068760
Sat Feb  2 05:07:29 2008  relations with 1 large ideals: 8814134
Sat Feb  2 05:07:29 2008  relations with 2 large ideals: 13888356
Sat Feb  2 05:07:29 2008  relations with 3 large ideals: 10189875
Sat Feb  2 05:07:29 2008  relations with 4 large ideals: 3462233
Sat Feb  2 05:07:29 2008  relations with 5 large ideals: 441580
Sat Feb  2 05:07:29 2008  relations with 6 large ideals: 0
Sat Feb  2 05:07:29 2008  relations with 7+ large ideals: 0
Sat Feb  2 05:07:29 2008  commencing 2-way merge
Sat Feb  2 05:07:55 2008  reduce to 28634130 relation sets and 13058092 unique ideals
Sat Feb  2 05:07:55 2008  commencing full merge
Sat Feb  2 05:11:50 2008  memory use: 1215.2 MB
Sat Feb  2 05:11:50 2008  found 16223504 cycles, need 10430315
Sat Feb  2 05:11:59 2008  weight of 10430315 cycles is about 514662887 (49.34/cycle)
Sat Feb  2 05:11:59 2008  distribution of cycle lengths:
Sat Feb  2 05:11:59 2008  1 relations: 2237118
Sat Feb  2 05:11:59 2008  2 relations: 1676315
Sat Feb  2 05:11:59 2008  3 relations: 1623758
Sat Feb  2 05:11:59 2008  4 relations: 1535425
Sat Feb  2 05:11:59 2008  5 relations: 1425221
Sat Feb  2 05:11:59 2008  6 relations: 1212911
Sat Feb  2 05:11:59 2008  7 relations: 718760
Sat Feb  2 05:11:59 2008  8 relations: 807
Sat Feb  2 05:11:59 2008  9 relations: 0
Sat Feb  2 05:11:59 2008  10+ relations: 0
Sat Feb  2 05:11:59 2008  heaviest cycle: 8 relations
Sat Feb  2 05:12:01 2008  matrix not dense enough, retrying
Sat Feb  2 05:12:03 2008  dataset has -1.5% excess relations
Sat Feb  2 05:12:41 2008  ignoring smallest 4115855 rational and 4116628 algebraic ideals
Sat Feb  2 05:12:41 2008  filtering rational ideals above 69959680
Sat Feb  2 05:12:41 2008  filtering algebraic ideals above 69959680
Sat Feb  2 05:12:41 2008  need 17082338 more relations than ideals
Sat Feb  2 05:12:41 2008  commencing singleton removal, final pass
Sat Feb  2 05:34:34 2008  memory use: 653.6 MB
Sat Feb  2 05:35:16 2008  commencing in-memory singleton removal
Sat Feb  2 05:35:20 2008  begin with 38864938 relations and 24818311 unique ideals
Sat Feb  2 05:35:40 2008  reduce to 38792132 relations and 24745471 ideals in 5 passes
Sat Feb  2 05:35:40 2008  max relations containing the same ideal: 36
Sat Feb  2 05:35:51 2008  dataset has -11.2% excess relations
Sat Feb  2 05:35:55 2008  relations with 0 large ideals: 1317773
Sat Feb  2 05:35:55 2008  relations with 1 large ideals: 6623437
Sat Feb  2 05:35:55 2008  relations with 2 large ideals: 12797772
Sat Feb  2 05:35:55 2008  relations with 3 large ideals: 11924983
Sat Feb  2 05:35:55 2008  relations with 4 large ideals: 5239424
Sat Feb  2 05:35:55 2008  relations with 5 large ideals: 888743
Sat Feb  2 05:35:55 2008  relations with 6 large ideals: 0
Sat Feb  2 05:35:55 2008  relations with 7+ large ideals: 0
Sat Feb  2 05:35:55 2008  commencing 2-way merge
Sat Feb  2 05:36:23 2008  reduce to 28461160 relation sets and 14414499 unique ideals
Sat Feb  2 05:36:23 2008  commencing full merge
Sat Feb  2 05:41:13 2008  memory use: 1282.0 MB
Sat Feb  2 05:41:13 2008  found 15255948 cycles, need 9469183
Sat Feb  2 05:41:21 2008  weight of 9469183 cycles is about 528705802 (55.83/cycle)
Sat Feb  2 05:41:21 2008  distribution of cycle lengths:
Sat Feb  2 05:41:21 2008  1 relations: 1549848
Sat Feb  2 05:41:21 2008  2 relations: 1365605
Sat Feb  2 05:41:21 2008  3 relations: 1387149
Sat Feb  2 05:41:21 2008  4 relations: 1354489
Sat Feb  2 05:41:21 2008  5 relations: 1302076
Sat Feb  2 05:41:21 2008  6 relations: 1185563
Sat Feb  2 05:41:21 2008  7 relations: 987298
Sat Feb  2 05:41:21 2008  8 relations: 337088
Sat Feb  2 05:41:21 2008  9 relations: 67
Sat Feb  2 05:41:21 2008  10+ relations: 0
Sat Feb  2 05:41:21 2008  heaviest cycle: 9 relations
Sat Feb  2 05:41:23 2008  matrix not dense enough, retrying
Sat Feb  2 05:41:25 2008  dataset has -11.2% excess relations
Sat Feb  2 05:41:56 2008  ignoring smallest 3336478 rational and 3337166 algebraic ideals
Sat Feb  2 05:41:56 2008  filtering rational ideals above 55967743
Sat Feb  2 05:41:56 2008  filtering algebraic ideals above 55967743
Sat Feb  2 05:41:56 2008  need 17082338 more relations than ideals
Sat Feb  2 05:41:56 2008  commencing singleton removal, final pass
Sat Feb  2 06:04:43 2008  memory use: 697.9 MB
Sat Feb  2 06:05:26 2008  commencing in-memory singleton removal
Sat Feb  2 06:05:30 2008  begin with 38792132 relations and 26299310 unique ideals
Sat Feb  2 06:05:52 2008  reduce to 38752473 relations and 26259644 ideals in 5 passes
Sat Feb  2 06:05:52 2008  max relations containing the same ideal: 36
Sat Feb  2 06:06:03 2008  dataset has -21.0% excess relations
Sat Feb  2 06:06:07 2008  relations with 0 large ideals: 737715
Sat Feb  2 06:06:07 2008  relations with 1 large ideals: 4516923
Sat Feb  2 06:06:07 2008  relations with 2 large ideals: 10918249
Sat Feb  2 06:06:07 2008  relations with 3 large ideals: 13190671
Sat Feb  2 06:06:07 2008  relations with 4 large ideals: 7630168
Sat Feb  2 06:06:07 2008  relations with 5 large ideals: 1758747
Sat Feb  2 06:06:07 2008  relations with 6 large ideals: 0
Sat Feb  2 06:06:07 2008  relations with 7+ large ideals: 0
Sat Feb  2 06:06:07 2008  commencing 2-way merge
Sat Feb  2 06:06:38 2008  reduce to 28357840 relation sets and 15865011 unique ideals
Sat Feb  2 06:06:38 2008  commencing full merge
Sat Feb  2 06:12:49 2008  memory use: 1380.3 MB
Sat Feb  2 06:12:50 2008  found 14308470 cycles, need 8575211
Sat Feb  2 06:12:57 2008  weight of 8575211 cycles is about 557692281 (65.04/cycle)
Sat Feb  2 06:12:57 2008  distribution of cycle lengths:
Sat Feb  2 06:12:57 2008  1 relations: 980082
Sat Feb  2 06:12:57 2008  2 relations: 1043152
Sat Feb  2 06:12:57 2008  3 relations: 1124089
Sat Feb  2 06:12:57 2008  4 relations: 1136680
Sat Feb  2 06:12:57 2008  5 relations: 1122920
Sat Feb  2 06:12:57 2008  6 relations: 1072071
Sat Feb  2 06:12:57 2008  7 relations: 982123
Sat Feb  2 06:12:57 2008  8 relations: 815648
Sat Feb  2 06:12:57 2008  9 relations: 298298
Sat Feb  2 06:12:57 2008  10+ relations: 148
Sat Feb  2 06:12:57 2008  heaviest cycle: 10 relations
Sat Feb  2 06:13:00 2008  commencing cycle optimization
Sat Feb  2 06:13:17 2008  start with 39118606 relations
Sat Feb  2 06:14:22 2008  pruned 1215512 relations
Sat Feb  2 06:14:22 2008  memory use: 1338.0 MB
Sat Feb  2 06:14:22 2008  distribution of cycle lengths:
Sat Feb  2 06:14:22 2008  1 relations: 980082
Sat Feb  2 06:14:22 2008  2 relations: 1084860
Sat Feb  2 06:14:22 2008  3 relations: 1197476
Sat Feb  2 06:14:22 2008  4 relations: 1201166
Sat Feb  2 06:14:22 2008  5 relations: 1192699
Sat Feb  2 06:14:22 2008  6 relations: 1105461
Sat Feb  2 06:14:22 2008  7 relations: 950065
Sat Feb  2 06:14:22 2008  8 relations: 661228
Sat Feb  2 06:14:22 2008  9 relations: 202080
Sat Feb  2 06:14:22 2008  10+ relations: 94
Sat Feb  2 06:14:22 2008  heaviest cycle: 10 relations
Sat Feb  2 06:14:40 2008  
Sat Feb  2 06:14:40 2008  commencing linear algebra
Sat Feb  2 06:14:59 2008  read 8575211 cycles
Sat Feb  2 06:15:57 2008  cycles contain 22136943 unique relations
Sat Feb  2 06:44:07 2008  read 22136943 relations
Sat Feb  2 06:44:54 2008  using 32 quadratic characters above 2147481360
Sat Feb  2 06:48:47 2008  building initial matrix
Sat Feb  2 06:58:40 2008  memory use: 3033.7 MB
Sat Feb  2 06:59:05 2008  read 8575211 cycles
Sat Feb  2 06:59:28 2008  matrix is 8547933 x 8575211 (2409.0 MB) with weight 805680220 (93.95/col)
Sat Feb  2 06:59:28 2008  sparse part has weight 528595381 (61.64/col)
Sat Feb  2 07:03:53 2008  filtering completed in 3 passes
Sat Feb  2 07:03:56 2008  matrix is 8340443 x 8340634 (2359.4 MB) with weight 788048578 (94.48/col)
Sat Feb  2 07:03:57 2008  sparse part has weight 518418993 (62.16/col)
Sat Feb  2 07:06:26 2008  read 8340634 cycles
Sat Feb  2 07:06:48 2008  matrix is 8340443 x 8340634 (2359.4 MB) with weight 788048578 (94.48/col)
Sat Feb  2 07:06:48 2008  sparse part has weight 518418993 (62.16/col)
Sat Feb  2 07:06:48 2008  saving the first 48 matrix rows for later
Sat Feb  2 07:06:52 2008  matrix is 8340395 x 8340634 (2285.7 MB) with weight 605146918 (72.55/col)
Sat Feb  2 07:06:52 2008  sparse part has weight 515782599 (61.84/col)
Sat Feb  2 07:06:52 2008  matrix includes 64 packed rows
Sat Feb  2 07:06:52 2008  using block size 65536 for processor cache size 4096 kB
Sat Feb  2 07:07:53 2008  commencing Lanczos iteration (4 threads)
Sat Feb  2 07:07:53 2008  memory use: 2402.8 MB
Mon Feb 11 08:37:32 2008  lanczos halted after 131898 iterations (dim = 8340313)
Mon Feb 11 08:37:59 2008  lanczos error: only trivial dependencies found
Mon Feb 11 08:37:59 2008  
Mon Feb 11 08:37:59 2008  commencing square root phase
Mon Feb 11 08:37:59 2008  reading relations for dependency 1

Andi47 · 2008-02-17, 10:51

Have you already restarted 6,383+? I guess this one is the oversieved one - did you try to restart with less relations?

jasonp · 2008-02-17, 14:51

[I just got back from a week-long vacation. Note to those about to take a cruise: the world will sway back and forth for several days afterwards]

The initial filtering bound is chosen as the point where the estimated number of occurrences of a given large ideal drops below a limit (this assumes you're using the patch to v1.33 that was previously posted in this thread). The choice of bound occurs before singleton removal starts, to avoid needing yet another pass through the entire dataset. This means the data is quite noisy, since it includes contributions from relations that will later be pruned, and the initial estimate is usually too conservative. Subsequent removal passes lower the bound by 10% at a time until the amount of excess (relations minus unique ideals) drops below 8% of the number of ideals below the current bound. There is always at least one subsequent pass.

Once the above has happened, the merge phase runs and the density of the resulting matrix is tested. If it's too low, the bound continues to be reduced by 10% at a time. Many of the logs that have been posted here show that the latter step is necessary. When there is a huge amount of oversieving, it looks like additional strictness is needed. I don't want to resort to a fixed-size filtering bound, since a bound that's too high will not do anything and a bound that's too low will consume excessive memory (some of these jobs use gigs of memory with a filtering bound in the tens of millions).

Tom, I also think the threading changes are suspicious. The histograms output during filtering on the C165 definitely indicate the final filtering bound should have been lower.

Wacky · 2008-02-20, 04:20

Quote:

Originally Posted by jasonp

Tom, I also think the threading changes are suspicious.

Jason,

I'm running 1.33 as fetched from your website with "-t 4".
I had to make a few changes to get it to avoid crashing on memory allocation on my Mac running OSX 10.5.

Should I obtain a different version rather than spend another 500 hours on this LA and hope that it works?

I was also wondering if I can stop at a checkpoint and restart with a different "num_threads" ?

jasonp · 2008-02-20, 13:38

Quote:

Originally Posted by Wacky

Jason,

I'm running 1.33 as fetched from your website with "-t 4".
I had to make a few changes to get it to avoid crashing on memory allocation on my Mac running OSX 10.5.

Should I obtain a different version rather than spend another 500 hours on this LA and hope that it works?

I was also wondering if I can stop at a checkpoint and restart with a different "num_threads" ?

It is safe to stop and restart with a different number of threads; the on-disk format does not care about how the matrix is partitioned before running the linear algebra.

I'm not sure what you mean by 'obtain a different version'.

My experience is that the time needed for a matrix multiply, on the same machine, varies approximately linearly with the number of nonzeros in the sparse part of the matrix. I agree that bus speed is likely the limiting factor, since the multithreaded speedup on dual-CPU machines started off much higher than the speedup on dual-core machines (recent patches to the threading code have improved that).

I've also wondered about finding more than 64 nullspace vectors, at least for really big problems. The source changes would be pretty invasive, and the memory use would jump noticeably, but it might be a good idea to try. If you want instead to push the matrix dimension down (and the density up), just increase the value of TARGET_DENSITY in gnfs/filter/merge.c; the main merge code will continue merging ideals until the resulting matrix has more than this many estimated average nonzeros per column.

Wacky · 2008-02-20, 15:16

Quote:

Originally Posted by jasonp

I'm not sure what you mean by 'obtain a different version'.

My concern relates to Tom's report of difficulty and your reply

Quote:

Tom, I also think the threading changes are suspicious.

I don't want to spend weeks only to find out that the results are worthless.
Greg has certainly been successful using version 1.32

2008-02-09, 20:33	#210
FactorEyes Oct 2006 vomit_frame_pointer 168₁₆ Posts	Can't you just avoid procrels entirely? If you're going to perform post-processing with msieve, then procrels is really only useful as a sequence of samples to see where you are with your sieving, right? Is there a way of performing this test on your raw relations, using some means other than procrels? Doesn't "msieve -v -nc1" tell you you're a few relations short of a matrix () if you show up without enough? Or is procrels much quicker as a meter of progress? If you have not yet used msieve for NFS post-processing, then you're in for a treat. Last fiddled with by FactorEyes on 2008-02-09 at 20:36

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
error when running msieve 1.53 with cuda	aein	Msieve	9	2019-02-25 14:09
Help need to running Msieve	appleseed	Msieve	12	2016-04-10 02:31
Problem in running msieve with CUDA	mohamed	Msieve	20	2013-08-01 08:27
CUDA_ERROR_LAUNCH_OUT_OF_RESOURCES when running msieve 1.5.0 with CUDA	ryanp	Msieve	3	2012-06-12 03:27
Trouble Running Msieve	Sab	Msieve	4	2009-07-07 06:19

2008-02-17, 10:51	#216
Andi47 Oct 2004 Austria 2·17·73 Posts	Have you already restarted 6,383+? I guess this one is the oversieved one - did you try to restart with less relations?

2008-02-17, 14:51	#217
jasonp Tribal Bullet Oct 2004 3,541 Posts	[I just got back from a week-long vacation. Note to those about to take a cruise: the world will sway back and forth for several days afterwards] The initial filtering bound is chosen as the point where the estimated number of occurrences of a given large ideal drops below a limit (this assumes you're using the patch to v1.33 that was previously posted in this thread). The choice of bound occurs before singleton removal starts, to avoid needing yet another pass through the entire dataset. This means the data is quite noisy, since it includes contributions from relations that will later be pruned, and the initial estimate is usually too conservative. Subsequent removal passes lower the bound by 10% at a time until the amount of excess (relations minus unique ideals) drops below 8% of the number of ideals below the current bound. There is always at least one subsequent pass. Once the above has happened, the merge phase runs and the density of the resulting matrix is tested. If it's too low, the bound continues to be reduced by 10% at a time. Many of the logs that have been posted here show that the latter step is necessary. When there is a huge amount of oversieving, it looks like additional strictness is needed. I don't want to resort to a fixed-size filtering bound, since a bound that's too high will not do anything and a bound that's too low will consume excessive memory (some of these jobs use gigs of memory with a filtering bound in the tens of millions). Tom, I also think the threading changes are suspicious. The histograms output during filtering on the C165 definitely indicate the final filtering bound should have been lower.