Msieve v1.46 feedback

[tgrdy]

weight xxx/col print format error?
both in intel(march=nocona) and amd(mach=k8) , Linux x86 32 bit version.

Here it is the log, thanks,

Tue Aug 24 17:33:40 2010  
Tue Aug 24 17:33:40 2010  
Tue Aug 24 17:33:40 2010  Msieve v. 1.47
Tue Aug 24 17:33:40 2010  random seeds: 9ae80f89 77f63583
Tue Aug 24 17:33:40 2010  factoring  ...(157 digits)
...
Tue Aug 24 17:33:42 2010  skew 468676.75, size 2.585e-15, alpha -7.130, combined = 1.913e-12 rroots = 3
Tue Aug 24 17:33:42 2010  
Tue Aug 24 17:33:42 2010  commencing linear algebra
Tue Aug 24 17:38:38 2010  matrix starts at (0, 0)
Tue Aug 24 17:38:40 2010  matrix is 4895033 x 4895210 (1408.6 MB) with weight 470679799 (96.15/col)
Tue Aug 24 17:38:40 2010  sparse part has weight 330087858 (-1388557958659491449960930129592021408789550482319091262508786343129246955111700424862787405372458865398879990433798520910008535446241616403655082672363712028928001949282183195684168225812590430303018571883167524212052086660772268101647676122274677135852798941046701168373832920230896419954870822644796620800.00/col)
Tue Aug 24 17:38:40 2010  saving the first 48 matrix rows for later
Tue Aug 24 17:38:42 2010  matrix includes 64 packed rows
Tue Aug 24 17:38:43 2010  matrix is 4894985 x 4895210 (1363.0 MB) with weight 375386110 (76.68/col)
Tue Aug 24 17:38:43 2010  sparse part has weight 327941077 (-488438086314922565818526936846739985716284058195083095903320746364690187325138788817242597438608443259924512768.00/col)
Tue Aug 24 17:38:43 2010  using block size 65536 for processor cache size 6144 kB
Tue Aug 24 17:39:12 2010  commencing Lanczos iteration (7 threads)
Tue Aug 24 17:39:12 2010  memory use: 1374.4 MB
Tue Aug 24 17:39:52 2010  restarting at iteration 46496 (dim = 2940041)
Tue Aug 24 17:40:34 2010  linear algebra at 60.1%, ETA 14h38m
Tue Aug 24 17:40:47 2010  checkpointing every 133356 dimensions
Tue Aug 24 18:02:54 2010  error: corrupt state, please restart from checkpoint

Tue Aug 24 18:12:01 2010  
Tue Aug 24 18:12:01 2010  
Tue Aug 24 18:12:01 2010  Msieve v. 1.47
Tue Aug 24 18:12:01 2010  random seeds: 710bb90c c434c400
Tue Aug 24 18:12:01 2010  factoring ...... (157 digits)
Tue Aug 24 18:12:02 2010  ......
Tue Aug 24 18:12:02 2010  commencing linear algebra
Tue Aug 24 18:12:23 2010  matrix starts at (0, 0)
Tue Aug 24 18:12:25 2010  matrix is 4895033 x 4895210 (1408.6 MB) with weight 470679799 (96.15/col)
Tue Aug 24 18:12:25 2010  sparse part has weight 330087858 (-1388557958659491449960930129592021408789550482319091262508786343129246955111700424862787405372458865398879990433798520910008535446241616403655082672363712028928001949282183195684168225812590430303018571883167524212052086660772268101647676122274677135852798941046701168373832920230896419954870822644796620800.00/col)
Tue Aug 24 18:12:25 2010  saving the first 48 matrix rows for later
Tue Aug 24 18:12:27 2010  matrix includes 64 packed rows
Tue Aug 24 18:12:29 2010  matrix is 4894985 x 4895210 (1363.0 MB) with weight 375386110 (76.68/col)
Tue Aug 24 18:12:29 2010  sparse part has weight 327941077 (-488438086314922565818526936846739985716284058195083095903320746364690187325138788817242597438608443259924512768.00/col)
Tue Aug 24 18:12:29 2010  using block size 43690 for processor cache size 1024 kB
Tue Aug 24 18:13:01 2010  commencing Lanczos iteration (8 threads)
Tue Aug 24 18:13:01 2010  memory use: 1412.4 MB
Tue Aug 24 18:13:03 2010  restarting at iteration 46656 (dim = 2950159)

but v1.45 windows x86 32 bit version works well:

Mon Aug 23 15:48:06 2010  
Mon Aug 23 15:48:06 2010  
Mon Aug 23 15:48:06 2010  Msieve v. 1.45
Mon Aug 23 15:48:06 2010  random seeds: d9eb38b8 d2609e66
Mon Aug 23 15:48:06 2010  factoring ...... (157 digits)
Mon Aug 23 15:48:08 2010  searching for 15-digit factors
Mon Aug 23 15:48:09 2010  commencing number field sieve (157-digit input)
Mon Aug 23 15:48:09 2010  R0: -718286775230264074412218462160
Mon Aug 23 15:48:09 2010  R1:  330882384079102889
Mon Aug 23 15:48:09 2010  A0: -574785673446953991103337093662087263
Mon Aug 23 15:48:09 2010  A1: -2849845913901309456445653108969
Mon Aug 23 15:48:09 2010  A2:  19403845861276275944787296
Mon Aug 23 15:48:09 2010  A3:  40609767955771924744
Mon Aug 23 15:48:09 2010  A4:  185982982727102
Mon Aug 23 15:48:09 2010  A5:  33078600
Mon Aug 23 15:48:09 2010  skew 468676.75, size 2.585210e-015, alpha -7.129592, combined = 1.913363e-012
Mon Aug 23 15:48:09 2010  
Mon Aug 23 15:48:09 2010  commencing linear algebra
Mon Aug 23 15:48:12 2010  read 4895210 cycles
Mon Aug 23 15:48:20 2010  matrix is 4895033 x 4895210 (1408.6 MB) with weight 470679799 (96.15/col)
Mon Aug 23 15:48:20 2010  sparse part has weight 330087858 (67.43/col)
Mon Aug 23 15:48:21 2010  saving the first 48 matrix rows for later
Mon Aug 23 15:48:24 2010  matrix is 4894985 x 4895210 (1363.0 MB) with weight 375386110 (76.68/col)
Mon Aug 23 15:48:24 2010  sparse part has weight 327941077 (66.99/col)
Mon Aug 23 15:48:24 2010  matrix includes 64 packed rows
Mon Aug 23 15:48:24 2010  using block size 65536 for processor cache size 8192 kB
Mon Aug 23 15:49:09 2010  commencing Lanczos iteration (8 threads)
Mon Aug 23 15:49:09 2010  memory use: 1677.9 MB
Mon Aug 23 15:49:11 2010  restarting at iteration 169 (dim = 10691)
Mon Aug 23 15:49:37 2010  linear algebra at 0.2%, ETA 46h35m
Wed Aug 25 08:09:27 2010  lanczos halted after 77412 iterations (dim = 4894985)
Wed Aug 25 08:09:50 2010  recovered 31 nontrivial dependencies
Wed Aug 25 08:10:04 2010  BLanczosTime: 145315
Wed Aug 25 08:10:04 2010  elapsed time 40:21:58
Wed Aug 25 08:10:05 2010  
Wed Aug 25 08:10:05 2010  
Wed Aug 25 08:10:05 2010  Msieve v. 1.45
Wed Aug 25 08:10:05 2010  random seeds: f3029690 d5850aba
Wed Aug 25 08:10:05 2010  factoring ...... (157 digits)
Wed Aug 25 08:10:07 2010  searching for 15-digit factors
Wed Aug 25 08:10:08 2010  commencing number field sieve (157-digit input)
Wed Aug 25 08:10:08 2010  ......
Wed Aug 25 08:10:08 2010  commencing square root phase
Wed Aug 25 08:10:08 2010  reading relations for dependency 10
Wed Aug 25 08:10:10 2010  read 2446011 cycles
Wed Aug 25 08:10:17 2010  cycles contain 6625862 unique relations
Wed Aug 25 08:12:30 2010  read 6625862 relations
Wed Aug 25 08:13:36 2010  multiplying 6625862 relations
Wed Aug 25 09:03:44 2010  multiply complete, coefficients have about 405.83 million bits
Wed Aug 25 09:03:57 2010  initial square root is modulo 19138993
......

[jasonp]

Does it work any better if you change '%5.2f' to '%5.2lf' , i.e. add an L to the floating point format code, in common/lanczos/lanczos_pre.c lines 75 and 80?

[tgrdy]

I'll test it this Friday.
Sorry but be busy on Thursday.
Thanks,

Originally Posted by jasonp

Does it work any better if you change '%5.2f' to '%5.2lf' , i.e. add an L to the floating point format code, in common/lanczos/lanczos_pre.c lines 75 and 80?

[tgrdy]

I am doing relation filtering on a C163, but there are
too many error -11 in log file, the log file has more the 100,000,000 bytes (nearly 100MB). My last C157 done, and it only created about 30KB log file.
Why there are so many (error -11)?

All the relations from 12143120 to 13778734 are printed error -11. That is 1635615 lines.

Does it matter if there are 1 million of error -11 in 100 millions of relations?
I have check the C163.dat, it has no messy dat or non-ascii data.
Thanks,

The error -11 from:
/gnfs/relation.c
line 198, func: int32 nfs_read_relation

if (!mp_is_one(&polyval.num))
return -11;

08/18/2010 20:48 165 c163.ini
08/27/2010 14:11 485 step345.bat
08/27/2010 13:50 490 c163.fb
08/20/2010 16:13 902,144 msieve.exe
08/27/2010 14:48 6,587,248 c163.dat.br
08/27/2010 14:48 78,260,920 c163.dat.hc
08/27/2010 14:48 103,747,209 c163_step345.log
08/27/2010 13:22 17,960,780,775 c163.dat

Fri Aug 27 14:11:27 2010 commencing number field sieve (163-digit input)
......
Fri Aug 27 14:11:27 2010 A0: 360487238294144721913628110556205285
Fri Aug 27 14:11:27 2010 A1: 1132654025229352805189159662989
Fri Aug 27 14:11:27 2010 A2: -214331411232140294532642735
Fri Aug 27 14:11:27 2010 A3: 1188242218322082959447
Fri Aug 27 14:11:27 2010 A4: 3601468549877046
Fri Aug 27 14:11:27 2010 A5: 152680320
Fri Aug 27 14:11:27 2010 skew 244861.34, size 6.185862e-016, alpha -7.003328, combined = 8.120503e-013
Fri Aug 27 14:11:27 2010
Fri Aug 27 14:11:27 2010 commencing relation filtering
Fri Aug 27 14:11:27 2010 estimated available RAM is 4096.0 MB
Fri Aug 27 14:11:27 2010 commencing duplicate removal, pass 1
Fri Aug 27 14:11:32 2010 error -15 reading relation 317645
Fri Aug 27 14:11:36 2010 error -15 reading relation 630571
Fri Aug 27 14:11:41 2010 error -15 reading relation 940806
Fri Aug 27 14:11:46 2010 error -1 reading relation 1248485
Fri Aug 27 14:14:31 2010 error -15 reading relation 12143118
Fri Aug 27 14:14:31 2010 error -11 reading relation 12143119
Fri Aug 27 14:14:31 2010 error -11 reading relation 12143120
...... 1635613 lines error -11
Fri Aug 27 14:18:50 2010 error -11 reading relation 13778734

Fri Aug 27 14:18:51 2010 error -1 reading relation 13832363
......
Fri Aug 27 14:38:53 2010 error -1 reading relation 91168233
Fri Aug 27 14:38:53 2010 error -11 reading relation 91168564
Fri Aug 27 14:39:26 2010 error -5 reading relation 93245957
Fri Aug 27 14:46:01 2010 error -15 reading relation 120741558
Fri Aug 27 14:46:22 2010 error -15 reading relation 122241478
Fri Aug 27 14:48:46 2010 found 19565230 hash collisions in 130489264 relations

[Batalov]

these relations are from another project (or another polynomial for the same number).

[frmky]

Originally Posted by tgrdy

Does it matter if there are 1 million of error -11 in 100 millions of relations?

And no, it does not matter. You can safely ignore these errors.

[Andi47]

Originally Posted by frmky

And no, it does not matter. You can safely ignore these errors.

The only problems are an unnecessary big logfile and much time wasted for printing thousands of error -11 lines to screen and logfile.

If the logfile (100+ MB ) doesn't eat up all of your disk space, your factorization should savely continue.

[jrk]

Originally Posted by tgrdy

All the relations from 12143120 to 13778734 are printed error -11. That is 1635615 lines.

Does it matter if there are 1 million of error -11 in 100 millions of relations?
I have check the C163.dat, it has no messy dat or non-ascii data.
Thanks,

I agree with Batalov that those relations are probably from the wrong polynomial.

Maybe it would be nicer to limit the maximum number of read error reports, and afterward printing something like "too many read errors, further errors will not be reported." Since if there are more than a million such errors, something is likely to be systematically wrong.

[tgrdy]

Thanks,

I am sieveing the C163 on 20+ Linux PCs.
More than 1M relations are wrong, because one PC takes a wrong poly for another number.
I 'll delete the bad part, and sieve more again.

Originally Posted by Batalov

these relations are from another project (or another polynomial for the same number).

[jasonp]

Originally Posted by tgrdy

Tue Aug 24 18:12:25 2010 sparse part has weight 330087858 (-13885579586594914499609301295920214087895504823190912...44796620800.00/col)

This should be fixed in SVN now.

[frmky]

This is a test run and not the final matrix that I'll be using for 5,409-, but I thought you might enjoy seeing msieve filter nearly 720M unique relations down to a 28M matrix.