[QUOTE=akruppa;119665]The matrix is both larger and heavier than the one from 3,499+, but takes only 18.5 days to solve? I'll have to learn how to use msieve for the linear algebra! Is there a way to feed readily merged relations into msieve for matrix building and solving?
[/QUOTE]
You would have to convert your relations to GGNFS format and use msieve for all of the postprocessing. The latter technically isn't necessary, but using msieve for just the linear algebra and the CWI suite for everything else would require converting the CWI relation-set format to msieve's relation cycle format, then converting the generated dependencies back into the format the CWI suite wants. Not that that would be especially difficult, I just think it's easier to let msieve's filtering and square root run for half a day.

Greg has a reasonably fast relation converter.

[QUOTE=akruppa;119665]Is there a way to feed readily merged relations into msieve for matrix building and solving?
[/QUOTE]

I think that CWI's merged relations are still in the standard text relation format. If so, then it would be faster to convert only those merged relations to the GGNFS format. Then let msieve do a final filtering of those, which won't take long, do the linear algebra, which is much faster than the CWI tools, and the square roots, which are a bit slower than the CWI but take only a couple of hours each.

The CWI->GGNFS converter I have requires a GGNFS factorbase file and input files in GGNFS format. I've been thinking about removing these dependencies on GGNFS, but haven't gotten around to it. Basically, it just refactors the remaining norms (after the large primes are removed) using a little trial division, SQUFOF, and finally Pollard Rho if SQUFOF fails. I'll be happy to send you the source of the current program.

Greg

[QUOTE=frmky;119692]I think that CWI's merged relations are still in the standard text relation format. If so, then it would be faster to convert only those merged relations to the GGNFS format. Then let msieve do a final filtering of those, which won't take long, do the linear algebra, which is much faster than the CWI tools, and the square roots, which are a bit slower than the CWI but take only a couple of hours each.[/QUOTE]

The text format of CWI relations uses the final character in each relation to indicate the merged groups. ';' designates that the relation is the first of a merged set. ':' designates that the relation is a part of the previous set.

I have not understood why they designate the first relation in a merged set rather than the last since the indicator is at the end of the relation rather than at its beginning. That requires look ahead in the parser.

what happened to msieve v1.30

[QUOTE=henryzz;120660]what happened to msieve v1.30[/QUOTE]
The older source distribution on the web page is not necessarily one release number less than the newest one.

Hi Jason,

I've found what may be the last way to crash msieve with a malformed relation. In the latest factorization, one of the bad relations started with 0,0:. The simple fix was just after a and b are read in, I added a check to make sure that both a and b aren't 0 before continuing to process the relation.

Greg

[QUOTE=frmky;120802]Hi Jason,

I've found what may be the last way to crash msieve with a malformed relation. In the latest factorization, one of the bad relations started with 0,0:. The simple fix was just after a and b are read in, I added a check to make sure that both a and b aren't 0 before continuing to process the relation.
[/QUOTE]
It seems obvious in hindsight; fix applied, along with a contributed patch that changes multithreaded lanczos to use a thread pool instead of spawn-and-join. The latter dramatically speeds up multithreaded runs on multi-core CPUs.

jasonp

how much ecm does msieve do at each bound

[QUOTE=henryzz;120928]how much ecm does msieve do at each bound[/QUOTE]
The number of curves run is specified by the Readme in the ECM-6.1.3 sources as the smallest number of curves needed to find a factor at the given digit level with probability (1 - 1/e). ECM switches to the next digit level when this number of curves have run, and stops when a factor is found and the remaining composite cofactor is under ~65 digits.

15 digits: < 1 second 30 curves (run on 26+ digit cofactors)
20 digits: 20 seconds 74 curves (run on 67+ digit cofactors)
25 digits: 5 minutes 214 curves (run on 85+ digit cofactors)
30 digits: ~30 minutes 430 curves (run on 97+ digit cofactors)
35 digits: ~5 hours 904 curves (run on 109+ digit cofactors)
40 digits: ?? 2350 curves (run on 120+ digit cofactors)

Cofactor sizes assume trial division and pollard rho have already run.

The first level always happens, other levels require passing '-e' on the command line since they can take a big chunk of of the total time that isn't necessary if you know the input is immune to ECM. Each digit level is tried in turn, in order to minimize the average runtime.

I guess the code should print out the number of curves for people keeping track of total counts.

how many curves it has already done would be nice too

[QUOTE=frmky;119649]For your reading pleasure, here's the log of the filtering for the NFSNet factorization of 10,239-. The computer had only 3 GB of memory, but msieve wanted 3.6 GB to build the matrix. I just let it run, hard drive thrashing, so build_matrix_core took 16 hours rather than 16 minutes to complete. But it was successful, and the matrix fits comfortably in memory for the BL. The BL will take about 18.5 days to complete on this 2.4GHz single core Opteron.

[/QUOTE]

Here's the rest of the log for 10,239-. The filtering portion of the log is in an earlier post. The restart code was tested on this one. About 60% of the way through the BL, the computer went down. It restarted successfully. The sqrt again used less memory than build_matrix_core, so it currently seems that if there's enough memory to build the matrix, there's enough for sqrt.

Greg

[CODE]Fri Nov 30 15:43:10 2007 commencing Lanczos iteration
Wed Dec 12 18:08:29 2007
Wed Dec 12 18:08:29 2007
Wed Dec 12 18:08:29 2007 Msieve v. 1.30
Wed Dec 12 18:08:29 2007 random seeds: 2a56b2de 51c82775
Wed Dec 12 18:08:29 2007 factoring 162385812809900583261295597372983559948698484946321658540735656202018003505845104737134070357362269119992033891923896707742660065148483572244656969761256208962241824944638737217461295395400432531624931235045560966776497496089919 (228 digits)
Wed Dec 12 18:08:30 2007 commencing number field sieve (227-digit input)
Wed Dec 12 18:08:30 2007 R0: -10000000000000000000000000000000000000000
Wed Dec 12 18:08:30 2007 R1: 1
Wed Dec 12 18:08:30 2007 A0: -10
Wed Dec 12 18:08:30 2007 A1: 0
Wed Dec 12 18:08:30 2007 A2: 0
Wed Dec 12 18:08:30 2007 A3: 0
Wed Dec 12 18:08:30 2007 A4: 0
Wed Dec 12 18:08:30 2007 A5: 0
Wed Dec 12 18:08:30 2007 A6: 1
Wed Dec 12 18:08:30 2007 size score = 1.465344e-11, Murphy alpha = 1.209006, combined = 1.037340e-11
Wed Dec 12 18:08:30 2007
Wed Dec 12 18:08:30 2007 commencing linear algebra
Wed Dec 12 18:08:38 2007 read 8639035 cycles
Wed Dec 12 18:10:21 2007 matrix is 8638835 x 8639035 with weight 768104215 (avg 88.91/col)
Wed Dec 12 18:10:22 2007 saving the first 48 matrix rows for later
Wed Dec 12 18:10:34 2007 matrix is 8638787 x 8639035 with weight 581239234 (avg 67.28/col)
Wed Dec 12 18:10:34 2007 matrix includes 64 packed rows
Wed Dec 12 18:10:34 2007 using block size 43690 for processor cache size 1024 kB
Wed Dec 12 18:12:16 2007 commencing Lanczos iteration
Wed Dec 12 18:12:37 2007 restarting at iteration 83847 (dim = 5301785)
Thu Dec 20 14:14:00 2007 lanczos halted after 136617 iterations (dim = 8638773)
Thu Dec 20 14:14:35 2007 recovered 40 nontrivial dependencies
Thu Dec 20 14:14:35 2007
Thu Dec 20 14:14:35 2007 commencing square root phase
Thu Dec 20 14:14:35 2007 reading relations for dependency 1
Thu Dec 20 14:14:46 2007 read 4318353 cycles
Thu Dec 20 14:15:08 2007 cycles contain 14491832 unique relations
Thu Dec 20 14:22:03 2007 read 14491832 relations
Thu Dec 20 14:24:53 2007 multiplying 20304012 relations
Thu Dec 20 15:19:46 2007 multiply complete, coefficients have about 502.65 million bits
Thu Dec 20 15:20:01 2007 initial square root is modulo 1044982177
Thu Dec 20 17:23:23 2007 reading relations for dependency 2
Thu Dec 20 17:23:40 2007 read 4321725 cycles
Thu Dec 20 17:24:02 2007 cycles contain 14502072 unique relations
Thu Dec 20 17:30:54 2007 read 14502072 relations
Thu Dec 20 17:33:44 2007 multiplying 20327228 relations
Thu Dec 20 18:28:26 2007 multiply complete, coefficients have about 503.23 million bits
Thu Dec 20 18:28:41 2007 initial square root is modulo 1070239207
Thu Dec 20 20:32:01 2007 reading relations for dependency 3
Thu Dec 20 20:32:18 2007 read 4319090 cycles
Thu Dec 20 20:32:41 2007 cycles contain 14496063 unique relations
Thu Dec 20 20:39:33 2007 read 14496063 relations
Thu Dec 20 20:42:23 2007 multiplying 20311474 relations
Thu Dec 20 21:37:15 2007 multiply complete, coefficients have about 502.84 million bits
Thu Dec 20 21:37:31 2007 initial square root is modulo 1053046741
Thu Dec 20 23:41:13 2007 reading relations for dependency 4
Thu Dec 20 23:41:30 2007 read 4319485 cycles
Thu Dec 20 23:41:52 2007 cycles contain 14495543 unique relations
Thu Dec 20 23:48:45 2007 read 14495543 relations
Thu Dec 20 23:51:34 2007 multiplying 20311698 relations
Fri Dec 21 00:46:14 2007 multiply complete, coefficients have about 502.84 million bits
Fri Dec 21 00:46:30 2007 initial square root is modulo 1053240901
Fri Dec 21 02:58:16 2007 prp54 factor: 383155477843726029783939406113226468701730728790004161
Fri Dec 21 02:58:16 2007 prp81 factor: 128780300340244872385688233345188210841783983757299260103530718169486826135819357
Fri Dec 21 02:58:16 2007 prp94 factor: 3290967632861131703281828943635774383301940171982919699073443165222894023742681701403432993547
Fri Dec 21 02:58:16 2007 elapsed time 200:49:47
[/CODE]