The future of Msieve

[alpertron]

An easy change to msieve is to add statistics about the sieving in the log. For example, after factoring the number 25626911509016950901475527595812441191940963052854598209, my factoring applet shows:

Factorization complete in 0d 0h 1m 28s
ECM: 1930358 modular multiplications
Prime checking: 51023 modular multiplications
SIQS: 14681 polynomials sieved
      12859 sets of trial divisions
      1867 smooth congruences found (1 out of every 471805 values)
      10179 partial congruences found (1 out of every 86536 values)
      1353 useful partial congruences

Of course the timing is too high but I cannot control the cache using Java.

[jasonp]

Well, if we're going to consider QS as well, then I'd like to

- add 3LP support

- make the batch factoring, hashtable and NFS filtering code usable by QS. The filtering is divided into a front end that reads relations and a back end that optimizes an intermediate representation of relation sets, and the back end can work just as well for QS and it does for NFS. Indeed you have to have this stuff if you want 3LP

- change the relation format to be stateless; this has caused no end of problems

- clean up the sieving code and find out why gcc 4.x produces a slower binary (MinGW has gcc 4.x available as a beta release, and the slowdowns that some of you reported years ago are finally visible...I never saw them in linux). Much of the cleanup should involve forking separate versions of the sieve code so that different size jobs can avoid having to deal with a single incredibly complex code path that covers all cases


The big hurdle with revamping the QS side of things is (has always been) that nobody would seriously want them done, at least at the expense of the NFS side of things. I've started the changes at least twice before getting demoralized and giving up.

[henryzz]

Originally Posted by jasonp

- add 3LP support

how much would this speed up qs sieving

my dream for msieve would be a lattice siever but you have stated you dont want to do that
would a lattice sieve work for QS

[xilman]

Originally Posted by henryzz

would a lattice sieve work for QS

Would you explain what you mean by this please? I don't quite follow you.


Paul

[henryzz]

Originally Posted by xilman

Would you explain what you mean by this please? I don't quite follow you.


Paul

could u use a lattice sieve for QS

[jasonp]

Originally Posted by henryzz

could u use a lattice sieve for QS

The multiple polynomial quadratic sieve actually is a lattice sieve; it samples the (one) sieve interval of ordinary QS at only a few points that have known divisors.

To answer your other question, I don't know what kind of speedup 3LP would provide, or how large a job would have to be before getting completed faster with 3LP QS, but Paul's experience makes it likely to be a factor of two speedup for inputs of perhaps 100 digits and up. This only moves the crossover point between QS and NFS up a few digits, which is why it's not very interesting. I only consider it a good use of my time for aesthetic reasons

[R.D. Silverman]

Originally Posted by jasonp

The multiple polynomial quadratic sieve actually is a lattice sieve; it samples the (one) sieve interval of ordinary QS at only a few points that have known divisors.

To answer your other question, I don't know what kind of speedup 3LP would provide, or how large a job would have to be before getting completed faster with 3LP QS, but Paul's experience makes it likely to be a factor of two speedup for inputs of perhaps 100 digits and up. This only moves the crossover point between QS and NFS up a few digits, which is why it's not very interesting. I only consider it a good use of my time for aesthetic reasons

In terms of the time commitment needed to implement the change,
2LP --> 3LP doesn't require much new code.

[Batalov]

Jason probably means that he wants his code to remain both effective and elegant. Tough luck!
___________

"Fast, Good, Cheap - choose any two."

[xilman]

Originally Posted by R.D. Silverman

In terms of the time commitment needed to implement the change,
2LP --> 3LP doesn't require much new code.

The major part of the time, IME, was implementing the post-processing. Extending the siever was rather easy, though slightly fiddly in places.

Paul

[jasonp]

Alex Kruppa showed at CADO that keeping the sieving stage efficient in the presence of 3LP relations is actually very difficult, though that probably applies much more to NFS than QS due to the smaller average size of norms. I agree that if you just want to get something untuned up and running then the postprocessing is the biggest part of the work needed, especially if you pick that time to add long-needed improvements.

[XYYXF]

Originally Posted by frmky

The only two teams pushing msieve to its limits are Bruce and myself, and Tom's coordinated group here. As far as I am aware, the current version of msieve meets the needs of everyone else.

Agree.

I think Msieve is being used on such large jobs just because there's no good alternative for them. But other packages are being developed, so let's leave these awful 300+ SNFS and 200+ GNFS for CWI et.al., while Msieve will remain an outstanding tool for normal NFS and QS tasks :-)

Just IMHO.