How about merging YAFU and Msieve?

[jasonp]

Bsquared and I have been discussing the merits of combining our respective factoring codebases into a single package. Both of us have a lot to gain from the merger: Msieve has really good NFS code and highly parallel linear algebra, and YAFU has fantastic versions of everything else. The SIQS code there can expand to use three large primes using the filtering code in Msieve, and both packages can benefit from using GMP instead of the other libraries they grew up using. Maybe someday we could add an NFS lattice sieve too.

We think the best idea is to make another sourceforge project and switch everything over once the new codebase is stable enough. The problem is I tried to do this before, by merging Msieve's NFS postprocessing into GGNFS, but it became too much work to backport changes to the GGNFS version and I wound up giving up maintaining the two codebases. The same thing is going to happen here, because this code is extremely complex and both packages will require major changes to work together, which means that eventually development on YAFU and Msieve would stop.

Since several of you are involved in one or both of these projects, we could use a little consensus. Would you trade separate YAFU and Msieve packages for a single replacement that tries to do the best of both? And if so, what should we name it?

[R.D. Silverman]

Originally Posted by jasonp

Bsquared and I have been discussing the merits of combining our respective factoring codebases into a single package. Both of us have a lot to gain from the merger: Msieve has really good NFS code and highly parallel linear algebra, and YAFU has fantastic versions of everything else. The SIQS code there can expand to use three large primes using the filtering code in Msieve, and both packages can benefit from using GMP instead of the other libraries they grew up using. Maybe someday we could add an NFS lattice sieve too.

We think the best idea is to make another sourceforge project and switch everything over once the new codebase is stable enough. The problem is I tried to do this before, by merging Msieve's NFS postprocessing into GGNFS, but it became too much work to backport changes to the GGNFS version and I wound up giving up maintaining the two codebases. The same thing is going to happen here, because this code is extremely complex and both packages will require major changes to work together.

So would you trade separate YAFU and Msieve packages for a single replacement that tries to do the best of both? And if so, what should we name it?

I'd be happy to provide my lattice siever, if you want it.

[Batalov]

"The Mfu is strong in this one."

[jasonp]

The source for it that I have probably is very old, go ahead and email me your latest.

Edit: latest lattice sieve, not latest Mfu :)

[bsquared]

Originally Posted by jasonp

The source for it that I have probably is very old, go ahead and email me your latest.

Edit: latest lattice sieve, not latest Mfu :)

My latest is a year and a half old... could you send it to me as well?

[Andi47]

Originally Posted by jasonp

The same thing is going to happen here, because this code is extremely complex and both packages will require major changes to work together, which means that eventually development on YAFU and Msieve would stop.

This sentence is not entirely clear to me: Do you mean that the development might stop if yafu and msieve will be merged, or it might stop when they are kept as seperate programs?

[frmky]

I actually prefer multiple smaller programs, but I can deal with it. Jason certainly has a knack for combining nontrivial specialized code (MPI, CUDA) in a single codebase!

[jasonp]

Originally Posted by Andi47

This sentence is not entirely clear to me: Do you mean that the development might stop if yafu and msieve will be merged, or it might stop when they are kept as seperate programs?

The former. No matter how similar you try to make two versions of the same codebase, it's inevitable that there will be something in one that will be very different in the other, and making improvements in one version available for the other one will inevitably become too much work. So I'll get lazy, and changes will accumulate, and before long one version will be out of date. The only exception is when both codebases are exactly the same, so why did you fork off a separate version in that case?

In the case of a merge, I want to do things like have a single directory of utilities and common code that we both can use, and if I get used to e.g. the fast tinyQS code currently in YAFU then I have to scramble when it's not available in the current Msieve.

[Wacky]

Originally Posted by jasonp

... the merits of combining our respective factoring codebases into a single package.

Do it!

One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them.


Yes, Bob, I know the definition of "Field" and "Ring". But "One Field" just doesn't have the right impact :(

PL1 anyone?

[Brian Gladman]

Originally Posted by jasonp

Bsquared and I have been discussing the merits of combining our respective factoring codebases into a single package. Both of us have a lot to gain from the merger: Msieve has really good NFS code and highly parallel linear algebra, and YAFU has fantastic versions of everything else. The SIQS code there can expand to use three large primes using the filtering code in Msieve, and both packages can benefit from using GMP instead of the other libraries they grew up using. Maybe someday we could add an NFS lattice sieve too.

We think the best idea is to make another sourceforge project and switch everything over once the new codebase is stable enough. The problem is I tried to do this before, by merging Msieve's NFS postprocessing into GGNFS, but it became too much work to backport changes to the GGNFS version and I wound up giving up maintaining the two codebases. The same thing is going to happen here, because this code is extremely complex and both packages will require major changes to work together, which means that eventually development on YAFU and Msieve would stop.

Since several of you are involved in one or both of these projects, we could use a little consensus. Would you trade separate YAFU and Msieve packages for a single replacement that tries to do the best of both? And if so, what should we name it?

In principle I would support this since it would allow the multiple precision code to be rationalised. Up to date lattice sieve code from Bob would also be an advantage.

But I would like to see a clear commitment in any combined project to support both Linux and Windows (including the MSVC and Intel compilers) and not to support the addition of major features on only one of these host systems (as I recall happened with thread support in parts of YAFU).

Given such a commitment I would be willing to maaintain the native Windows x64 aspects of such a project (i.e those using the MSVC and Intel compilers for Windows x64).

[Jeff Gilchrist]

It sounds like a plan to me, but I don't have any free time to contribute to coding right now. I would be happy to do testing though to help ensure a combined version is stable and as bug free as possible.