Yafu

[Mr. Odd]

Thanks for that! Any chance you can get the parallel ecm option for Windows machines?

[bsquared]

Originally Posted by Mr. Odd

Thanks for that! Any chance you can get the parallel ecm option for Windows machines?

As of now, no. bchaffin used shared memory together with a function called fork() to get parallel ecm going. From what I understand, shared memory is possible to do in windows, but there just isn't a good (any?) way to emulate fork() on windows. The only alternative I can see is a completely different approach using external processes and parsing of log files, which is not an attractive option. So for the foreseeable future, it will remain linux only :(

[bchaffin]

Originally Posted by bsquared

As of now, no. bchaffin used shared memory together with a function called fork() to get parallel ecm going. From what I understand, shared memory is possible to do in windows, but there just isn't a good (any?) way to emulate fork() on windows. The only alternative I can see is a completely different approach using external processes and parsing of log files, which is not an attractive option. So for the foreseeable future, it will remain linux only :(

Yes, I did a bunch of googling on this, but I don't really know much about Windows programming and it seems very difficult. fork() creates two copies of the process in its current state, including all data; the processes can then continue independently from the point where fork() was called, each updating their own copy of all program state. Windows has nothing like this. You can use threads, which share all program state, or you can launch a new process from the very beginning. So you would need a totally different approach on Windows, essentially equivalent to running multiple YAFUs from the command line and having them all automatically join together to collaborate.

Before I attempt that (if I ever do) I would look at gmp-ecm to see how hard it would be to give it a thread-safe library call. That's a pretty daunting task since it's a big code base which haven't looked at yet, but it's on my list.

In any event, multi-threaded ecm probably isn't coming to Windows any time soon. Sorry, I'd do it if I knew how.

One thought though: I read that Cygwin supports fork() on Windows (reportedly with some really ugly hacks). I haven't tried it, but if you could get YAFU to compile under Cygwin, it might work.

[lorgix]

Originally Posted by Mr. Odd

Thanks for that! Any chance you can get the parallel ecm option for Windows machines?

I actually asked for that before parallell ecm was in the Linux version. I'm very futuristic you see, I even wrote Isaac Asimov's autobiography before he was born (under a pseudonym).

Originally Posted by bsquared

As of now, no. bchaffin used shared memory together with a function called fork() to get parallel ecm going. From what I understand, shared memory is possible to do in windows, but there just isn't a good (any?) way to emulate fork() on windows. The only alternative I can see is a completely different approach using external processes and parsing of log files, which is not an attractive option. So for the foreseeable future, it will remain linux only :(

That's too bad. Also, there is no spoon so no help there... I'll just keep running several instances then.

Looking forward to the patch. Thanks Ben!

[bsquared]

Originally Posted by bchaffin

Before I attempt that (if I ever do) I would look at gmp-ecm to see how hard it would be to give it a thread-safe library call. That's a pretty daunting task since it's a big code base which haven't looked at yet, but it's on my list.

I started to look into this once. If I remember right, it seemed like the major problem came from a global repository of primes (or nextprimes). There may be other roadblocks too, but I remember that being one of them.

Originally Posted by bchaffin

In any event, multi-threaded ecm probably isn't coming to Windows any time soon. Sorry, I'd do it if I knew how.

One thought though: I read that Cygwin supports fork() on Windows (reportedly with some really ugly hacks). I haven't tried it, but if you could get YAFU to compile under Cygwin, it might work.

Yeah, maybe so. I didn't go the Cygwin route on purpose, of course, because it requires extra baggage (cygwin.dll) to run on windows. Does anyone know if it will build 64 bit binaries?

[Brian Gladman]

Originally Posted by bsquared

I started to look into this once. If I remember right, it seemed like the major problem came from a global repository of primes (or nextprimes). There may be other roadblocks too, but I remember that being one of them.

Yeah, maybe so. I didn't go the Cygwin route on purpose, of course, because it requires extra baggage (cygwin.dll) to run on windows. Does anyone know if it will build 64 bit binaries?

What is wrong with using a portable library such as pthreads for this. I tend to use this rather than the Windows model since my code is then available to Linux users. I would hope that YAFU would adopt a similar strategy and not adopt a thread execution model that is not portable when, as far as I am aware, a suitable portable alternative exists.

[bsquared]

Originally Posted by Brian Gladman

What is wrong with using a portable library such as pthreads for this. I tend to use this rather than the Windows model since my code is then available to Linux users. I would hope that YAFU would adopt a similar strategy and not adopt a thread execution model that is not portable when, as far as I am aware, a suitable portable alternative exists.

Maybe you know something I don't.... but I haven't been able to use pthreads or any other threading library to parallelize GMP-ECM because the threads stomp on each other's global storage. I.e, gmp-ecm is not inherently thread-safe. Am I missing knowledge of some capability of pthreads?

[Brian Gladman]

Originally Posted by bsquared

Maybe you know something I don't.... but I haven't been able to use pthreads or any other threading library to parallelize GMP-ECM because the threads stomp on each other's global storage. I.e, gmp-ecm is not inherently thread-safe. Am I missing knowledge of some capability of pthreads?

No, Ben, I wasn't commenting on the specifics of your problem but on what appeared to be an introduction of a way of running something with multiple threads based only on the *nix model (fork etc.).

I don't know anything about the specifics of ECM thread safety (or lack of it) but I doubt that pthreads can magically introduce thread safety where there is none. How much global data is involved and is it well defined and well structured?

[bsquared]

Originally Posted by Brian Gladman

No, Ben, I wasn't commenting on the specifics of your problem but on what appeared to be an introduction of a way of running something with multiple threads based only on the *nix model (fork etc.).

Ah, I see, sorry for misunderstanding. Yes, I generally do not like to use non-portable solutions either. This specific feature was generously contributed by bchaffin who strongly wanted support for parallel ECM, enough to write his own code to accomplish it. He obviously works primarily (entirely?) within linux. Since it was relatively easy to make his work invisible to windows builds, I saw no reason not to add it. But in general this doesn't represent a departure from the desire to maintain equivalent cross-platform functionality and performance.

Originally Posted by Brian Gladman

How much global data is involved and is it well defined and well structured?

That's the problem - I don't know for sure. I'm not familiar enough with the code and all my (brief) attempts to become so have ended in frustration, so far.

[jasonp]

You could ask the GMP-ECM mailing list, the maintainers are very responsive. Alex can comment too if he's reading :)

IIRC the global state for generating primes was supposed to be the only non-reentrant part of the library. You might be able to add a __thread keyword to the relevant data items and see if it still works.

[bsquared]

NFS should now be working again in the 1.24 download.

Cheers!
- b.