View Single Post
Old 2017-02-04, 15:55   #1
ET_'s Avatar
Aug 2002
Team Italia

10010110011102 Posts
Default A distributed ECM project for Fermat numbers

--- Message crossposted on Factoring and FermatSearch threads ---
---         Please follow up on FermatSearch only             ---
Hi folks. I have been asked by many users to push and enhance the usability of the ECM method for the Fermat factors search.
Many asked to "boinc-ify" the ecm stage, or set up an ECM server and distribute the work.
Many (Mark, Carlos and Yoyo among them) offered time and resourcees to come up with a new solution.
Unfortunately, Fermat numbers are quite BIG, and require a lot of time and free memory to be correctly "ecm-ed".
They also lay (far) above the limit of 1018 digits imposed by GPU-ECM.

But I think a good way to come up with a solution exists.
Prime95 writes a savefile whenever an ecm run in interrupted. The size of the savefile is related to the FFT used.
I prepared a short table with the relevant data:

Number   Size of savefile
F12    |     1.1 kB
F13    |     2.1 kB
F14    |     4.2 kB
F15    |     8.3 kB
F16    |    16.5 kB
F17    |    32.9 kB
F18    |    65.6 kB
F19    |   131,1 kB 
F20    |   262.2 kB
F21    |   524.4 kB
F22    |     1.0 MB
F23    |     2.1 MB
F24    |     4.2 MB
F25    |     8.4 MB
F26    |    16.8 MB
The Idea
Instead of distributing workunits of one or more curves, often requiring days or even weeks to complete, let's redefine the meaning of a workunit like this: "How long should a PC sit on a single WU?". If we agree for (say) 3-4 hours, we might distribute a triple composed by the exponent, the correlated sigma and the actual savefile on each WU - where no savefile means a new curve.

More handling required, but...
We might also create 2 different subprojects: one for smaller PCs doing stage1, and one for stronger workstations doing high-memory stage2, possibly extending a single stage1 into more than 1 "telescopic" stage2s, using Prime95 for stage1 and gmp-ecm for stage2...

Another good reason
I noticed that more and more people is setting up microfarms using the new Single Board Computer platforms (Arduinos, Pines, Corteces, Oranges): Such SBCs would be perfect for a distributed work like this.

The requirements
Once such ambitious project is decided, we shall need the following sections to be managed:

1 - Hardware. An online server to share the WU and maintain the project.
2 - Application software. Something like a boinc wrapper, or an ecmnet server whatever the easier.
3 - Web management software. The management of the online archives and the statistics.
4 - A couple of administrators. They will monitor the traffic, get in charge of queue management and take care of the system.
5 - A lot of volunteers. They will participate and eventually spread the word on the net.

What I can do
I can handle the points 1, 3, half of 4 and be a part of 5.
My time and actual skills would prevent me from delivering a working software in less than 3 years, and I could have more RealLife(TM) issues on the way.
Moreover, my knowledge of the subject is more historical than technical (I never deep-delved into ECM): I could start now, but it would be easier with a good company on my side.
Finally, I would like to discuss pros and cons with you before applying any change to the actual (working) state of things: there are geniuses of programming, web developing, IT security, distributed computing among you, and I am asking your volunteering time and dedication for free (or, better, for a higher scope: Fermat factors discovery!).

Anybody interested in (at least) a theoretical discussion about feasibility?

ET_ is offline