mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > And now for something completely different

Reply
 
Thread Tools
Old 2021-01-03, 07:33   #34
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

3,527 Posts
Default

It would be a nice feature if Primo showed % done and an ETA.

Have you experimented with running more than one instance of Primo, maybe at different priorities? There are a lot of free cycles during phase 1.

Since ryanp is certifying M78737 cofactor, I'll have a stab at running M82939 cofactor (prp24948) at a low priority in parrallel with R49081,

Last fiddled with by paulunderwood on 2021-01-03 at 07:49
paulunderwood is offline   Reply With Quote
Old 2021-01-03, 08:09   #35
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

9,257 Posts
Cool

There are pros and cons. Yes, you can run some other computations, but then (just my guess) primo master thread doesn't serve slaves with that share memory that it holds (that and queuing the spawned kids is its only work) all too well, and primo progress slows down disproportionally.

If you observed "top" then you can see mysterious parameters passed onto "stk*1" workers. I think the second to last parameter is a reference to its private offset into shared mem. They all borrow data from master by reference (master holds a couple Gb of RAM); the spawns are born small and then suddenly show 1.5g of RAM, too.

Ok, I'll tell you more. I once tried to distribute slaves across cluster. (I wrote wrappers that passed all parameters and managed quasi-queues.) But it was all in vain because they could not access memory of the master, and all quit.
Conclusion: the 1st stage has to be done on one physical node (with shmem).

I was able to run 2nd stage faster (if you have many nodes), though it is tedious and easy to make a mistake. YOU'VE BEEN WARNED; if you don't have a firm hand - don't do it. And disclaimer: that was with old versions - pre 4.3, so all .t files were processed in order.

Here is the sketch of how:
  • keep in mind that what 2nd stage does is: it takes any 'uncooked' .t file, and cooks it, and writes it back. That is embarrassingly parallel and doesn't have any if/else branching.
  • have at least one .t file reprocessed in master node (it becomes smaller and its datestamp is newer than the .tmp file)*
  • clone the complete work folder to slave node
  • on the slave node: copy the finished .t over half of the unfinished .t files (this will make the state of the "proof object" patently violated, but you will later only use all other .t files, and send them back to master)
  • start another primo on the slave node.
  • the slave primo will skip over half of the "fake-done" .t files (they have the 'done' bit set in their header) and will do all others. (and then it will fail and will probably delete all temp files, so you have to babysit and stop it before it is fully done.)
  • At that time master will still be busy. Stop it and transfer only the "really done" .t files from the slave, overwrite the corresponding undone .t files
  • restart master primo
  • it will finish the half that is its own and will find all others already done and will combine all .t files into the .out file.
  • Caution: one wrong move and you shot yourself in the foot**. Have a legitimate backup of non-faked native primo work folder.

TL;DR version: it is not worth the saved time. Don't do it.
____
* perhaps a cooked .t file can be taken frozen for some previously done project. Never tried. Who knows - maybe primo master does light sanity check on the cooked .t file? maybe it only reads its magic header (several bytes). Maybe it checks a few more things and then it will reject a foreign .t file.

** there was a comparative list of programming languages; what happens when the programmer shoots himself in the foot. There was one language where the foot then explodes. That is similar to what will happen here.
Batalov is offline   Reply With Quote
Old 2021-01-03, 08:23   #36
paulunderwood
 
paulunderwood's Avatar
 
Sep 2002
Database er0rr

DC716 Posts
Default

Hmm, I have just created a second Primo folder and am running M-cofactor at niceness 19. I have 128 threads after all. But you seem to be right about top -- I'll monitor the situation for a week or two,
paulunderwood is offline   Reply With Quote
Old 2021-01-03, 08:37   #37
Batalov
 
Batalov's Avatar
 
"Serge"
Mar 2008
Phi(4,2^7658614+1)/2

9,257 Posts
Default

What Primo does is it takes "an onion" and peels layers, down to the inner core. (Well for the onion, the volume is R^3, but this is a nearly good analogy. Think of it as if volume formula was R^4.5.)
So, here is a neat rule of thumb: if you peel off only 1/7 of the diameter, you have done half of the job.

The reverse is true:
Q: suppose you wanted to do a series of projects each twice as hard as the last.
A: Recipe to achieve that: always add +1/6 of the size of the last number.

In "Bits", primo shows how small the current onion has become. It doesn't matter for primo if you are just beginning a 70,000-bit number or if you peeled your 88,000-bit number to a 70,000-bit intermediate number. If I started right now another 70,000-bit number on another sibling machine, it would progress nearly exactly like this number in progress.
Batalov is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
15*2^n-1, n>1M Reservation Thread Kosmaj Riesel Prime Search 702 2018-04-11 07:36
5*2^n-1 Reservation Thread VBCurtis Riesel Prime Search 657 2017-09-29 15:52
Octoproth Reservation Thread Greenbank Octoproth Search 2 2007-12-26 09:58
Dodecaproth Reservation Thread Greenbank Octoproth Search 30 2006-02-09 00:33
Hexadecaproth Reservation Thread (n=76) Greenbank Octoproth Search 0 2006-01-25 13:41

All times are UTC. The time now is 02:50.

Sat Jan 16 02:50:55 UTC 2021 up 43 days, 23:02, 1 user, load averages: 1.54, 1.33, 1.36

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.