Primo/FastECPP reservation thread - Page 4

paulunderwood · 2021-01-03, 07:33

It would be a nice feature if Primo showed % done and an ETA.

Have you experimented with running more than one instance of Primo, maybe at different priorities? There are a lot of free cycles during phase 1.

Since ryanp is certifying M78737 cofactor, I'll have a stab at running M82939 cofactor (prp24948) at a low priority in parrallel with R49081,

Batalov · 2021-01-03, 08:09

There are pros and cons. Yes, you can run some other computations, but then (just my guess) primo master thread doesn't serve slaves with that share memory that it holds (that and queuing the spawned kids is its only work) all too well, and primo progress slows down disproportionally.

If you observed "top" then you can see mysterious parameters passed onto "stk*1" workers. I think the second to last parameter is a reference to its private offset into shared mem. They all borrow data from master by reference (master holds a couple Gb of RAM); the spawns are born small and then suddenly show 1.5g of RAM, too.

Ok, I'll tell you more. I once tried to distribute slaves across cluster. (I wrote wrappers that passed all parameters and managed quasi-queues.) But it was all in vain because they could not access memory of the master, and all quit.
Conclusion: the 1st stage has to be done on one physical node (with shmem).

I was able to run 2nd stage faster (if you have many nodes), though it is tedious and easy to make a mistake. YOU'VE BEEN WARNED; if you don't have a firm hand - don't do it. And disclaimer: that was with old versions - pre 4.3, so all .t files were processed in order.

Here is the sketch of how:

keep in mind that what 2nd stage does is: it takes any 'uncooked' .t file, and cooks it, and writes it back. That is embarrassingly parallel and doesn't have any if/else branching.
have at least one .t file reprocessed in master node (it becomes smaller and its datestamp is newer than the .tmp file)*
clone the complete work folder to slave node
on the slave node: copy the finished .t over half of the unfinished .t files (this will make the state of the "proof object" patently violated, but you will later only use all other .t files, and send them back to master)
start another primo on the slave node.
the slave primo will skip over half of the "fake-done" .t files (they have the 'done' bit set in their header) and will do all others. (and then it will fail and will probably delete all temp files, so you have to babysit and stop it before it is fully done.)
At that time master will still be busy. Stop it and transfer only the "really done" .t files from the slave, overwrite the corresponding undone .t files
restart master primo
it will finish the half that is its own and will find all others already done and will combine all .t files into the .out file.
Caution: one wrong move and you shot yourself in the foot**. Have a legitimate backup of non-faked native primo work folder.

TL;DR version: it is not worth the saved time. Don't do it.
____
* perhaps a cooked .t file can be taken frozen for some previously done project. Never tried. Who knows - maybe primo master does light sanity check on the cooked .t file? maybe it only reads its magic header (several bytes). Maybe it checks a few more things and then it will reject a foreign .t file.

** there was a comparative list of programming languages; what happens when the programmer shoots himself in the foot. There was one language where the foot then explodes. That is similar to what will happen here.

paulunderwood · 2021-01-03, 08:23

Hmm, I have just created a second Primo folder and am running M-cofactor at niceness 19. I have 128 threads after all. But you seem to be right about top -- I'll monitor the situation for a week or two,

Batalov · 2021-01-03, 08:37

What Primo does is it takes "an onion" and peels layers, down to the inner core. (Well for the onion, the volume is R^3, but this is a nearly good analogy. Think of it as if volume formula was R^4.5.)
So, here is a neat rule of thumb: if you peel off only 1/7 of the diameter, you have done half of the job.

The reverse is true:
Q: suppose you wanted to do a series of projects each twice as hard as the last.
A: Recipe to achieve that: always add +1/6 of the size of the last number.

In "Bits", primo shows how small the current onion has become. It doesn't matter for primo if you are just beginning a 70,000-bit number or if you peeled your 88,000-bit number to a 70,000-bit intermediate number. If I started right now another 70,000-bit number on another sibling machine, it would progress nearly exactly like this number in progress.

Gelly · 2021-02-02, 17:32

Quote:

Originally Posted by Gelly

Got the wrong one. I evidently don't understand how to order candidates in Primo right, so instead I proved 798*Bern(8766)/(487*4657*6468702182951641) (23743 digits).

i did it twice.

i proved the bern(8766) with ecpp twice.

it'll be a bit before i get the cofactor of bern(8286).

ughhhhhhh.

paulunderwood · 2021-02-02, 18:10

So you submitted a wrong entry to the top5000 at this time? Oops

Gelly · 2021-02-02, 23:14

Quote:

Originally Posted by paulunderwood

So you submitted a wrong entry to the top5000 at this time? Oops

mhm. i imagine it's not a super big deal, but i don't know how to let caldwell know based on the page.

Batalov · 2021-02-03, 00:46

I had a couple of mishaps with Chris C, because over the years I'd submitted over 600 primes and many in very different formats. (I tried to get at least one entry in each archivable category, unless totally unfeasible). I remember that one was an irregular and E() function produces half positive and half negative values, so I submitted an entry without leading '-' so he rejected it as a negative. A few times I had simply had a typo in some formula.

His usual decision is to delete entry (and not reuse that id), but advise user to submit again corrected entry later. He typically declines to manually edit entries by direct access to the database (which he probably could). It is a robust approach to editing: "edit" = "delete" + "resubmit".

You can write using his email at the bottom of every page - he responds quite quickly (except he takes weekends properly, - he probably could read emails but he replies on next weekday). or you can add &comment=1 to the URL and write a comment.

Batalov · 2021-02-03, 20:32

Apparently Alfred proved a ranked #8 ECPP prime, too. Congrats!

(10²⁷⁶⁶⁹+7)/8313493832818655929448065598763458531111

Alfred · 2021-02-03, 20:51

Thank you - for your congrats and your correct attribution.

Gelly · 2021-02-04, 13:49

Quote:

Originally Posted by Alfred

Thank you - for your congrats and your correct attribution.

Wow, that's a good one. Nice work!

In the meantime, I have been meaning to (actually) prove the cofactor to Bern(8286), only to find what is the strangest return I have seen in a while from Primo:

Code:

[Candidate]
Input=8286.in
Output=prime-B422001C4C67B.out
Status=Sorry, Primo cannot certify the candidate

Failed in 26008 seconds. I genuinely am at a loss, as I have never ever seen this before. I checked my candidate against pfgw64, with all bases up to 15, and it seems to be really really PRP, so I guess Primo has its limitations. Will probably shoot an email to Marcel if no one here has ever seen it before.

Meanwhile, in the interest of getting my irregular prime record back, I will be doing PRP28507, a cofactor of Bern(10264).

2021-01-03, 07:33	#34
paulunderwood Sep 2002 Database er0rr 3·1,499 Posts	It would be a nice feature if Primo showed % done and an ETA. Have you experimented with running more than one instance of Primo, maybe at different priorities? There are a lot of free cycles during phase 1. Since ryanp is certifying M78737 cofactor, I'll have a stab at running M82939 cofactor (prp24948) at a low priority in parrallel with R49081, Last fiddled with by paulunderwood on 2021-01-03 at 07:49

2021-01-03, 08:09	#35
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 23515₈ Posts	There are pros and cons. Yes, you can run some other computations, but then (just my guess) primo master thread doesn't serve slaves with that share memory that it holds (that and queuing the spawned kids is its only work) all too well, and primo progress slows down disproportionally. If you observed "top" then you can see mysterious parameters passed onto "stk1" workers. I think the second to last parameter is a reference to its private offset into shared mem. They all borrow data from master by reference (master holds a couple Gb of RAM); the spawns are born small and then suddenly show 1.5g of RAM, too. Ok, I'll tell you more. I once tried to distribute slaves across cluster. (I wrote wrappers that passed all parameters and managed quasi-queues.) But it was all in vain because they could not access memory of the master, and all quit. Conclusion: the 1st stage has to be done on one physical node (with shmem). I was able to run 2nd stage faster (if you have many nodes), though it is tedious and easy to make a mistake. YOU'VE BEEN WARNED; if you don't have a firm hand - don't do it. And disclaimer: that was with old versions - pre 4.3, so all .t files were processed in order. Here is the sketch of how: keep in mind that what 2nd stage does is: it takes any 'uncooked' .t file, and cooks it, and writes it back. That is embarrassingly parallel and doesn't have any if/else branching. have at least one .t file reprocessed in master node (it becomes smaller and its datestamp is newer than the .tmp file) clone the complete work folder to slave node on the slave node: copy the finished .t over half of the unfinished .t files (this will make the state of the "proof object" patently violated, but you will later only use all other .t files, and send them back to master) start another primo on the slave node. the slave primo will skip over half of the "fake-done" .t files (they have the 'done' bit set in their header) and will do all others. (and then it will fail and will probably delete all temp files, so you have to babysit and stop it before it is fully done.) At that time master will still be busy. Stop it and transfer only the "really done" .t files from the slave, overwrite the corresponding undone .t files restart master primo it will finish the half that is its own and will find all others already done and will combine all .t files into the .out file. Caution: one wrong move and you shot yourself in the foot. Have a legitimate backup of non-faked native primo work folder. TL;DR version*: it is not worth the saved time. Don't do it. ____ perhaps a cooked .t file can be taken frozen for some previously done project. Never tried. Who knows - maybe primo master does light sanity check on the cooked .t file? maybe it only reads its magic header (several bytes). Maybe it checks a few more things and then it will reject a foreign .t file. ** there was a comparative list of programming languages; what happens when the programmer shoots himself in the foot. There was one language where the foot then explodes. That is similar to what will happen here.

2021-02-03, 20:51	#43
Alfred May 2013 Germany 97 Posts	Thank you - for your congrats and your correct attribution. Last fiddled with by Alfred on 2021-02-03 at 21:14

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
15*2^n-1, n>1M Reservation Thread	Kosmaj	Riesel Prime Search	707	2021-11-20 19:13
Where on Earth can I download FastECPP? How fast is it compared with Primo?	Stargate38	Miscellaneous Math	5	2021-11-16 17:33
5*2^n-1 Reservation Thread	VBCurtis	Riesel Prime Search	679	2021-10-09 17:33
Octoproth Reservation Thread	Greenbank	Octoproth Search	2	2007-12-26 09:58
Dodecaproth Reservation Thread	Greenbank	Octoproth Search	30	2006-02-09 00:33

2021-01-03, 08:23	#36
paulunderwood Sep 2002 Database er0rr 3·1,499 Posts	Hmm, I have just created a second Primo folder and am running M-cofactor at niceness 19. I have 128 threads after all. But you seem to be right about top -- I'll monitor the situation for a week or two,

2021-01-03, 08:37	#37
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 23515₈ Posts	What Primo does is it takes "an onion" and peels layers, down to the inner core. (Well for the onion, the volume is R^3, but this is a nearly good analogy. Think of it as if volume formula was R^4.5.) So, here is a neat rule of thumb: if you peel off only 1/7 of the diameter, you have done half of the job. The reverse is true: Q: suppose you wanted to do a series of projects each twice as hard as the last. A: Recipe to achieve that: always add +1/6 of the size of the last number. In "Bits", primo shows how small the current onion has become. It doesn't matter for primo if you are just beginning a 70,000-bit number or if you peeled your 88,000-bit number to a 70,000-bit intermediate number. If I started right now another 70,000-bit number on another sibling machine, it would progress nearly exactly like this number in progress.

2021-02-02, 18:10	#39
paulunderwood Sep 2002 Database er0rr 3·1,499 Posts	So you submitted a wrong entry to the top5000 at this time? Oops

2021-02-03, 00:46	#41
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 23515₈ Posts	I had a couple of mishaps with Chris C, because over the years I'd submitted over 600 primes and many in very different formats. (I tried to get at least one entry in each archivable category, unless totally unfeasible). I remember that one was an irregular and E() function produces half positive and half negative values, so I submitted an entry without leading '-' so he rejected it as a negative. A few times I had simply had a typo in some formula. His usual decision is to delete entry (and not reuse that id), but advise user to submit again corrected entry later. He typically declines to manually edit entries by direct access to the database (which he probably could). It is a robust approach to editing: "edit" = "delete" + "resubmit". You can write using his email at the bottom of every page - he responds quite quickly (except he takes weekends properly, - he probably could read emails but he replies on next weekday). or you can add &comment=1 to the URL and write a comment.

2021-02-03, 20:32	#42
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 10011101001101₂ Posts	Apparently Alfred proved a ranked #8 ECPP prime, too. Congrats! (10²⁷⁶⁶⁹+7)/8313493832818655929448065598763458531111