P-1: two stages, two machines?

James Heinrich · 2004-07-20, 16:59

Is it possible to have two machines working on the same set of P-1 (not at the same time) -- I have 2 machines, one I can dedicated 512-768MB (day-night) of RAM to, the other can only have about 300MB dedicated, and only at night. So I wonder if it's possible to have the low-RAM machine do all the stage-1 factoring (which takes minimal RAM), then copy the worktodo entries and the temp files over the high-RAM machine and let it do all the stage-2 factoring. Is this a reasonable idea, or is there a reason this won't work?

dave_0273 · 2004-07-20, 17:26

It would work to an extent. The main problem would be with the size of the save files. Each save file from memory is about 4mb or so. Over 100 exponents would be about 400mb. As long as it isn't a problem to store and transfer that much data, you could split stage one and stage two between two machines.

If you do test it, let me know how it goes.

James Heinrich · 2004-07-20, 18:17

I will test it and report back.
I have 400GB of disk space between the two machines, and 400MB takes about a minute to copy from one to the other - not a problem :)

James Heinrich · 2004-07-21, 02:40

OK, I'm starting the test...

First question: is it possible to override B2 values saved in the file? I assume that the B2 value is calculated on the low-memory machine based on how much RAM is available there, which is obviously less than it is on the machine I'm doing stage-2 on.

Hmm, I see the B2 value at offset 28-32 in the savefile - but if I edit that I get a warning about "Error reading save file", so I guess there's a checksum somewhere

Anyhow, I'd prefer not to have to hexedit every save file... Any suggestions for how I can set up my computers so that the savefiles come to the high-RAM computer with an appropriate B2 value? I can't set the nighttime RAM setting for the low-RAM computer higher than 459MB, so I have, and I have "nighttime" set to between 00:00h and 00:01h, so it won't really do much stage-2.

geoff · 2004-07-21, 04:37

Quote:

Originally Posted by James Heinrich

First question: is it possible to override B2 values saved in the file?

Don't edit the save file!

I think it is best to set the B1 value manually on the stage one machine using the Pminus1 work entry rather than PFactor. If your stage two machine wants stage one for exponent 13xxxxxx done to B1=80000 for example, then put the following line in the worktodo.ini of the stage one machine:

Pminus1=13xxxxxx,80000,1,0,0

The '1' in the B2 field means stage two will not be done. A save file will be left once stage one has finished. When it has been copied, use this line in the worktodo.ini for the stage two machine:

Pfactor=13xxxxxx,65,1

That will then just do the stage two step, and the save file will be deleted as usual when finished. You can also use Pminus1 to set B2 manually if you want, but then you must remember to delete the save files manually.

garo · 2004-07-21, 08:50

Stage-2 will compute a new B2 value when it finds that the memory allocated has changed. It will ignore the B2 value in the savefile.

geoff · 2004-07-21, 10:37

Quote:

Originally Posted by garo

Stage-2 will compute a new B2 value when it finds that the memory allocated has changed. It will ignore the B2 value in the savefile.

I mean that instead of the Pfactor=13xxxxxx,65,1 line in the worktodo.ini for the stage two machine, use a Pminus1 line like this:

Pminus1=13xxxxxx,80000,1640000,0,0

then that B2 value will be used regardless of memory settings or what the stage one save file says. (The memory settings will still be used to limit the actual amount of memory used though).

James Heinrich · 2004-07-21, 13:47

Thanks for the replies.

How does the B2 value compare with the size of the exponent and the amount of RAM available? How are the values auto-computed?

garo · 2004-07-22, 14:14

It is complicated formula. The values are computed so that the project throughput is maximized. The addition of extra memory can either
1) Speed the test up with the same B2 bounds.
2) Slow the test by taking higher B2 bounds but increasing the chance of finding the factor sufficiently so as to compensate for the increase in test time thereby increasing the project throughput.

Usually higher exponents result in higher B1/B2 values. As for the rest, try the ame exponent in both your machines and see what bounds are chosen.

James Heinrich · 2004-07-28, 04:37

Quote:

Originally Posted by James Heinrich

I will test it and report back.

Yes, it works, but it's probably more hassle than it's worth in my case. If I had a case where I had one reasonably fast-CPU machine but with very little RAM to spare then this could be useful: it can do useful work using only 10MB or whatever, rather than requiring hundreds of MB (as long as there's a high-RAM machine available to hand work off to).

As for me, I've created a pagefile on my low-RAM machine and set Prime95 to use 459MB overnight (supposedly max 80% of installed RAM, but 459 ~= 90% of 512MB??) That'll just have to do

biwema · 2004-07-28, 07:26

Maybe one of the fastest way to split up p-1 assignments on a computer with little ram and one computer with much ram is this one.

Getting parameters:
Start one Pfactor assignment on your computer with much ram. (with 512Mb, 459M (90%) is the max, what prime95 uses to prevent swapping).

When started, you can see the optimal B1 and B2 for this magnitude of candidates. For example B1=80000 B2=1500000.

Change the type of assignment in the worktodo.ini

from
Pfactor=xxxxxxxx,65,1

to
Pminus1=xxxxxxxx,80000,1500000,0,0

If you have many candidates of Pfactor line, you can use search/replace for that process.

Copy that file on both computers.

On the computer with few memory (for stage 1), search/replace all 1500000
to 80000.

you get
Pminus1=xxxxxxxx,80000,80000,0,0

Now you can start computer 1. Whenm the first few ranges are complete, you can move the savefiles to the computer with much memory. And start calculating then.
As long as the computer withm uch memory does not catch up with the first one, you can let both run permanently and move the savefiles before the second computer sdtarts these exponents.

If the second computer is faster, you can either stop it for some time or remove some Pminus-lines from the first comuter (the second computer will start these assignments from bottom (both stages) and the first computer has time to finish some more stage 1.

2004-07-20, 16:59	#1
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 3·13·83 Posts	P-1: two stages, two machines? Is it possible to have two machines working on the same set of P-1 (not at the same time) -- I have 2 machines, one I can dedicated 512-768MB (day-night) of RAM to, the other can only have about 300MB dedicated, and only at night. So I wonder if it's possible to have the low-RAM machine do all the stage-1 factoring (which takes minimal RAM), then copy the worktodo entries and the temp files over the high-RAM machine and let it do all the stage-2 factoring. Is this a reasonable idea, or is there a reason this won't work?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
mpi Two Machines Take longer than One for LA	EdH	Msieve	9	2016-12-10 20:53
Can stages of polynomial selection be run separately	jux	YAFU	11	2016-01-14 17:18
Splitting P-1 stages	TheMawn	Information & Answers	3	2013-10-13 00:07
Machines	R.D. Silverman	NFSNET Discussion	12	2004-07-02 12:12
Question "How many stages"?	Matthias C. Noc	Software	6	2003-12-03 18:13

2004-07-20, 17:26	#2
dave_0273 Oct 2003 Australia, Brisbane 2·5·47 Posts	It would work to an extent. The main problem would be with the size of the save files. Each save file from memory is about 4mb or so. Over 100 exponents would be about 400mb. As long as it isn't a problem to store and transfer that much data, you could split stage one and stage two between two machines. If you do test it, let me know how it goes.

2004-07-20, 18:17	#3
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 3×13×83 Posts	I will test it and report back. I have 400GB of disk space between the two machines, and 400MB takes about a minute to copy from one to the other - not a problem :)

2004-07-21, 02:40	#4
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 3·13·83 Posts	OK, I'm starting the test... First question: is it possible to override B2 values saved in the file? I assume that the B2 value is calculated on the low-memory machine based on how much RAM is available there, which is obviously less than it is on the machine I'm doing stage-2 on. Hmm, I see the B2 value at offset 28-32 in the savefile - but if I edit that I get a warning about "Error reading save file", so I guess there's a checksum somewhere Anyhow, I'd prefer not to have to hexedit every save file... Any suggestions for how I can set up my computers so that the savefiles come to the high-RAM computer with an appropriate B2 value? I can't set the nighttime RAM setting for the low-RAM computer higher than 459MB, so I have, and I have "nighttime" set to between 00:00h and 00:01h, so it won't really do much stage-2.

2004-07-21, 08:50	#6
garo Aug 2002 Termonfeckin, IE 2757₁₀ Posts	Stage-2 will compute a new B2 value when it finds that the memory allocated has changed. It will ignore the B2 value in the savefile.

2004-07-21, 13:47	#8
James Heinrich "James Heinrich" May 2004 ex-Northern Ontario 3237₁₀ Posts	Thanks for the replies. How does the B2 value compare with the size of the exponent and the amount of RAM available? How are the values auto-computed?

2004-07-22, 14:14	#9
garo Aug 2002 Termonfeckin, IE 3×919 Posts	It is complicated formula. The values are computed so that the project throughput is maximized. The addition of extra memory can either 1) Speed the test up with the same B2 bounds. 2) Slow the test by taking higher B2 bounds but increasing the chance of finding the factor sufficiently so as to compensate for the increase in test time thereby increasing the project throughput. Usually higher exponents result in higher B1/B2 values. As for the rest, try the ame exponent in both your machines and see what bounds are chosen.

2004-07-28, 07:26	#11
biwema Mar 2004 3·127 Posts	Maybe one of the fastest way to split up p-1 assignments on a computer with little ram and one computer with much ram is this one. Getting parameters: Start one Pfactor assignment on your computer with much ram. (with 512Mb, 459M (90%) is the max, what prime95 uses to prevent swapping). When started, you can see the optimal B1 and B2 for this magnitude of candidates. For example B1=80000 B2=1500000. Change the type of assignment in the worktodo.ini from Pfactor=xxxxxxxx,65,1 to Pminus1=xxxxxxxx,80000,1500000,0,0 If you have many candidates of Pfactor line, you can use search/replace for that process. Copy that file on both computers. On the computer with few memory (for stage 1), search/replace all 1500000 to 80000. you get Pminus1=xxxxxxxx,80000,80000,0,0 Now you can start computer 1. Whenm the first few ranges are complete, you can move the savefiles to the computer with much memory. And start calculating then. As long as the computer withm uch memory does not catch up with the first one, you can let both run permanently and move the savefiles before the second computer sdtarts these exponents. If the second computer is faster, you can either stop it for some time or remove some Pminus-lines from the first comuter (the second computer will start these assignments from bottom (both stages) and the first computer has time to finish some more stage 1.