mersenneforum.org (https://www.mersenneforum.org/index.php)
-   GMP-ECM (https://www.mersenneforum.org/forumdisplay.php?f=55)
-   -   Feature Request!!! Save after p-1 Step 1 (https://www.mersenneforum.org/showthread.php?t=7782)

 Andi47 2007-04-07 20:23

Feature Request!!! Save after p-1 Step 1

This happened after a p-1 attempt with:
ecm -v -resume 5_3_257+_pm1_1e9.sav -save 5_3_257+_pm1_10e9.sav -maxmem 2500 10e9 >5_3_257+_pm1_10e9.out

Resuming P-1 residue saved by Andreas@ANDREAS with GMP-ECM 6.1.1 on Sat Apr 07 12:40:48 2007
Input number is 175619522044094444730646441116709020054232412086062437698861604586816985151966452610210857901106313441605170982007750547 (120 digits)
Using mpz_powm
Using B1=1000000000-10000000000, B2=1410065408-15893193115816, polynomial x^120
dF=524288, k=5, d=5705700, d2=17, i0=231
Step 1 took 10567515ms
Using 28 small primes for NTT
Estimated memory usage: 2102M
Initializing table of differences for F took 1141ms
Computing roots of F took 66844ms
Building F from its roots took 167125ms
Computing 1/F took 109921ms
Initializing table of differences for G took 1468ms
Computing roots of G took 53750ms
[/code]

Error:

Savefile is empty - 3 cpu-hours for nothing... :furious: :nuke:

 jasong 2007-04-08 00:33

[QUOTE=Andi47;103232]
Error:

Savefile is empty - 3 cpu-hours for nothing... :furious: :nuke:[/QUOTE]
You think that's annoying? Try ecming on wblipp's "Most Wanted" server. A lightning storm at the right(wrong?) time could make me cry like a little girl. (not really, but it's still really stressful).

I think it would be great if any curve that takes more than an hour could have a setable save feature.

 Andi47 2007-04-08 07:04

[QUOTE=jasong;103245]You think that's annoying? Try ecming on wblipp's "Most Wanted" server. A lightning storm at the right(wrong?) time could make me cry like a little girl. (not really, but it's still really stressful).

I think it would be great if any curve that takes more than an hour could have a setable save feature.[/QUOTE]

I agree about the save feature. This time it was *only* three hours (normally I set B2 to 1 to avoid this kind of loss and do stage 2 later, but this time I forgot and Murphys Law striked), but one time I had lost 5 *days* of a p-1 stage 2 (was B1=150e9, B2=739,720,038,949,720) because I had to switch off the Laptop for transport and Standby did not work properly.

 akruppa 2007-04-11 16:21

I fully agree that this feature is missing. Only problem is that in order to put the feature in, we'll have to rearrange some code - the ECM computations are part of the library, whereas the savefile I/O are part of the application. We'll either have to move the savefile I/O into the library, where it doesn't belong imho, or add some kind of callback interface. It'll happen eventually...

Alex

 Andi47 2007-04-11 18:06

[QUOTE=akruppa;103470]I fully agree that this feature is missing. Only problem is that in order to put the feature in, we'll have to rearrange some code - the ECM computations are part of the library, whereas the savefile I/O are part of the application. We'll either have to move the savefile I/O into the library, where it doesn't belong imho, or add some kind of callback interface. It'll happen eventually...

Alex[/QUOTE]

This feature missing and even more the inability to do interim saves during step 2 are the main reasons why I stopped attacking M1061. I don't want to lose 3 weeks of cpu-time during p-1ing with B1=5e11 (or even 1e12 :showoff: ) and b2=2e15 due to a power outage or the need to transport my laptop and the windoze standby not working properly...

 akruppa 2007-04-12 09:29

You can always do stage 1 in smaller pieces. I.e. go to B1=5e10 and save, resume that and to to B1=1e11 and save, resume that, etc...

There might be an interim save feature for stage 2 some day, but the working data set is very large. Then again, the -treefile option saves most of that intermediate data already, so saving the rest at periodic intervals (say, after each block) shouldn't be too much worse. As a workaround, you can do stage 2 in steps as well, using the B2min-B2max options, i.e.

ecm -resume savefile.1e10 1e10 1e10-1e14
ecm -resume savefile.1e10 1e10 1e14-2e14
ecm -resume savefile.1e10 1e10 2e14-3e14
...

adding the (currently broken, sorry!) -maxmem or (functioning, yay!) -k parameters as necessary to get the polynomial degree and hence memory use you want.

Alex

 Andi47 2007-04-12 10:14

[QUOTE=akruppa;103527]You can always do stage 1 in smaller pieces. I.e. go to B1=5e10 and save, resume that and to to B1=1e11 and save, resume that, etc...

There might be an interim save feature for stage 2 some day, but the working data set is very large. Then again, the -treefile option saves most of that intermediate data already, so saving the rest at periodic intervals (say, after each block) shouldn't be too much worse. As a workaround, you can do stage 2 in steps as well, using the B2min-B2max options, i.e.

ecm -resume savefile.1e10 1e10 1e10-1e14
ecm -resume savefile.1e10 1e10 1e14-2e14
ecm -resume savefile.1e10 1e10 2e14-3e14
...

adding the (currently broken, sorry!) -maxmem or (functioning, yay!) -k parameters as necessary to get the polynomial degree and hence memory use you want.

Alex[/QUOTE]

I know, that I can do stage 1 in steps - I am frequently using this feature.

I did not know, that this is also possible for stage 2, thanks for pointing out.

Questions:

When I do like you said (i.e. entering B2 ranges of 1e10-1e14, 1e14-2e14, etc.), does it save the results of B2, so that I can continue for example at 2e14, or - if not - how does this work? (example: If the second largest factor of p-1 is 9e9 and the biggest is 2.5e14, the factor is found when doing b2=2e14 -3e14, but not with 1e14-2e14 or 3e14-4e14 in your example, am I right?)

 akruppa 2007-04-12 11:04

It does not save the highest B2 value tried so far anywhere, you'll have to specify the B2 range you want to do manually each time.

Where exactly the factor [I]p[/I] with largest factor in [I]p[/I]-1 around 2.5e14 will be found is hard to tell, due to the Brent-Suyama extension.
With -power 2, it will be as you suggested. However, if the B2 value is much higher than 2.5e14, it might be included as a composite value, i.e. if [I]q[/I]*2.5e14 is in your B2min-B2max range for some [I]q[/I] that does not divide d1*d2.
With Brent-Suyama, all bets are off... the missing prime in [I]p[/I]-1 might be included pretty much at any time during stage 2, if you're lucky. But it is still true that if the missing prime is in B2min-B2max, it will be included in any case, and the factor will be found.

Alex

 Andi47 2007-04-12 11:12

Thanks.

So - do I understand right, that - if I have got enough RAM or specify -maxmem smaller half of available RAM - I could also do parallelized Stage 2 - i.e. one CPU does B2 = 1e14-2e14 and the other does B2 = 2e14-3e14 at the same time ?

 akruppa 2007-04-12 11:40

You could, but stage 2 gets more efficient as the degree dF of the polynomial increases. A better way of scheduling work would be to let one cpu do stage 1 on another number which uses almost no memory, and let the other cpu do stage 2 and give it all the memory your machine has.

Alex

 All times are UTC. The time now is 09:52.