mersenneforum.org Saving interim residues of huge P-1?
 Register FAQ Search Today's Posts Mark Forums Read

 2006-06-15, 07:52 #1 Andi47     Oct 2004 Austria 2×17×73 Posts Saving interim residues of huge P-1? Is it possible to save interim residues of (huge) p-1 factorisation attempts with GMP-ECM - say every hour or so? I am doing a p-1 with large B1 and B2 bounds now, and have the problem that windows has installed an update and now comes up with a massage like "an update has been installed and needs a reboot to complete installation. It will reboot in 5 minutes". When I leave the PC allone and fail to click "reboot later" I will definitly lose nearly 60 CPU hours of P-1 work.... Last fiddled with by Andi47 on 2006-06-15 at 07:53
 2006-06-15, 17:29 #2 akruppa     "Nancy" Aug 2002 Alexandria 2,467 Posts Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"? Alex
2006-06-15, 18:47   #3
fetofs

Aug 2005
Brazil

2×181 Posts

Quote:
 Originally Posted by akruppa Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"? Alex
On my MSys GMP-ECM directory, it's:

/* Define to 1 if you have the signal' function. */
#define HAVE_SIGNAL 1

2006-06-15, 18:50   #4
Andi47

Oct 2004
Austria

1001101100102 Posts

Quote:
 Originally Posted by akruppa Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"? Alex
Here they are, there are two of them:

Code:
/* Define to 1 if you have the signal' function. */
#define HAVE_SIGNAL 1

/* Define to 1 if you have the <signal.h> header file. */
#define HAVE_SIGNAL_H 1
As for my huge p-1 attempt: I had to restart from scratch; now I increase B1 stepwise using -save and -resume options. This is a bit more time consuming, but I won't lose *everything* again if my computer crashes or desides to reboot for windows updates.

One of these steps crashed halfway through stage 2, I guess it was out of memory. The save file was empty. Why doesn't it write the residue to the save file just after finishing stage 1?

One more strange observation:
The README file suggests to resume using a line like this:

Code:
$./ecm -resume toto 1e7 If I do this (adding the statement >outputfile.out), the output file looks like this Code: GMP-ECM 6.1 [powered by GMP 4.2.1] [ECM] Resuming P-1 residue saved by Andreas@ANDREAS with GMP-ECM 6.1 on Thu Jun 15 12:00:42 2006 Input number is 2^1061-1 (320 digits) Using B1=1000000000-3000000000, B2=3181298620516, polynomial x^120 Step 1 took 8718250ms [ECM]? So what did it do actually? ECM or P-1? If I use this imput line Code: $ ./ecm -resume toto -pm1 1e7 >outputfile.out
the outputfile starts like this:

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1] [P-1]
so NOW it looks like p-1...

Last fiddled with by Andi47 on 2006-06-15 at 18:52

2006-06-15, 18:55   #5
ColdFury

Aug 2002

26·5 Posts

Quote:
 It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW.
Windows supports some signals, including SIGTERM I believe.

2006-06-15, 19:34   #6
akruppa

"Nancy"
Aug 2002
Alexandria

2,467 Posts

Quote:
 Originally Posted by Andi47 Here they are, there are two of them: Code: /* Define to 1 if you have the signal' function. */ #define HAVE_SIGNAL 1 /* Define to 1 if you have the header file. */ #define HAVE_SIGNAL_H 1
Oh good, so GMP-ECM will register the signal handler. Can you try interrupting GMP-ECM in stage 1 with the kill command? It should say "Interrupted at prime xxx" and exit.

Quote:
 Originally Posted by Andi47 As for my huge p-1 attempt: I had to restart from scratch; now I increase B1 stepwise using -save and -resume options. This is a bit more time consuming, but I won't lose *everything* again if my computer crashes or desides to reboot for windows updates.
That's what I do when I do a very long P-1 run. Not the most elegant solution, but works.

Quote:
 Originally Posted by Andi47 One of these steps crashed halfway through stage 2, I guess it was out of memory. The save file was empty. Why doesn't it write the residue to the save file just after finishing stage 1?
We don't know whether we are to write a save file line after stage 1. Stage 2 might find a factor and complete the factorisation. Or it might find a factor and leave a composite cofactor, but in this case the new cofactor should go in the save file. Another reason is that at the moment, all the save/resume logic is in the main module, not in the ecm/p-1/p+1 modules. At least the code for resuming needs to be, as main() makes the choice which factoring method to call. I've thought about moving the save code into the respective method's module. But I think it might be better to write checkpoints in addition to the regular save files, where the checkpoint is written periodically during and at the end of stage 1.

Quote:
 Originally Posted by Andi47 [ECM]? So what did it do actually? ECM or P-1?
The info stored in the save files overrides the method given on the command line (default ecm, if -pp1 and -pm1 are not given). The "Resuming P-1 residue" is what matters here. The "[ECM]" text is a bit confusing in this case, maybe we should just remove that.

Alex

2006-06-15, 20:12   #7
Andi47

Oct 2004
Austria

2·17·73 Posts

Quote:
 Originally Posted by akruppa Oh good, so GMP-ECM will register the signal handler. Can you try interrupting GMP-ECM in stage 1 with the kill command? It should say "Interrupted at prime xxx" and exit.
Where do I enter the command, and what's the exact line?

Quote:
 We don't know whether we are to write a save file line after stage 1. Stage 2 might find a factor and complete the factorisation. Or it might find a factor and leave a composite cofactor, but in this case the new cofactor should go in the save file. Another reason is that at the moment, all the save/resume logic is in the main module, not in the ecm/p-1/p+1 modules. At least the code for resuming needs to be, as main() makes the choice which factoring method to call. I've thought about moving the save code into the respective method's module. But I think it might be better to write checkpoints in addition to the regular save files, where the checkpoint is written periodically during and at the end of stage 1.
As I have seen, it would be nice to have a save file just at the end of Stage 1 - in case the program crashes at stage two. Otherwise the time of stage 1 AND two are lost. IF a factor is found, the save file can still be modified, overwritten or deleted.

See it like this: Writing the save file consumes a split second of CPU-time. A crash halfway through stage 2 (out of memory, or something?) whithout having a savefile may loose a couple of hours of cpu-time.

As for the checkpoints: I agree - no more 'ing because of windoze updates forcing a reboot after 60 hours of unsaved p-1 halfway through stage 1...

Quote:
 The info stored in the save files overrides the method given on the command line (default ecm, if -pp1 and -pm1 are not given). The "Resuming P-1 residue" is what matters here. The "[ECM]" text is a bit confusing in this case, maybe we should just remove that.
Or move it to the ecm, p-1 and p+1 -modules: The main module could write a line like

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1]
and the submodule could add the [ECM] or [P-1] or [P+1].

(edit: typos)

Last fiddled with by Andi47 on 2006-06-15 at 20:20

 2006-06-15, 21:36 #8 smh     "Sander" Oct 2002 52.345322,5.52471 118910 Posts Regarding Windows Update, why don't you download the updates automatically but choose to install them when it suits you? I think if you open an application like wordpad, type a few characters but NOT save the document, windows can't reboot. It will stop at the save document dialog box. It might have closed gmp-ecm before that though.
 2006-06-16, 11:47 #9 Andi47     Oct 2004 Austria 9B216 Posts Code: GMP-ECM 6.1 [powered by GMP 4.2.1] [P-1] Resuming P-1 residue saved by Andreas@ANDREAS with GMP-ECM 6.1 on Thu Jun 15 12:00:42 2006 Input number is 2^1061-1 (320 digits) Using B1=1000000000-5000000000, B2=705032704-6357399348316, polynomial x^120 Step 1 took 17439172ms Step 2 took 56984ms Savefile is empty...
2006-06-16, 11:56   #10
akruppa

"Nancy"
Aug 2002
Alexandria

2,467 Posts

Quote:
 Originally Posted by Andi47 Where do I enter the command, and what's the exact line?
At the MinGW bash command prompt. Try

$ecm 1e9 1 <somecomposite.txt & [1] pid <GMP-ECM output>$ kill pid

The number after [1] is the process id of the GMP-ECM process. You need to give it to the kill command to specify which process to kill. Note: if there's no $prompt after the GMP-ECM output, just hit enter once. Also, can you please run GMP-ECM as above, but instead using the kill command terminate the process in the Windows task manager? Does it print the "Interrupted at prime ..." message then? Quote:  Originally Posted by Andi47 Savefile is empty... What exact command line did you use for this run? Did you specify both -resume and -save? Alex 2006-06-16, 15:41 #11 Andi47 Oct 2004 Austria 2·17·73 Posts Quote:  Originally Posted by akruppa At the MinGW bash command prompt. Try$ ecm 1e9 1 $kill pid The number after [1] is the process id of the GMP-ECM process. You need to give it to the kill command to specify which process to kill. Note: if there's no$ prompt after the GMP-ECM output, just hit enter once.
The process dies without any "interrupted at prime..." message.

Quote:
 Also, can you please run GMP-ECM as above, but instead using the kill command terminate the process in the Windows task manager? Does it print the "Interrupted at prime ..." message then?
I did this after trying the kill command and get this:

Code:
\$ecm 1e9 <1701-c205 &
[2] 1080
[1]   Terminated           ecm 1e9 <1701-c205
Quote:
 What exact command line did you use for this run? Did you specify both -resume and -save?
I typed
Code:
ecm -save m1061_5e9.sav -resume m1601_1e9.sav -pm1 -mpzmod -maxmem 1600 -treefile /var/tmp/ecmtree 5e9 >m1601_5e9.out
My savefile which I resumed from (m1601_1e9.sav) reads
Code:
METHOD=P-1; B1=1000000000; N=2^1061-1; X=0x18754a43f3406f17854426c70fcb5066cd33b9a9b8317e965f7f9dabe3d8ca3527b32ab0f5633fc4c257db1e969dbf8a4d91a774ad7e728f94af6ea8c110a5b086baa319ba7cb044ca8e20fdfae80db0e7781286bac6143cec25f2c687cb6f0aafe32df371e50fa05ee6641b522f4877da01e14bce95c69741b36b0fd8eff307fe9fa700aa; CHECKSUM=653811958; PROGRAM=GMP-ECM 6.1; X0=0xd3acfc5e; WHO=Andreas@ANDREAS; TIME=Thu Jun 15 12:00:42 2006;`
I did analogous lines with lower bounds before, but without the -maxmem and -treefile statements, and it worked with no problems.

Edit: Corrected my input line

Last fiddled with by Andi47 on 2006-06-16 at 15:59

 Similar Threads Thread Thread Starter Forum Replies Last Post smslca Math 0 2012-10-12 06:42 zanmato Software 12 2012-04-18 14:56 garambois Aliquot Sequences 50 2012-01-19 18:25 Primeinator Information & Answers 14 2008-09-19 08:46 GP2 Data 2 2003-10-28 22:51

All times are UTC. The time now is 04:04.

Sun May 22 04:04:38 UTC 2022 up 38 days, 2:05, 0 users, load averages: 1.45, 1.40, 1.30