mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > GMP-ECM

Reply
 
Thread Tools
Old 2006-06-15, 07:52   #1
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

248210 Posts
Angry Saving interim residues of huge P-1?

Is it possible to save interim residues of (huge) p-1 factorisation attempts with GMP-ECM - say every hour or so?

I am doing a p-1 with large B1 and B2 bounds now, and have the problem that windows has installed an update and now comes up with a massage like "an update has been installed and needs a reboot to complete installation. It will reboot in 5 minutes". When I leave the PC allone and fail to click "reboot later" I will definitly lose nearly 60 CPU hours of P-1 work....

Last fiddled with by Andi47 on 2006-06-15 at 07:53
Andi47 is offline   Reply With Quote
Old 2006-06-15, 17:29   #2
akruppa
 
akruppa's Avatar
 
"Nancy"
Aug 2002
Alexandria

2,467 Posts
Default

Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"?

Alex
akruppa is offline   Reply With Quote
Old 2006-06-15, 18:47   #3
fetofs
 
fetofs's Avatar
 
Aug 2005
Brazil

2·181 Posts
Default

Quote:
Originally Posted by akruppa
Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"?

Alex
On my MSys GMP-ECM directory, it's:

/* Define to 1 if you have the `signal' function. */
#define HAVE_SIGNAL 1
fetofs is offline   Reply With Quote
Old 2006-06-15, 18:50   #4
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

2·17·73 Posts
Default

Quote:
Originally Posted by akruppa
Sorry, GMP-ECM has no feature to save residues periodically yet. In 6.1, we catch signals and try to exit gracefully upon SIGTERM and SIGINT, including writing the residue of stage 1 done so far. It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW. Can you post the lines from config.h in your GMP-ECM 6.1 directory that contain the string "HAVE_SIGNAL"?

Alex
Here they are, there are two of them:

Code:
/* Define to 1 if you have the `signal' function. */
#define HAVE_SIGNAL 1

/* Define to 1 if you have the <signal.h> header file. */
#define HAVE_SIGNAL_H 1
As for my huge p-1 attempt: I had to restart from scratch; now I increase B1 stepwise using -save and -resume options. This is a bit more time consuming, but I won't lose *everything* again if my computer crashes or desides to reboot for windows updates.

One of these steps crashed halfway through stage 2, I guess it was out of memory. The save file was empty. Why doesn't it write the residue to the save file just after finishing stage 1?

One more strange observation:
The README file suggests to resume using a line like this:

Code:
$ ./ecm -resume toto 1e7
If I do this (adding the statement >outputfile.out), the output file looks like this

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1] [ECM]
Resuming P-1 residue saved by Andreas@ANDREAS with GMP-ECM 6.1 on Thu Jun 15 12:00:42 2006 
Input number is 2^1061-1 (320 digits)
Using B1=1000000000-3000000000, B2=3181298620516, polynomial x^120
Step 1 took 8718250ms
[ECM]? So what did it do actually? ECM or P-1?

If I use this imput line

Code:
$ ./ecm -resume toto -pm1 1e7 >outputfile.out
the outputfile starts like this:

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1] [P-1]
so NOW it looks like p-1...

Last fiddled with by Andi47 on 2006-06-15 at 18:52
Andi47 is offline   Reply With Quote
Old 2006-06-15, 18:55   #5
ColdFury
 
ColdFury's Avatar
 
Aug 2002

26·5 Posts
Default

Quote:
It works nicely on unix-like OSs, but I don't know if it works at all under Windows/MinGW.
Windows supports some signals, including SIGTERM I believe.
ColdFury is offline   Reply With Quote
Old 2006-06-15, 19:34   #6
akruppa
 
akruppa's Avatar
 
"Nancy"
Aug 2002
Alexandria

2,467 Posts
Default

Quote:
Originally Posted by Andi47
Here they are, there are two of them:

Code:
/* Define to 1 if you have the `signal' function. */
#define HAVE_SIGNAL 1

/* Define to 1 if you have the <signal.h> header file. */
#define HAVE_SIGNAL_H 1
Oh good, so GMP-ECM will register the signal handler. Can you try interrupting GMP-ECM in stage 1 with the kill command? It should say "Interrupted at prime xxx" and exit.

Quote:
Originally Posted by Andi47
As for my huge p-1 attempt: I had to restart from scratch; now I increase B1 stepwise using -save and -resume options. This is a bit more time consuming, but I won't lose *everything* again if my computer crashes or desides to reboot for windows updates.
That's what I do when I do a very long P-1 run. Not the most elegant solution, but works.

Quote:
Originally Posted by Andi47
One of these steps crashed halfway through stage 2, I guess it was out of memory. The save file was empty. Why doesn't it write the residue to the save file just after finishing stage 1?
We don't know whether we are to write a save file line after stage 1. Stage 2 might find a factor and complete the factorisation. Or it might find a factor and leave a composite cofactor, but in this case the new cofactor should go in the save file. Another reason is that at the moment, all the save/resume logic is in the main module, not in the ecm/p-1/p+1 modules. At least the code for resuming needs to be, as main() makes the choice which factoring method to call. I've thought about moving the save code into the respective method's module. But I think it might be better to write checkpoints in addition to the regular save files, where the checkpoint is written periodically during and at the end of stage 1.


Quote:
Originally Posted by Andi47
[ECM]? So what did it do actually? ECM or P-1?
The info stored in the save files overrides the method given on the command line (default ecm, if -pp1 and -pm1 are not given). The "Resuming P-1 residue" is what matters here. The "[ECM]" text is a bit confusing in this case, maybe we should just remove that.

Alex
akruppa is offline   Reply With Quote
Old 2006-06-15, 20:12   #7
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

2×17×73 Posts
Default

Quote:
Originally Posted by akruppa
Oh good, so GMP-ECM will register the signal handler. Can you try interrupting GMP-ECM in stage 1 with the kill command? It should say "Interrupted at prime xxx" and exit.
Where do I enter the command, and what's the exact line?


Quote:
We don't know whether we are to write a save file line after stage 1. Stage 2 might find a factor and complete the factorisation. Or it might find a factor and leave a composite cofactor, but in this case the new cofactor should go in the save file. Another reason is that at the moment, all the save/resume logic is in the main module, not in the ecm/p-1/p+1 modules. At least the code for resuming needs to be, as main() makes the choice which factoring method to call. I've thought about moving the save code into the respective method's module. But I think it might be better to write checkpoints in addition to the regular save files, where the checkpoint is written periodically during and at the end of stage 1.
As I have seen, it would be nice to have a save file just at the end of Stage 1 - in case the program crashes at stage two. Otherwise the time of stage 1 AND two are lost. IF a factor is found, the save file can still be modified, overwritten or deleted.

See it like this: Writing the save file consumes a split second of CPU-time. A crash halfway through stage 2 (out of memory, or something?) whithout having a savefile may loose a couple of hours of cpu-time.

As for the checkpoints: I agree - no more 'ing because of windoze updates forcing a reboot after 60 hours of unsaved p-1 halfway through stage 1...

Quote:
The info stored in the save files overrides the method given on the command line (default ecm, if -pp1 and -pm1 are not given). The "Resuming P-1 residue" is what matters here. The "[ECM]" text is a bit confusing in this case, maybe we should just remove that.
Or move it to the ecm, p-1 and p+1 -modules: The main module could write a line like

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1]
and the submodule could add the [ECM] or [P-1] or [P+1].

(edit: typos)

Last fiddled with by Andi47 on 2006-06-15 at 20:20
Andi47 is offline   Reply With Quote
Old 2006-06-15, 21:36   #8
smh
 
smh's Avatar
 
"Sander"
Oct 2002
52.345322,5.52471

22458 Posts
Default

Regarding Windows Update, why don't you download the updates automatically but choose to install them when it suits you?

I think if you open an application like wordpad, type a few characters but NOT save the document, windows can't reboot. It will stop at the save document dialog box.

It might have closed gmp-ecm before that though.
smh is offline   Reply With Quote
Old 2006-06-16, 11:47   #9
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

2·17·73 Posts
Default

Code:
GMP-ECM 6.1 [powered by GMP 4.2.1] [P-1]
Resuming P-1 residue saved by Andreas@ANDREAS with GMP-ECM 6.1 on Thu Jun 15 12:00:42 2006 
Input number is 2^1061-1 (320 digits)
Using B1=1000000000-5000000000, B2=705032704-6357399348316, polynomial x^120
Step 1 took 17439172ms
Step 2 took 56984ms
Savefile is empty...
Andi47 is offline   Reply With Quote
Old 2006-06-16, 11:56   #10
akruppa
 
akruppa's Avatar
 
"Nancy"
Aug 2002
Alexandria

2,467 Posts
Default

Quote:
Originally Posted by Andi47
Where do I enter the command, and what's the exact line?
At the MinGW bash command prompt. Try

$ ecm 1e9 1 <somecomposite.txt &
[1] pid
<GMP-ECM output>
$ kill pid

The number after [1] is the process id of the GMP-ECM process. You need to give it to the kill command to specify which process to kill. Note: if there's no $ prompt after the GMP-ECM output, just hit enter once.

Also, can you please run GMP-ECM as above, but instead using the kill command terminate the process in the Windows task manager? Does it print the "Interrupted at prime ..." message then?

Quote:
Originally Posted by Andi47
Savefile is empty...
What exact command line did you use for this run? Did you specify both -resume and -save?

Alex
akruppa is offline   Reply With Quote
Old 2006-06-16, 15:41   #11
Andi47
 
Andi47's Avatar
 
Oct 2004
Austria

2·17·73 Posts
Default

Quote:
Originally Posted by akruppa
At the MinGW bash command prompt. Try

$ ecm 1e9 1 <somecomposite.txt &
[1] pid
<GMP-ECM output>
$ kill pid

The number after [1] is the process id of the GMP-ECM process. You need to give it to the kill command to specify which process to kill. Note: if there's no $ prompt after the GMP-ECM output, just hit enter once.
The process dies without any "interrupted at prime..." message.

Quote:
Also, can you please run GMP-ECM as above, but instead using the kill command terminate the process in the Windows task manager? Does it print the "Interrupted at prime ..." message then?
I did this after trying the kill command and get this:

Code:
$ecm 1e9 <1701-c205 &
[2] 1080
[1]   Terminated           ecm 1e9 <1701-c205
Quote:
What exact command line did you use for this run? Did you specify both -resume and -save?
I typed
Code:
ecm -save m1061_5e9.sav -resume m1601_1e9.sav -pm1 -mpzmod -maxmem 1600 -treefile /var/tmp/ecmtree 5e9 >m1601_5e9.out
My savefile which I resumed from (m1601_1e9.sav) reads
Code:
METHOD=P-1; B1=1000000000; N=2^1061-1; X=0x18754a43f3406f17854426c70fcb5066cd33b9a9b8317e965f7f9dabe3d8ca3527b32ab0f5633fc4c257db1e969dbf8a4d91a774ad7e728f94af6ea8c110a5b086baa319ba7cb044ca8e20fdfae80db0e7781286bac6143cec25f2c687cb6f0aafe32df371e50fa05ee6641b522f4877da01e14bce95c69741b36b0fd8eff307fe9fa700aa; CHECKSUM=653811958; PROGRAM=GMP-ECM 6.1; X0=0xd3acfc5e; WHO=Andreas@ANDREAS; TIME=Thu Jun 15 12:00:42 2006;
I did analogous lines with lower bounds before, but without the -maxmem and -treefile statements, and it worked with no problems.

Edit: Corrected my input line

Last fiddled with by Andi47 on 2006-06-16 at 15:59
Andi47 is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
residues and non residues of general quadratic congruences smslca Math 0 2012-10-12 06:42
Compare interim files with different start shifts? zanmato Software 12 2012-04-18 14:56
An aliquot sequence with huge, huge, huge tracts of sand !!! garambois Aliquot Sequences 50 2012-01-19 18:25
Interim and Res64 Residues Primeinator Information & Answers 14 2008-09-19 08:46
Interim iterations in status.txt are not random GP2 Data 2 2003-10-28 22:51

All times are UTC. The time now is 08:46.

Sun May 9 08:46:31 UTC 2021 up 31 days, 3:27, 0 users, load averages: 2.24, 2.76, 2.73

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.