MSieve 1.45 Polyselection question (154 Digit Num) - Page 2

miklin · 2010-07-17, 12:53

Quote:

Originally Posted by Carlo

Hi Jeff,

Tue Jun 29 16:32:05 2010 -> Working with NAME = bmw_mode_85
Tue Jun 29 16:32:05 2010 factoring 8224973201493734039216932833462996815932154044113673505636726252834676063695616729466358005376619469264571014058650019804568205019013693877262015651491183 (154 digits)

Regards,
-Carlo

And it is possible to learn that it for a key such. From what block BMW (lock). And I such did not meet that.

Karl M Johnson · 2010-07-17, 16:32

Quote:

Originally Posted by asp

There are issues with the NVIDIA drivers though and they do crash.

It's not the drivers, it's Win's WDDM.
The fix is simple: create file called patch.reg, paste this in it

Code:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers]
"TdrLevel"=dword:00000000

and run it.Should not crash after that.

asp · 2010-07-17, 19:17

It's a driver issue. I've already tracked it down. Thanks for the suggestion though.

Carlo · 2010-08-23, 21:19

Sofar it finished the finding the needed 57million relations needed for this 154 digit number. The next step is the -nc1 step, it loads all the relations then starts working but craps out after about an hour.

Saying it failed to calloc 231881660 bytes :-) This machine is running Vista32 with 4 GB of RAM.

Does any one have an estimate of what the required amount of memory is to make this run? Or do I need to use a 64 bit OS for this to work?

-Carlo

jasonp · 2010-08-23, 22:25

Postprocessing for a number this size should fit in 1.5 - 2.5GB, but if your memory is fragmented or you need 2GB then you have run out of VM, no matter how much physical RAM you have. Unless you can boot the machine to allow 3GB of VM for processes, then you will need a 64-bit OS. IIRC linux x86 gives processes 3GB of VM by default.

Carlo · 2010-08-24, 15:35

That did help (did the BCDEDIT /set increaseuserva 3072 and rebooted), now I get:

Found 57664076 relations, 101.1% of the estimated minimum (57029236).
-> msieve -s num.dat -l num.log -i num.ini -nf num.fb -t 4 -nc1
read 10M relations
read 20M relations
read 30M relations
read 40M relations
read 50M relations
-> Running matrix solving step ...
-> msieve -s num.dat -l num.log -i
num.ini -nf num.fb -t 4 -nc2
linear algebra completed 69843 of 4310723 dimensions (1.6%, ETA 60h23m)
error: corrupt state, please restart from checkpoint
Return value -1. Terminating...
siever terminated

What would cause this error? Last few lines of log:

Tue Aug 24 08:45:39 2010 Msieve v. 1.46
Tue Aug 24 08:45:39 2010 random seeds: 6922a894 549e0ca8
Tue Aug 24 08:45:39 2010 factoring 8224973201493734039216932833462996815932154044113673505636726252834676063695616729466358005376619469264571014058650019804568205019013693877262015651491183 (154 digits)
Tue Aug 24 08:45:41 2010 searching for 15-digit factors
Tue Aug 24 08:45:42 2010 commencing number field sieve (154-digit input)
Tue Aug 24 08:45:42 2010 R0: -885345494398234005630543262076
Tue Aug 24 08:45:42 2010 R1: 249843090087716533
Tue Aug 24 08:45:42 2010 A0: -8294699117965540061436793792349975162625
Tue Aug 24 08:45:42 2010 A1: 135954959572494343196175400442215
Tue Aug 24 08:45:42 2010 A2: -93888007586351928671641113
Tue Aug 24 08:45:42 2010 A3: -10081478007408346455
Tue Aug 24 08:45:42 2010 A4: 2318577493178
Tue Aug 24 08:45:42 2010 A5: 15120
Tue Aug 24 08:45:42 2010 skew 16624150.69, size 4.492e-015, alpha -7.981, combined = 2.619e-012 rroots = 3
Tue Aug 24 08:45:42 2010
Tue Aug 24 08:45:42 2010 commencing linear algebra
Tue Aug 24 08:45:45 2010 read 4310723 cycles
Tue Aug 24 08:45:55 2010 cycles contain 12066493 unique relations
Tue Aug 24 08:45:55 2010 read 0 relations
Tue Aug 24 08:45:55 2010 error: cannot locate relation 57048268

Batalov · 2010-08-24, 16:32

The usual reason is the stressed memory (or memory controller, or the bus - the whole subsystem). msieve stresses the memory subsystem as hard as (and possibly harder than) the Prime95 torture.

Especially during hot summer, the solution may be to downclock the memory (then if it doesn't help, CPU and CPU IMC) a bit. (E.g. my 8Gb of memory simply doesn't do msieve BL step well enough at its stock frequency of 1066MHz, ...had to find the stable spot which turned to be at 1040MHz.)

Maybe Jasonp should add to the coded error message that this is a hardware error, to avoid confusion.
_______

P.S. While I was writing, the message was edited with additions. Well. This was about the "corrupt state" condition.
The log continuation (and the next problem) probably stems from the python script trying to restart the job, but not properly.
Note: for large/medium jobs, you should graduate from using off-the-shelf scripts and develop your own, or at least know how to do everything step by step.

Carlo · 2010-08-24, 16:40

Ok, I'll give that a try (yes, it dissappeared because I was editing it, I added the last lines from my LOG)

Just tried changing the memory (it's PC6400 memory) on a Core2 Quad Computer running Vista 32 bit, the
issue is still the same. Is the state now corrupted in a way, that a restart is not possible?

I tried editing the python script to use -ncr instead -nc2 but that did not change anything,
the folder now has

num.dat.chk (170mb)
num.dat.cyc (100mb)
num.dat.mat (0 bytes)
spairs.save.gz (3.8gb)

I did back up everything right before it found 100% of the needed relations, would it be best to restart from there?

Regards,
-Carlo
Thanks!

Batalov · 2010-08-24, 16:57

Yeah, it basically boils down to the lines that the script prints with small variations to the -nc<something> parameter (the script knows the names of the working files it created; we don't).

You probably want to
1. get the very latest msieve from SVN, build it
2. run
msieve -s num.dat -l num.log -i num.ini -nf num.fb -t 4 -ncr

P.S. The message
Tue Aug 24 08:45:55 2010 read 0 relations
doesn't look too good. Check if num.dat still exists. I hope that script didn't delete it? :surprised You hopefully have a backup? (You should. Always.)

Batalov · 2010-08-24, 17:04

Ok, I am switching off to real work (end of coffee break).

last words (as I've read the latest message):
1. recreate num.dat
2. downclock the memory (maybe CPU, too) to be safe
3. start again from -nc2
4. after corrupt state messages, don't despair, restart with -ncr; it will continue fine with high probability
5. repeat #4 (and continue easing off your computer) as necessary
6. when .dep file will exist (i.e. algebra will succeed), run -nc3
7. PROFIT!!

jasonp · 2010-08-24, 17:17

Quote:

Originally Posted by Batalov

Maybe Jasonp should add to the coded error message that this is a hardware error, to avoid confusion.

Actually, when this error appears it could be for many reasons, chief of which is a hardware problem. The last time you got this error it was a programmer problem, i.e. my fault :)

2010-08-24, 15:35	#17
Carlo Jul 2010 8₁₆ Posts	That did help (did the BCDEDIT /set increaseuserva 3072 and rebooted), now I get: Found 57664076 relations, 101.1% of the estimated minimum (57029236). -> msieve -s num.dat -l num.log -i num.ini -nf num.fb -t 4 -nc1 read 10M relations read 20M relations read 30M relations read 40M relations read 50M relations -> Running matrix solving step ... -> msieve -s num.dat -l num.log -i num.ini -nf num.fb -t 4 -nc2 linear algebra completed 69843 of 4310723 dimensions (1.6%, ETA 60h23m) error: corrupt state, please restart from checkpoint Return value -1. Terminating... siever terminated What would cause this error? Last few lines of log: Tue Aug 24 08:45:39 2010 Msieve v. 1.46 Tue Aug 24 08:45:39 2010 random seeds: 6922a894 549e0ca8 Tue Aug 24 08:45:39 2010 factoring 8224973201493734039216932833462996815932154044113673505636726252834676063695616729466358005376619469264571014058650019804568205019013693877262015651491183 (154 digits) Tue Aug 24 08:45:41 2010 searching for 15-digit factors Tue Aug 24 08:45:42 2010 commencing number field sieve (154-digit input) Tue Aug 24 08:45:42 2010 R0: -885345494398234005630543262076 Tue Aug 24 08:45:42 2010 R1: 249843090087716533 Tue Aug 24 08:45:42 2010 A0: -8294699117965540061436793792349975162625 Tue Aug 24 08:45:42 2010 A1: 135954959572494343196175400442215 Tue Aug 24 08:45:42 2010 A2: -93888007586351928671641113 Tue Aug 24 08:45:42 2010 A3: -10081478007408346455 Tue Aug 24 08:45:42 2010 A4: 2318577493178 Tue Aug 24 08:45:42 2010 A5: 15120 Tue Aug 24 08:45:42 2010 skew 16624150.69, size 4.492e-015, alpha -7.981, combined = 2.619e-012 rroots = 3 Tue Aug 24 08:45:42 2010 Tue Aug 24 08:45:42 2010 commencing linear algebra Tue Aug 24 08:45:45 2010 read 4310723 cycles Tue Aug 24 08:45:55 2010 cycles contain 12066493 unique relations Tue Aug 24 08:45:55 2010 read 0 relations Tue Aug 24 08:45:55 2010 error: cannot locate relation 57048268 Last fiddled with by Carlo on 2010-08-24 at 16:28

2010-08-24, 16:32	#18
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 2·47·101 Posts	The usual reason is the stressed memory (or memory controller, or the bus - the whole subsystem). msieve stresses the memory subsystem as hard as (and possibly harder than) the Prime95 torture. Especially during hot summer, the solution may be to downclock the memory (then if it doesn't help, CPU and CPU IMC) a bit. (E.g. my 8Gb of memory simply doesn't do msieve BL step well enough at its stock frequency of 1066MHz, ...had to find the stable spot which turned to be at 1040MHz.) Maybe Jasonp should add to the coded error message that this is a hardware error, to avoid confusion. _______ P.S. While I was writing, the message was edited with additions. Well. This was about the "corrupt state" condition. The log continuation (and the next problem) probably stems from the python script trying to restart the job, but not properly. Note: for large/medium jobs, you should graduate from using off-the-shelf scripts and develop your own, or at least know how to do everything step by step. Last fiddled with by Batalov on 2010-08-24 at 16:44

2010-08-24, 16:40	#19
Carlo Jul 2010 2³ Posts	Ok, I'll give that a try (yes, it dissappeared because I was editing it, I added the last lines from my LOG) Just tried changing the memory (it's PC6400 memory) on a Core2 Quad Computer running Vista 32 bit, the issue is still the same. Is the state now corrupted in a way, that a restart is not possible? I tried editing the python script to use -ncr instead -nc2 but that did not change anything, the folder now has num.dat.chk (170mb) num.dat.cyc (100mb) num.dat.mat (0 bytes) spairs.save.gz (3.8gb) I did back up everything right before it found 100% of the needed relations, would it be best to restart from there? Regards, -Carlo Thanks! Last fiddled with by Carlo on 2010-08-24 at 16:51

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve v. 1.47 and 148 digit composite	Unregistered	Information & Answers	8	2010-10-04 07:31
Quick msieve question	alkirah	Msieve	2	2009-12-30 14:00
222-digit SNFS completed with msieve	frmky	Factoring	2	2007-10-01 18:23
Question about cycle counting in MSieve	schickel	Msieve	3	2006-11-25 07:14
MSieve - problem with 106-digit effort	schickel	Msieve	5	2006-08-31 03:19

2010-07-17, 19:17	#14
asp Jul 2010 2 Posts	It's a driver issue. I've already tracked it down. Thanks for the suggestion though.

2010-08-23, 21:19	#15
Carlo Jul 2010 2³ Posts	Sofar it finished the finding the needed 57million relations needed for this 154 digit number. The next step is the -nc1 step, it loads all the relations then starts working but craps out after about an hour. Saying it failed to calloc 231881660 bytes :-) This machine is running Vista32 with 4 GB of RAM. Does any one have an estimate of what the required amount of memory is to make this run? Or do I need to use a 64 bit OS for this to work? -Carlo

2010-08-23, 22:25	#16
jasonp Tribal Bullet Oct 2004 3×1,181 Posts	Postprocessing for a number this size should fit in 1.5 - 2.5GB, but if your memory is fragmented or you need 2GB then you have run out of VM, no matter how much physical RAM you have. Unless you can boot the machine to allow 3GB of VM for processes, then you will need a 64-bit OS. IIRC linux x86 gives processes 3GB of VM by default.

2010-08-24, 16:57	#20
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 22426₈ Posts	Yeah, it basically boils down to the lines that the script prints with small variations to the -nc<something> parameter (the script knows the names of the working files it created; we don't). You probably want to 1. get the very latest msieve from SVN, build it 2. run msieve -s num.dat -l num.log -i num.ini -nf num.fb -t 4 -ncr P.S. The message Tue Aug 24 08:45:55 2010 read 0 relations doesn't look too good. Check if num.dat still exists. I hope that script didn't delete it? :surprised You hopefully have a backup? (You should. Always.)

2010-08-24, 17:04	#21
Batalov "Serge" Mar 2008 Phi(4,2^7658614+1)/2 2516₁₆ Posts	Ok, I am switching off to real work (end of coffee break). last words (as I've read the latest message): 1. recreate num.dat 2. downclock the memory (maybe CPU, too) to be safe 3. start again from -nc2 4. after corrupt state messages, don't despair, restart with -ncr; it will continue fine with high probability 5. repeat #4 (and continue easing off your computer) as necessary 6. when .dep file will exist (i.e. algebra will succeed), run -nc3 7. PROFIT!!