Python Driver for GGNFS and MSIEVE - Page 56

Walter Nissen · 2011-09-14, 13:55

A few days ago I asked how to resume a ggnfs run after a cold
shutdown during sieving.
A possible strategy would be to start from the beginning in a new
directory and ctrl-c to a good shutdown and then copy in the
large files and then twiddle the parameters .
Has anyone succeeded with that ?
Is it possible it could work ?
Is there a better way ?

Walter Nissen · 2011-09-19, 17:13

Maybe there is no general way to resume from a crash during
sieving , but there was a simple fix which resulted in a
successful factorization .

I screwed up my courage and took on a programming language in which
I've never written a program ; and I also read the error messages
very carefully .
Buried in those messages was a reference to read_spq which led
to a syntax-type error .
In the working directory , there were left 2 spq files :

Code:

2011-09-09  23:53                 8 .last_spq0
2011-09-09  23:53                 8 .last_spq1

These are files which I know , from peeking , are updated
continuously during sieving .
A few days ago , I posted here the contents of .last_spq0 :
4505029
My guess about the contents of .last_spq1 was dramatically
reversed when debug revealed all 64 bits were actually 0 .
So I used debug to patch in 4507000 .
This I did in the file in a full copy of the working directory .
( I think making a full copy of the working directory at
the first sign of trouble is a good idea .
Repeat as necessary .
)
With that little patch in place , I reinvoked with the original
command , and a day later , without further incident , I was
rewarded with a p56 and a p71 .
I was able to save the 85 % of the sieving already done .
Was I merely lucky ?

Among the many puzzles found in
http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE
is this :

Code:

  ... last_spq1234 ...
  It is necessary to remove the garbled last entry of the 
  resulting output file,

Does anyone think this could possibly be related to this recent
problem ?

This sub-thread begins at message # 589 .

Walter Nissen · 2011-09-19, 17:21

As previously noted , I've run factMsieve.py for 40 hours on a
cmd.exe box on XP on a 1-core , 2-thread Pentium 4
to factor a c114 using gnfs .
Factoring this recent c127 using fivemack's polynomial which gave
an SNFS difficulty of 153 took , coincidentally , 153 hours .
Isn't that a peculiarly long time relative to the c114 ?

______

That's my question , but I can mutter on .
http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE
can be processed to yield this table :

Code:

  lattice 
  sieving     difficulty
   region    GNFS    SNFS
 4096x2048    110     150  gnfs-lasieve4I12e
 8192x4096  130-140   180  gnfs-lasieve4I13e
16394x8192  155-160        gnfs-lasieve4I14e

From this :
SNFS 153 ~= GNFS 112
So why would a SNFS 153 take so much longer than a GNFS 114 ?
Especially since the SNFS 153 had THREADS_PER_CORE = 2 while the
GNFS 114 had THREADS_PER_CORE = 1 ?
As previously noted in message # 605 , 2 threads was measured to be
better than 1 for sieving .
The SNFS 153 found 6228982 relations ,
the GNFS 114 found 8313428 .
The SNFS 153 run had virtually no interference from other activity
on the machine , just me peeking in occasionally .

axn · 2011-09-19, 18:39

For 25-bit lpbr/a, you'd need something like 2.8M relations, not 6M relations like the script says.

fivemack · 2011-09-20, 14:50

Yes, I'm wondering which siever the script is choosing to use.

Computers are fast, so I'm redoing the whole thing locally (by hand, not using the script); 3.17M relations (Q=2M .. 3.5M) has taken 21 CPU-hours running the 12e siever and not quite produced a factorisation. This is with 64-bit Linux sievers on a 2.4GHz Q6600 while the original poster is using a single-core Pentium 4, but I'm not sure there's a factor seven there.

Andi47 · 2011-09-20, 17:05

Quote:

Originally Posted by fivemack

Yes, I'm wondering which siever the script is choosing to use.

Computers are fast, so I'm redoing the whole thing locally (by hand, not using the script); 3.17M relations (Q=2M .. 3.5M) has taken 21 CPU-hours running the 12e siever and not quite produced a factorisation. This is with 64-bit Linux sievers on a 2.4GHz Q6600 while the original poster is using a single-core Pentium 4, but I'm not sure there's a factor seven there.

64 bit Linux on a Core 2 Quad - I think that's ~3-4 times faster than a P4 on windows - presuming that they are running approx. at equal clock speed

Walter Nissen · 2011-09-20, 17:23

Thanks for the interesting point .
It just occurred to me , could the speed differential result from
the N provided to gnfs-lasieve4I13e being stripped of its
"small factors" ; i.e. ,
( 79^79+80^80 ) / ( 547 * 1381 * 66269726871718249753 )
instead of 79^79+80^80 ; the SNFS difficulty being reported
by factMsieve.py being thus deceptively low ?

Walter Nissen · 2011-09-20, 17:35

Quote:

Originally Posted by fivemack

Yes, I'm wondering which siever the script is choosing to use.

In message # 593 you suggested :
... sieve with 13e from q=2e6 to q=4e6 ...
I took that to mean gnfs-lasieve4I13e , and in fact that is what
factMsieve.py chose .
The range turned out to be 2e6 to 5100000 .
Thanks for your help .

Walter Nissen · 2011-09-20, 17:40

Quote:

Originally Posted by Andi47

64 bit Linux on a Core 2 Quad - I think that's ~3-4 times faster than a P4 on windows - presuming that they are running approx. at equal clock speed

The P4 is running at 3.33 GHz vs. the 2.4GHz Q6600 ,
but I'm not sure those rates are comparable .

Walter Nissen · 2011-09-20, 18:11

Quote:

Originally Posted by axn

For 25-bit lpbr/a, you'd need something like 2.8M relations, not 6M relations like the script says.

Thanks for the interesting point .
I simply accepted the parameters suggested by fivemack .
Then I found :
http://homepage2.nifty.com/m_kamada/math/graphs.htm
So I annotated a copy of the .poly file from fivemack :

Code:

n: 3545747554330427459757047726394524919008341604708116681613612166055111769302473489038103286321807499348516053117524995279478089
skew: 33
c5: 1
c0: 38950081
Y0: 2814749767106560000000000000000
Y1: -29134419507545592909032289199
type: snfs                                          Kamada
lpbr: 25                                                27
lpba: 25
mfbr: 54                                                50
mfba: 54
alambda: 2.6
rlambda: 2.6                                             2.4
alim: 4000000
rlim: 4000000                                      2480000
rels                                               6130000
equiv rels for gnfs   112  vs. 153 snfs
CPU  hrs                                                13

where I read Kamada's graphs to build the column to the right .
Being a novice with NFS , I can see there might be discrepancies
of significance unknown to me .
I would value comments .

axn · 2011-09-20, 18:35

Quote:

Originally Posted by Walter Nissen

Code:

lpbr: 25
lpba: 25
mfbr: 54
mfba: 54

The combination of 25/54 doesn't make sense. It should be 25/50. Apart from that, there's only the "relations needed" figure that is off. And maybe the siever choice. Everything else can be accounted by the CPU speed difference (P4s kinda suck at this).

PS:- The parameters found from Kamada's site will also work (though the actual relations needed figure maybe a touch higher).

2011-09-14, 13:55	#606
Walter Nissen Nov 2006 Terra 2×3×13 Posts	resuming after a cold shutdown during sieving A few days ago I asked how to resume a ggnfs run after a cold shutdown during sieving. A possible strategy would be to start from the beginning in a new directory and ctrl-c to a good shutdown and then copy in the large files and then twiddle the parameters . Has anyone succeeded with that ? Is it possible it could work ? Is there a better way ?

2011-09-19, 17:13	#607
Walter Nissen Nov 2006 Terra 2×3×13 Posts	a simple fix and success Maybe there is no general way to resume from a crash during sieving , but there was a simple fix which resulted in a successful factorization . I screwed up my courage and took on a programming language in which I've never written a program ; and I also read the error messages very carefully . Buried in those messages was a reference to read_spq which led to a syntax-type error . In the working directory , there were left 2 spq files : Code: 2011-09-09 23:53 8 .last_spq0 2011-09-09 23:53 8 .last_spq1 These are files which I know , from peeking , are updated continuously during sieving . A few days ago , I posted here the contents of .last_spq0 : 4505029 My guess about the contents of .last_spq1 was dramatically reversed when debug revealed all 64 bits were actually 0 . So I used debug to patch in 4507000 . This I did in the file in a full copy of the working directory . ( I think making a full copy of the working directory at the first sign of trouble is a good idea . Repeat as necessary . ) With that little patch in place , I reinvoked with the original command , and a day later , without further incident , I was rewarded with a p56 and a p71 . I was able to save the 85 % of the sieving already done . Was I merely lucky ? Among the many puzzles found in http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE is this : Code: ... last_spq1234 ... It is necessary to remove the garbled last entry of the resulting output file, Does anyone think this could possibly be related to this recent problem ? This sub-thread begins at message # 589 .

2011-09-19, 17:21	#608
Walter Nissen Nov 2006 Terra 2·3·13 Posts	SNFS 153 took a long time As previously noted , I've run factMsieve.py for 40 hours on a cmd.exe box on XP on a 1-core , 2-thread Pentium 4 to factor a c114 using gnfs . Factoring this recent c127 using fivemack's polynomial which gave an SNFS difficulty of 153 took , coincidentally , 153 hours . Isn't that a peculiarly long time relative to the c114 ? ______ That's my question , but I can mutter on . http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE can be processed to yield this table : Code: lattice sieving difficulty region GNFS SNFS 4096x2048 110 150 gnfs-lasieve4I12e 8192x4096 130-140 180 gnfs-lasieve4I13e 16394x8192 155-160 gnfs-lasieve4I14e From this : SNFS 153 ~= GNFS 112 So why would a SNFS 153 take so much longer than a GNFS 114 ? Especially since the SNFS 153 had THREADS_PER_CORE = 2 while the GNFS 114 had THREADS_PER_CORE = 1 ? As previously noted in message # 605 , 2 threads was measured to be better than 1 for sieving . The SNFS 153 found 6228982 relations , the GNFS 114 found 8313428 . The SNFS 153 run had virtually no interference from other activity on the machine , just me peeking in occasionally .

2011-09-20, 14:50	#610
fivemack (loop (#_fork)) Feb 2006 Cambridge, England 1916₁₆ Posts	Yes, I'm wondering which siever the script is choosing to use. Computers are fast, so I'm redoing the whole thing locally (by hand, not using the script); 3.17M relations (Q=2M .. 3.5M) has taken 21 CPU-hours running the 12e siever and not quite produced a factorisation. This is with 64-bit Linux sievers on a 2.4GHz Q6600 while the original poster is using a single-core Pentium 4, but I'm not sure there's a factor seven there. Last fiddled with by fivemack on 2011-09-20 at 14:51

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve & ggnfs on MacOS	xilman	Msieve	8	2017-05-20 00:12
Factorizing with MSIEVE, GGNFS & Factmsieve.py	Romuald	Msieve	24	2015-11-09 20:16
Infinite loop for ggnfs or msieve	Greebley	Aliquot Sequences	4	2013-02-06 19:28
Error running GGNFS+msieve+factmsieve.py	D. B. Staple	Factoring	6	2011-06-12 22:23
A new driver? (or type of driver?)	10metreh	Aliquot Sequences	3	2010-02-15 15:57

2011-09-19, 18:39	#609
axn Jun 2003 13D4₁₆ Posts	For 25-bit lpbr/a, you'd need something like 2.8M relations, not 6M relations like the script says.

2011-09-20, 17:23	#612
Walter Nissen Nov 2006 Terra 2×3×13 Posts	Thanks for the interesting point . It just occurred to me , could the speed differential result from the N provided to gnfs-lasieve4I13e being stripped of its "small factors" ; i.e. , ( 79^79+80^80 ) / ( 547 * 1381 * 66269726871718249753 ) instead of 79^79+80^80 ; the SNFS difficulty being reported by factMsieve.py being thus deceptively low ?