Python Driver for GGNFS and MSIEVE - Page 52

em99010pepe · 2011-04-13, 22:38

Quote:

Originally Posted by em99010pepe

For type:SNFS (on example.poly) how do I change the estimated minimum relations in function of the composite size? I want to increase the default settings.

Managed to increase estimated minimum relations by editing the factmsieve.py code. For a c140 (difficulty of 180) the script estimated minimum of relations of 15.3M but 21.4M were used.

em99010pepe · 2011-04-14, 06:35

Quote:

Originally Posted by em99010pepe

Managed to increase estimated minimum relations by editing the factmsieve.py code. For a c140 (difficulty of 180) the script estimated minimum of relations of 15.3M but 21.4M were used.

After reading all the posts in this thread I realized that minrels.txt is what I wanted.

em99010pepe · 2011-04-14, 22:11

I'm getting the following error

Code:

C:\ggnfs\example>factmsieve.py example
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.    Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 4 Cores with 1 hyper-thread per Core
-> Working with NAME = example
Traceback (most recent call last):
  File "C:\ggnfs\example\factmsieve.py", line 2049, in <module>
    read_parameters(fact_p, poly_p, lats_p)
  File "C:\ggnfs\example\factmsieve.py", line 1229, in read_parameters
    if CHECK_POLY and polyval % fact_p['n'] != 0:
TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'

C:\ggnfs\example>

for poly

Code:

n:
12380811774991271385033404268095594747324662220344347005125004853013567023344471440006549059290900843911893550448370364192024347613653243284371963296623271626223702670829613074132213257
Y0: -15286700631942576193765185769276826401
Y1: 1
c0: 1
c1: 0
c2: 0
c3: 0
c4: 0
c5: 10878
skew: 0.212
type: snfs

WraithX · 2011-04-14, 23:27

Quote:

Originally Posted by em99010pepe

I'm getting the following error

Code:

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'

for poly

Code:

n:
12380811774991271385033404268095594747324662220344347005125004853013567023344471440006549059290900843911893550448370364192024347613653243284371963296623271626223702670829613074132213257
Y0: -15286700631942576193765185769276826401
Y1: 1
c0: 1
c1: 0
c2: 0
c3: 0
c4: 0
c5: 10878
skew: 0.212
type: snfs

If the number 123808... is actually on a different line than n:, that is the problem. You should change it so that your poly file looks like:

Code:

n: 12380811774991271385033404268095594747324662220344347005125004853013567023344471440006549059290900843911893550448370364192024347613653243284371963296623271626223702670829613074132213257
Y0: -15286700631942576193765185769276826401
Y1: 1
c0: 1
c1: 0
c2: 0
c3: 0
c4: 0
c5: 10878
skew: 0.212
type: snfs

And all should be well...

em99010pepe · 2011-04-22, 09:35

Thank you WraithX, that was the issue.
Meanwhile I had two crashes running factmsieve.py. The first one I didn't manage to find the cause but on the second I was near the machine when it happened. Basically the script crashed while doing the algebra linear phase on a small number (2-3 mins). The ETA went up to 600 % and then the client (msieve) crashed. I then had to manually run msieve outside the py script to finish the post-processing phase.

PS: I can't remember if both crashes were in the same machine but the second one occurred under Win 7 64 bit OS.

Walter Nissen · 2011-04-25, 19:02

Hi , all ,

Looking around quite a bit I haven't found much about
multithreading of factmsieve and ecm .
Seems processors can have multiple cores and cores can have
multiple threads .
Based on CPU-Z of CPUID and also "Process Explorer" from
sysinternals , seems I am using a Pentium 4 with
1 processor , 1 core and 2 threads .

Very preliminarily , it seems that factmsieve , and thanks to
everyone who's made this wonderful facility work , benefits a
lot from multithreading .
Starting up factmsieve.py in each of 2 cmd.exe DOS boxes under
XP , and working on the smallest composites , i.e. , cnnn's , in
http://upforthecount.com/math/nnp.html
( nnp.html is still unreleased and only in very preliminary form )
factoring the 2 c109's simultaneously seemed to go much faster
than suggested by factoring the c114 and the c105 , each standing
alone .
How much ecm should precede factmsieve ?
I haven't seemed to be able to apply much from Silverman &
Wagstaff's
"A practical analysis of the elliptic curve factoring algorithm"
http://www.ams.org/journals/mcom/199...78-7/home.html
Seems like it might take a lot of work to refit it to the
currently available software and hardware .
( Probably not as much as writing it originally 8-)
( I would say that trial division up to 10^7 is very fast . )

ecm , on the other hand , doesn't seem to benefit much at all from
multithreading , except that if one run completes , the other
continues .
Should I be using cygwin or Msys ?
Would it be optimal to install and boot from Ubuntu ?
A suspect some of you have already tread these paths ,
I hope you can point me to commentary on other people's experience
with factmsieve.py and ecm , especially multithreading .
Details follow .

Thanks ,

Walter

---

32bit x86 SVN 413 4.0 MB ggnfs-svn413-win32-p4.zip 03/20/2011

32bit Intel 6.3 (SVN 1566 with MPIR 2.3.0) 386 KB ecm63_svn1566_win32_intel.zip 03/20/2011

32bit Intel 1.48 (with MPIR 2.2.1/GMP-ECM 6.3) 527 KB msieve148_win32.zip 01/08/2011

The machine is a Pentium 4 3.33 GHz with MMX , SSE1,2,3 ,
running XP Home SP3 , with 1.5 GiB of RAM and over 60 GiB of HD .

---

I haven't fiddled with priorities of the threads.
nnp.html contains prime factors of
np ( n ) = n^n + (n+1)^(n+1) .
In the following table , the first line "68 114" refers to the
line in nnp.html which reads :
68 5 * 2270532006439 * c114
which means , more or less , that
np ( 68 ) = 5 * 2270532006439 * p * q
In the following table , the first line "68 114" says that
factmsieve took 41 hours to split the c114 into p and q , now
seen to be a p52 and a p62 .
You can ignore the second column and probably the first column .

---

run # is order of execution initialization

Code:

np ( )     run #

                   total hrs 

       dir    # digits  

  68   114   1   114   41     ran alone
  71   105   2   105   18     ran alone
  72   72    3   109   25                           this is the larger c109
  76   76    4   109   36     did run #4 run at a priority lower than run #3 ?
  77   104   5   104   15
  79   127       127          now in ecm
  78             129          factored by ecm
  74             134          now in ecm
  93             142
  87             145
  83             147

warut · 2011-04-27, 22:03

While the Python script was running to factor a number, Windows Update automatically rebooted my computer. When I woke up and found that, I ran the script once more. It ran for a few seconds and terminated with this message:

Code:

 
C:\ggnfs\example>..\factMsieve.py example
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.    Version 0.72 (Python 2.6 or later) 2nd June 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Using 4 threads
-> Working with NAME = example
-> SNFS_DIFFICULTY is about 180.
-> Selected default factorization parameters for 180 digit level.
-> Selected lattice siever: gnfs-lasieve4I13e
-> Creating param file to detect parameter changes...
-> Running setup ...
-> Estimated minimum relations needed: 1.48398e+07
-> cleaning up before a restart
-> Running lattice siever ...
-> entering sieving loop
-> Running matrix solving step ...
-> msieve -s example\example.dat -l example\example.log -i example\example.ini -
nf example\example.fb -t 4 -nc2
Return value -1. Terminating...
siever terminated

It seems like Windows Update rebooted my machine in the middle of the linear algebra step. How can I continue the job without starting everything all over again?

jrk · 2011-04-28, 06:20

The way to restart linear algebra is to use -ncr instead of -nc2.

10metreh · 2011-04-28, 06:55

The "interrupting linear algebra wipes the relations file" bug may have occurred here too, although this doesn't seem to be related to the error.

warut · 2011-04-28, 07:36

Thanks. After I started the command:

Code:

msieve -s example\example.dat -l example\example.log -i example\example.ini -nf example\example.fb -t 4 -ncr

msieve ran for a second and then terminated, i.e., no success here. Here are the files I currently have:

Code:

C:\ggnfs\example>dir
 Volume in drive C has no label.
 Volume Serial Number is A0FD-98CE
 Directory of C:\ggnfs\example
04/28/2011  02:11 PM    <DIR>          .
04/28/2011  02:11 PM    <DIR>          ..
04/28/2011  01:22 AM                 8 .last_spq0
04/28/2011  01:22 AM                 8 .last_spq1
04/28/2011  01:22 AM                 8 .last_spq2
04/28/2011  01:22 AM                 8 .last_spq3
04/28/2011  02:11 PM               184 example.dat
04/28/2011  02:32 AM        63,882,056 example.dat.chk
04/28/2011  01:32 AM        44,791,704 example.dat.cyc
04/28/2011  02:10 PM                 0 example.dat.lp
04/28/2011  02:11 PM                 0 example.dat.mat
04/28/2011  02:11 PM               370 example.fb
04/28/2011  02:11 PM               182 example.ini
04/28/2011  02:14 PM           129,411 example.log
04/13/2011  05:34 PM               342 example.poly
04/28/2011  01:22 AM             4,956 ggnfs.log
04/28/2011  01:22 AM     1,062,270,585 spairs.save.gz
              15 File(s)  1,171,079,822 bytes
               2 Dir(s)  602,717,446,144 bytes free

Is there any chance that I can avoid sieving all over again?

Mini-Geek · 2011-04-28, 12:00

Quote:

Originally Posted by warut

Thanks. After I started the command:

Code:

msieve -s example\example.dat -l example\example.log -i example\example.ini -nf example\example.fb -t 4 -ncr

msieve ran for a second and then terminated, i.e., no success here. Here are the files I currently have:

Code:

04/28/2011  02:11 PM               184 example.dat
...
04/28/2011  01:22 AM     1,062,270,585 spairs.save.gz

Is there any chance that I can avoid sieving all over again?

Is spairs.save.gz a compressed copy of the pairs? If so:
You definitely don't have to sieve all over again. I'd recommend making a copy of all existing files, then unzip the relations to example.dat and run the full post-processing again (you can do this with one command, -nc, instead of doing separate -nc1, -nc2, and -nc3). If you run the command yourself, I recommend you add the -v option so that you can see what's happening in the console window.

2011-04-22, 09:35	#566
em99010pepe Sep 2004 2830₁₀ Posts	Thank you WraithX, that was the issue. Meanwhile I had two crashes running factmsieve.py. The first one I didn't manage to find the cause but on the second I was near the machine when it happened. Basically the script crashed while doing the algebra linear phase on a small number (2-3 mins). The ETA went up to 600 % and then the client (msieve) crashed. I then had to manually run msieve outside the py script to finish the post-processing phase. PS: I can't remember if both crashes were in the same machine but the second one occurred under Win 7 64 bit OS. Last fiddled with by em99010pepe on 2011-04-22 at 09:37

2011-04-25, 19:02	#567
Walter Nissen Nov 2006 Terra 78₁₀ Posts	multithreading of factmsieve , ggnfs , and ecm Hi , all , Looking around quite a bit I haven't found much about multithreading of factmsieve and ecm . Seems processors can have multiple cores and cores can have multiple threads . Based on CPU-Z of CPUID and also "Process Explorer" from sysinternals , seems I am using a Pentium 4 with 1 processor , 1 core and 2 threads . Very preliminarily , it seems that factmsieve , and thanks to everyone who's made this wonderful facility work , benefits a lot from multithreading . Starting up factmsieve.py in each of 2 cmd.exe DOS boxes under XP , and working on the smallest composites , i.e. , cnnn's , in http://upforthecount.com/math/nnp.html ( nnp.html is still unreleased and only in very preliminary form ) factoring the 2 c109's simultaneously seemed to go much faster than suggested by factoring the c114 and the c105 , each standing alone . How much ecm should precede factmsieve ? I haven't seemed to be able to apply much from Silverman & Wagstaff's "A practical analysis of the elliptic curve factoring algorithm" http://www.ams.org/journals/mcom/199...78-7/home.html Seems like it might take a lot of work to refit it to the currently available software and hardware . ( Probably not as much as writing it originally 8-) ( I would say that trial division up to 10^7 is very fast . ) ecm , on the other hand , doesn't seem to benefit much at all from multithreading , except that if one run completes , the other continues . Should I be using cygwin or Msys ? Would it be optimal to install and boot from Ubuntu ? A suspect some of you have already tread these paths , I hope you can point me to commentary on other people's experience with factmsieve.py and ecm , especially multithreading . Details follow . Thanks , Walter --- 32bit x86 SVN 413 4.0 MB ggnfs-svn413-win32-p4.zip 03/20/2011 32bit Intel 6.3 (SVN 1566 with MPIR 2.3.0) 386 KB ecm63_svn1566_win32_intel.zip 03/20/2011 32bit Intel 1.48 (with MPIR 2.2.1/GMP-ECM 6.3) 527 KB msieve148_win32.zip 01/08/2011 The machine is a Pentium 4 3.33 GHz with MMX , SSE1,2,3 , running XP Home SP3 , with 1.5 GiB of RAM and over 60 GiB of HD . --- I haven't fiddled with priorities of the threads. nnp.html contains prime factors of np ( n ) = n^n + (n+1)^(n+1) . In the following table , the first line "68 114" refers to the line in nnp.html which reads : 68 5 * 2270532006439 * c114 which means , more or less , that np ( 68 ) = 5 * 2270532006439 * p * q In the following table , the first line "68 114" says that factmsieve took 41 hours to split the c114 into p and q , now seen to be a p52 and a p62 . You can ignore the second column and probably the first column . --- run # is order of execution initialization Code: np ( ) run # total hrs dir # digits 68 114 1 114 41 ran alone 71 105 2 105 18 ran alone 72 72 3 109 25 this is the larger c109 76 76 4 109 36 did run #4 run at a priority lower than run #3 ? 77 104 5 104 15 79 127 127 now in ecm 78 129 factored by ecm 74 134 now in ecm 93 142 87 145 83 147

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve & ggnfs on MacOS	xilman	Msieve	8	2017-05-20 00:12
Factorizing with MSIEVE, GGNFS & Factmsieve.py	Romuald	Msieve	24	2015-11-09 20:16
Infinite loop for ggnfs or msieve	Greebley	Aliquot Sequences	4	2013-02-06 19:28
Error running GGNFS+msieve+factmsieve.py	D. B. Staple	Factoring	6	2011-06-12 22:23
A new driver? (or type of driver?)	10metreh	Aliquot Sequences	3	2010-02-15 15:57

2011-04-28, 06:20	#569
jrk May 2008 3·5·73 Posts	The way to restart linear algebra is to use -ncr instead of -nc2.

2011-04-28, 06:55	#570
10metreh Nov 2008 2·3³·43 Posts	The "interrupting linear algebra wipes the relations file" bug may have occurred here too, although this doesn't seem to be related to the error.