Python Driver for GGNFS and MSIEVE - Page 55

Walter Nissen · 2011-09-04, 13:15

fivemack , that looks extremely authoritative .
Thanks.
I have hundreds or thousands of clues about what you mean ,
but I'm missing a few tutorials and am unsure if I can even ask
good questions.
Is this a switch from ggnfs to something else ?
Is that ( part of ? ) a .job file or a .poly file ?
Is .job.T0 in play here ?
If I guess you mean gnfs-lasieve4I13e.exe , then
http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE
gives lots of clues about how to use lattice sieving , but I see a
sea of unexplained options , and don't feel close to knowing how to
reduce that to a command line or command lines .
Do I use "-f" ? "-c" ?
I think you're talking SNFS and that gnfs-la... makes me worry .
Can I simply insert a file into the directory and factMsieve.py
will figure it out ?

schickel · 2011-09-04, 14:19

Quote:

Originally Posted by Walter Nissen

fivemack , that looks extremely authoritative .
Thanks.

[ snip]

Is that ( part of ? ) a .job file or a .poly file ?
Is .job.T0 in play here ?

That is a poly file. Just put it in your working directory as *.poly and the script will figure things out.

Quote:

If I guess you mean gnfs-lasieve4I13e.exe , then
http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE
gives lots of clues about how to use lattice sieving , but I see a
sea of unexplained options , and don't feel close to knowing how to
reduce that to a command line or command lines .
Do I use "-f" ? "-c" ?

Normally the script makes the choice about the siever used depending on the size of the input composite. You could just fire it up and see which one it reports using. Either *I12e or *I13e will also work, but will probably take longer.....

Quote:

I think you're talking SNFS and that gnfs-la... makes me worry .
Can I simply insert a file into the directory and factMsieve.py
will figure it out ?

Right, this is an SNFS poly, but don't worry, the entire job works the same way.

The only difference between SNFS and GNFS is the poly. An SNFS poly will be much simpler and take much less time than a GNFS poly....but the siever used is the same for both types of job.

EdH · 2011-09-04, 15:25

@ Brian,

In setting up my latest machine (P4 w/ Ubuntu 11.04), I found that this version of Python 3 (Python 3.2 (r32:88445, Mar 25 2011, 19:28:28) [GCC 4.5.2] on linux2) won't run factmsieve.py (0.76). Python 2 (Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) [GCC 4.5.2] on linux2) runs fine.

Code:

../Aliqueit/ggnfs_115976328283548771533536102905431040151193675026997672535779074594594411222276287190480420777/test.fb -v -np
Traceback (most recent call last):
  File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 2033, in <module>
    if not run_msieve_poly(fact_p):
  File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 950, in run_msieve_poly
    ret = run_msieve(ap)
  File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 366, in run_msieve
    ret = run_exe(MSIEVE, args + ap)
  File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 338, in run_exe
    p = subprocess.Popen([ex] + args.split(' '), **al)
  File "/usr/lib/python3.2/subprocess.py", line 736, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.2/subprocess.py", line 1331, in _execute_child
    raise child_exception_type(err_msg)
RuntimeError: Exception occurred in preexec_fn.

My other (Ubuntu 10.04) machines are running Python 3.1 with no troubles.

Is this something I'm missing in the setup or a change between 3.1 and 3.2 that conflicts in the script?

This is not hindering me in any way. I'm simply using Python 2 on that machine, but I thought I'd let you know.

Thanks for all.

warut · 2011-09-05, 02:50

Quote:

Originally Posted by fivemack

79^79+80^80 = (79^4) * (79^15)^5 + (80^16)^5

so:

Code:

n: 3545747554330427459757047726394524919008341604708116681613612166055111769302473489038103286321807499348516053117524995279478089
skew: 33
c5: 1
c0: 38950081
Y0: 2814749767106560000000000000000
Y1: -29134419507545592909032289199
lpbr: 25
lpba: 25
mfbr: 54
mfba: 54
alambda: 2.6
rlambda: 2.6
alim: 4000000
rlim: 4000000

sieve with 13e from q=2e6 to q=4e6, it should take about 24 hours.

Or you can try the following .poly file which is based on

79*(80^80+79^79) = 79*(80^16)^5 + (79^16)^5:

Code:

n: 3545747554330427459757047726394524919008341604708116681613612166055111769302473489038103286321807499348516053117524995279478089
Y0: 2814749767106560000000000000000
Y1: -2301619141096101839813550846721
c0: 1
c1: 0
c2: 0
c3: 0
c4: 0
c5: 79
skew: 0.42
type: snfs

It could be slightly faster.

Brian Gladman · 2011-09-05, 07:29

Thanks for the bug report Ed - unfortunately (or fortunately!) the script runs fine with Python 3.2 on Windows so I cannot debug this (I don't run this code under Linux). If a Linnux guru can figure out what is wrong and how to fix it, I will happily update the script.

EdH · 2011-09-05, 13:40

Quote:

Originally Posted by Brian Gladman

Thanks for the bug report Ed - unfortunately (or fortunately!) the script runs fine with Python 3.2 on Windows so I cannot debug this (I don't run this code under Linux). If a Linnux guru can figure out what is wrong and how to fix it, I will happily update the script.

Thanks Brian,

It's not really hurting me at all. I just switched to 2 for now. Maybe I'll study it a little further later. Since my other machines are all using 3.1 with no hitches, I'm "assuming" it's a 3.1-3.2 issue, but this machine also has a newer Ubuntu, which could be contributing. Maybe I should start by trying to downgrade to Python 3.1 first. Not sure how to do that...

Walter Nissen · 2011-09-05, 21:36

Thanks much for the replies .
I inserted type: snfs into the .poly file kindly provided by
fivemack and followed the "Beginner's Guide" , except I used
THREADS_PER_CORE = 2
So , SNFS_DIFFICULTY is about 153 , relations needed are 6105402
and 13e is busily producing about 6 rels per sec , while
Process Explorer shows 2 threads eating 99 % of the CPU and
only minimal RAM usage .
The first 19:37 yielded 15.0 % so the sieving should be done in
about a week .

warut , I'd love to have another machine to run your .poly file .
But , do you think I should abort and start over ?

How can I calculate the equivalent gfns difficulty ?
Looking beyond this # , how does one calculate the coeffs from
an algebraic expression ?
Like I said , I've missed a few tutorials .
Thanks again .

warut · 2011-09-06, 08:32

You can pause your current work with Ctrl-C and run another .poly in another directory on another window to see if it's worth changing. If not, stop and go back to continue the previous job.

Walter Nissen · 2011-09-07, 00:33

Thanks .
Thinking along that line , I started a trial as you propose , except I
didn't pause the one which is running .
Neither one has written anything in the ggnfs directory , so I'm
hopeful all is well .

For this new poly , the SNFS_DIFFICULTY is about 155 , relations needed
are 6520570 and the default factorization parameters are for the
156-digit level ( as against 153 ) .
I don't know this authoritatively , but that seemed unfavorable enough
to induce me to stop the new one .

By now , the first run is 37% through the ( minimal ) sieving .

Walter Nissen · 2011-09-11, 00:56

Unfortunately , the machine crashed to total black , looks like a thermal
cutout , after completing :

Code:

Found 5162239 relations, 84.6% of the estimated minimum (6105402).

I copied the working dir and reinvoked :

Code:

..\factMsieve.py 79

( it's called 79 , but it's a c127 )
but it quickly died :

Code:

-> gnfs-lasieve4I13e -k -o spairs.out.T0 -v -n0 -r 79.job.T0
-> gnfs-lasieve4I13e -k -o spairs.out.T1 -v -n1 -r 79.job.T1
Traceback (most recent call last):
  File "C:\gg\factMsieve.py", line 2066, in <module>
gnfs-lasieve4I13e: L1_BITS=15, SVN $Revision: 406 $
    run_siever(client_id, num_clients, SV_THREADS, fact_p, lats_p)
  File "C:\gg\factMsieve.py", line 1701, in run_siever
    gnfs-lasieve4I13e: L1_BITS=15, SVN $Revision: 406 $
ret = monitor_sieve_threads()
  File "C:\gg\factMsieve.py", line 1650, in monitor_sieve_threads
    read_spq(fact_p)
  File "C:\gg\factMsieve.py", line 1638, in read_spq
    t = int(chomp(tmp))
ValueError: invalid literal for int() with base 10: ''
siever terminated

2011-09-10  4:51:19.78 C:\gg\79res>

Re-reinvoking failed similarly over in the copied dir .

This is the scene now :

Code:

2011-09-10  2:35:38.46 C:\gg\79> dir /od
 Volume in drive C is SQ003627
 Volume Serial Number is 6C40-871F

 Directory of C:\gg\79

2011-05-03  16:04               133 79.n
2011-09-04  21:47               344 79.poly
2011-09-04  21:49               129 79.ini
2011-09-04  21:49               304 79.fb
2011-09-04  21:49         2,265,496 79.job.T1.afb.0
2011-09-04  21:49         2,265,496 79.job.T0.afb.0
2011-09-09  23:22             1,100 ggnfs.log
2011-09-09  23:22       541,333,555 79.dat
2011-09-09  23:23       275,930,319 spairs.save.gz
2011-09-09  23:23               385 79.job.T1
2011-09-09  23:23                92 79.job.resume
2011-09-09  23:23            16,351 79.log
2011-09-09  23:23               385 79.job.T0
2011-09-09  23:23           995,328 spairs.out.T0
2011-09-09  23:23    <DIR>          ..
2011-09-09  23:23    <DIR>          .
2011-09-09  23:23           970,752 spairs.out.T1
2011-09-09  23:53                 8 .last_spq0
2011-09-09  23:53                 8 .last_spq1
              17 File(s)    823,780,185 bytes
               2 Dir(s)  79,779,442,688 bytes free

2011-09-10  2:35:46.17 C:\gg\79> 

2011-09-10 13:50:45.65 C:\gg\79> type 79.job.T0
n: 35457475543304274597570477263945249190083416047081166816136121660551117693024
73489038103286321807499348516053117524995279478089
c5: 1
c0: 38950081
Y1: -29134419507545592909032289199
Y0: 2814749767106560000000000000000
skew: 33.00
rlim: 4000000
alim: 4000000
lpbr: 25
lpba: 25
mfbr: 54
mfba: 54
rlambda: 2.6
alambda: 2.6
q0: 4500000
qintsize: 50000
#q1:4550000

2011-09-10 13:51:07.82 C:\gg\79> type .last_spq0
4505029

2011-09-10 13:51:42.95 C:\gg\79>

I've read probably an additional hundred messages here without
being able to answer this question :
Can you tell me what must be done to resume this factorization ?

Walter Nissen · 2011-09-11, 01:10

While the machine was frustrated ( and frustrating ) , I looked at
whether I should have varied from NUM_THREADS = 1 with
NUM_THREADS = 2 .

<<< with 2 threads first ( 2 ) jobs ran
307 min. Found 234318 relations, 3.8% of the estimated minimum (6105402).

<<< with 1 thread first job ran
380 min. Found 234315 relations, 3.8% of the estimated minimum (6105402).

I intend upon staying with NUM_THREADS = 2 .

( Details on these runs appear in the previous 15 messages or so in this
thread beginning with #589 .
)

2011-09-05, 07:29	#599
Brian Gladman May 2008 Worcester, United Kingdom 2²×7×19 Posts	Thanks for the bug report Ed - unfortunately (or fortunately!) the script runs fine with Python 3.2 on Windows so I cannot debug this (I don't run this code under Linux). If a Linnux guru can figure out what is wrong and how to fix it, I will happily update the script. Last fiddled with by Brian Gladman on 2011-09-05 at 07:30

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Msieve & ggnfs on MacOS	xilman	Msieve	8	2017-05-20 00:12
Factorizing with MSIEVE, GGNFS & Factmsieve.py	Romuald	Msieve	24	2015-11-09 20:16
Infinite loop for ggnfs or msieve	Greebley	Aliquot Sequences	4	2013-02-06 19:28
Error running GGNFS+msieve+factmsieve.py	D. B. Staple	Factoring	6	2011-06-12 22:23
A new driver? (or type of driver?)	10metreh	Aliquot Sequences	3	2010-02-15 15:57

2011-09-04, 13:15	#595
Walter Nissen Nov 2006 Terra 1001110₂ Posts	fivemack , that looks extremely authoritative . Thanks. I have hundreds or thousands of clues about what you mean , but I'm missing a few tutorials and am unsure if I can even ask good questions. Is this a switch from ggnfs to something else ? Is that ( part of ? ) a .job file or a .poly file ? Is .job.T0 in play here ? If I guess you mean gnfs-lasieve4I13e.exe , then http://stuff.mit.edu/afs/sipb/projec...NSTALL.and.USE gives lots of clues about how to use lattice sieving , but I see a sea of unexplained options , and don't feel close to knowing how to reduce that to a command line or command lines . Do I use "-f" ? "-c" ? I think you're talking SNFS and that gnfs-la... makes me worry . Can I simply insert a file into the directory and factMsieve.py will figure it out ?

2011-09-05, 21:36	#601
Walter Nissen Nov 2006 Terra 2×3×13 Posts	Thanks much for the replies . I inserted type: snfs into the .poly file kindly provided by fivemack and followed the "Beginner's Guide" , except I used THREADS_PER_CORE = 2 So , SNFS_DIFFICULTY is about 153 , relations needed are 6105402 and 13e is busily producing about 6 rels per sec , while Process Explorer shows 2 threads eating 99 % of the CPU and only minimal RAM usage . The first 19:37 yielded 15.0 % so the sieving should be done in about a week . warut , I'd love to have another machine to run your .poly file . But , do you think I should abort and start over ? How can I calculate the equivalent gfns difficulty ? Looking beyond this # , how does one calculate the coeffs from an algebraic expression ? Like I said , I've missed a few tutorials . Thanks again .

2011-09-06, 08:32	#602
warut Dec 2009 131₈ Posts	You can pause your current work with Ctrl-C and run another .poly in another directory on another window to see if it's worth changing. If not, stop and go back to continue the previous job.

2011-09-07, 00:33	#603
Walter Nissen Nov 2006 Terra 2·3·13 Posts	Thanks . Thinking along that line , I started a trial as you propose , except I didn't pause the one which is running . Neither one has written anything in the ggnfs directory , so I'm hopeful all is well . For this new poly , the SNFS_DIFFICULTY is about 155 , relations needed are 6520570 and the default factorization parameters are for the 156-digit level ( as against 153 ) . I don't know this authoritatively , but that seemed unfavorable enough to induce me to stop the new one . By now , the first run is 37% through the ( minimal ) sieving .

2011-09-11, 01:10	#605
Walter Nissen Nov 2006 Terra 1001110₂ Posts	While the machine was frustrated ( and frustrating ) , I looked at whether I should have varied from NUM_THREADS = 1 with NUM_THREADS = 2 . <<< with 2 threads first ( 2 ) jobs ran 307 min. Found 234318 relations, 3.8% of the estimated minimum (6105402). <<< with 1 thread first job ran 380 min. Found 234315 relations, 3.8% of the estimated minimum (6105402). I intend upon staying with NUM_THREADS = 2 . ( Details on these runs appear in the previous 15 messages or so in this thread beginning with #589 . )