mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Msieve (https://www.mersenneforum.org/forumdisplay.php?f=83)
-   -   Python Driver for GGNFS and MSIEVE (https://www.mersenneforum.org/showthread.php?t=12981)

Walter Nissen 2011-09-04 13:15

fivemack , that looks extremely authoritative .
Thanks.
I have hundreds or thousands of clues about what you mean ,
but I'm missing a few tutorials and am unsure if I can even ask
good questions.
Is this a switch from ggnfs to something else ?
Is that ( part of ? ) a .job file or a .poly file ?
Is .job.T0 in play here ?
If I guess you mean gnfs-lasieve4I13e.exe , then
[url]http://stuff.mit.edu/afs/sipb/project/pari-gp/ggnfs/Linux/src/lasieve4/INSTALL.and.USE[/url]
gives lots of clues about how to use lattice sieving , but I see a
sea of unexplained options , and don't feel close to knowing how to
reduce that to a command line or command lines .
Do I use "-f" ? "-c" ?
I think you're talking SNFS and that gnfs-la... makes me worry .
Can I simply insert a file into the directory and factMsieve.py
will figure it out ?

schickel 2011-09-04 14:19

[QUOTE=Walter Nissen;270803]fivemack , that looks extremely authoritative .
Thanks.

[ snip]

Is that ( part of ? ) a .job file or a .poly file ?
Is .job.T0 in play here ?[/quote]That is a poly file. Just put it in your working directory as *.poly and the script will figure things out.[quote]If I guess you mean gnfs-lasieve4I13e.exe , then
[url]http://stuff.mit.edu/afs/sipb/project/pari-gp/ggnfs/Linux/src/lasieve4/INSTALL.and.USE[/url]
gives lots of clues about how to use lattice sieving , but I see a
sea of unexplained options , and don't feel close to knowing how to
reduce that to a command line or command lines .
Do I use "-f" ? "-c" ?[/quote]Normally the script makes the choice about the siever used depending on the size of the input composite. You could just fire it up and see which one it reports using. Either *I12e or *I13e will also work, but will probably take longer.....[quote]I think you're talking SNFS and that gnfs-la... makes me worry .
Can I simply insert a file into the directory and factMsieve.py
will figure it out ?[/QUOTE]Right, this is an SNFS poly, but don't worry, the entire job works the same way.

The only difference between SNFS and GNFS is the poly. An SNFS poly will be much simpler and take much less time than a GNFS poly....but the siever used is the same for both types of job.

EdH 2011-09-04 15:25

@ Brian,

In setting up my latest machine (P4 w/ Ubuntu 11.04), I found that this version of Python 3 ([I]Python 3.2 (r32:88445, Mar 25 2011, 19:28:28) [GCC 4.5.2] on linux2[/I]) won't run factmsieve.py (0.76). Python 2 ([I]Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) [GCC 4.5.2] on linux2[/I]) runs fine.
[code]
../Aliqueit/ggnfs_115976328283548771533536102905431040151193675026997672535779074594594411222276287190480420777/test.fb -v -np
Traceback (most recent call last):
File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 2033, in <module>
if not run_msieve_poly(fact_p):
File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 950, in run_msieve_poly
ret = run_msieve(ap)
File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 366, in run_msieve
ret = run_exe(MSIEVE, args + ap)
File "/home/math09/Mathwork/Aliqueit/factmsieve.py", line 338, in run_exe
p = subprocess.Popen([ex] + args.split(' '), **al)
File "/usr/lib/python3.2/subprocess.py", line 736, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.2/subprocess.py", line 1331, in _execute_child
raise child_exception_type(err_msg)
RuntimeError: Exception occurred in preexec_fn.
[/code]My other (Ubuntu 10.04) machines are running Python 3.1 with no troubles.

Is this something I'm missing in the setup or a change between 3.1 and 3.2 that conflicts in the script?

This is not hindering me in any way. I'm simply using Python 2 on that machine, but I thought I'd let you know.

Thanks for all.

warut 2011-09-05 02:50

[QUOTE=fivemack;270773]79^79+80^80 = (79^4) * (79^15)^5 + (80^16)^5

so:
[code]
n: 3545747554330427459757047726394524919008341604708116681613612166055111769302473489038103286321807499348516053117524995279478089
skew: 33
c5: 1
c0: 38950081
Y0: 2814749767106560000000000000000
Y1: -29134419507545592909032289199
lpbr: 25
lpba: 25
mfbr: 54
mfba: 54
alambda: 2.6
rlambda: 2.6
alim: 4000000
rlim: 4000000
[/code]

sieve with 13e from q=2e6 to q=4e6, it should take about 24 hours.[/QUOTE]
Or you can try the following .poly file which is based on

79*(80^80+79^79) = 79*(80^16)^5 + (79^16)^5:
[CODE]n: 3545747554330427459757047726394524919008341604708116681613612166055111769302473489038103286321807499348516053117524995279478089
Y0: 2814749767106560000000000000000
Y1: -2301619141096101839813550846721
c0: 1
c1: 0
c2: 0
c3: 0
c4: 0
c5: 79
skew: 0.42
type: snfs[/CODE]
It could be slightly faster.

Brian Gladman 2011-09-05 07:29

Thanks for the bug report Ed - unfortunately (or fortunately!) the script runs fine with Python 3.2 on Windows so I cannot debug this (I don't run this code under Linux). If a Linnux guru can figure out what is wrong and how to fix it, I will happily update the script.

EdH 2011-09-05 13:40

[QUOTE=Brian Gladman;270871]Thanks for the bug report Ed - unfortunately (or fortunately!) the script runs fine with Python 3.2 on Windows so I cannot debug this (I don't run this code under Linux). If a Linnux guru can figure out what is wrong and how to fix it, I will happily update the script.[/QUOTE]
Thanks Brian,

It's not really hurting me at all. I just switched to 2 for now. Maybe I'll study it a little further later. Since my other machines are all using 3.1 with no hitches, I'm "assuming" it's a 3.1-3.2 issue, but this machine also has a newer Ubuntu, which could be contributing. Maybe I should start by trying to downgrade to Python 3.1 first. Not sure how to do that...

Walter Nissen 2011-09-05 21:36

Thanks much for the replies .
I inserted type: snfs into the .poly file kindly provided by
fivemack and followed the "Beginner's Guide" , except I used
THREADS_PER_CORE = 2
So , SNFS_DIFFICULTY is about 153 , relations needed are 6105402
and 13e is busily producing about 6 rels per sec , while
Process Explorer shows 2 threads eating 99 % of the CPU and
only minimal RAM usage .
The first 19:37 yielded 15.0 % so the sieving should be done in
about a week .

warut , I'd love to have another machine to run your .poly file .
But , do you think I should abort and start over ?

How can I calculate the equivalent gfns difficulty ?
Looking beyond this # , how does one calculate the coeffs from
an algebraic expression ?
Like I said , I've missed a few tutorials .
Thanks again .

warut 2011-09-06 08:32

You can pause your current work with Ctrl-C and run another .poly in another directory on another window to see if it's worth changing. If not, stop and go back to continue the previous job.

Walter Nissen 2011-09-07 00:33

Thanks .
Thinking along that line , I started a trial as you propose , except I
didn't pause the one which is running .
Neither one has written anything in the ggnfs directory , so I'm
hopeful all is well .

For this new poly , the SNFS_DIFFICULTY is about 155 , relations needed
are 6520570 and the default factorization parameters are for the
156-digit level ( as against 153 ) .
I don't know this authoritatively , but that seemed unfavorable enough
to induce me to stop the new one .

By now , the first run is 37% through the ( minimal ) sieving .

Walter Nissen 2011-09-11 00:56

Unfortunately , the machine crashed to total black , looks like a thermal
cutout , after completing :
[code]
Found 5162239 relations, 84.6% of the estimated minimum (6105402).
[/code]
I copied the working dir and reinvoked :
[code]
..\factMsieve.py 79
[/code]
( it's called 79 , but it's a c127 )
but it quickly died :
[code]
-> gnfs-lasieve4I13e -k -o spairs.out.T0 -v -n0 -r 79.job.T0
-> gnfs-lasieve4I13e -k -o spairs.out.T1 -v -n1 -r 79.job.T1
Traceback (most recent call last):
File "C:\gg\factMsieve.py", line 2066, in <module>
gnfs-lasieve4I13e: L1_BITS=15, SVN $Revision: 406 $
run_siever(client_id, num_clients, SV_THREADS, fact_p, lats_p)
File "C:\gg\factMsieve.py", line 1701, in run_siever
gnfs-lasieve4I13e: L1_BITS=15, SVN $Revision: 406 $
ret = monitor_sieve_threads()
File "C:\gg\factMsieve.py", line 1650, in monitor_sieve_threads
read_spq(fact_p)
File "C:\gg\factMsieve.py", line 1638, in read_spq
t = int(chomp(tmp))
ValueError: invalid literal for int() with base 10: ''
siever terminated

2011-09-10 4:51:19.78 C:\gg\79res>
[/code]
Re-reinvoking failed similarly over in the copied dir .

This is the scene now :
[code]
2011-09-10 2:35:38.46 C:\gg\79> dir /od
Volume in drive C is SQ003627
Volume Serial Number is 6C40-871F

Directory of C:\gg\79

2011-05-03 16:04 133 79.n
2011-09-04 21:47 344 79.poly
2011-09-04 21:49 129 79.ini
2011-09-04 21:49 304 79.fb
2011-09-04 21:49 2,265,496 79.job.T1.afb.0
2011-09-04 21:49 2,265,496 79.job.T0.afb.0
2011-09-09 23:22 1,100 ggnfs.log
2011-09-09 23:22 541,333,555 79.dat
2011-09-09 23:23 275,930,319 spairs.save.gz
2011-09-09 23:23 385 79.job.T1
2011-09-09 23:23 92 79.job.resume
2011-09-09 23:23 16,351 79.log
2011-09-09 23:23 385 79.job.T0
2011-09-09 23:23 995,328 spairs.out.T0
2011-09-09 23:23 <DIR> ..
2011-09-09 23:23 <DIR> .
2011-09-09 23:23 970,752 spairs.out.T1
2011-09-09 23:53 8 .last_spq0
2011-09-09 23:53 8 .last_spq1
17 File(s) 823,780,185 bytes
2 Dir(s) 79,779,442,688 bytes free

2011-09-10 2:35:46.17 C:\gg\79>

2011-09-10 13:50:45.65 C:\gg\79> type 79.job.T0
n: 35457475543304274597570477263945249190083416047081166816136121660551117693024
73489038103286321807499348516053117524995279478089
c5: 1
c0: 38950081
Y1: -29134419507545592909032289199
Y0: 2814749767106560000000000000000
skew: 33.00
rlim: 4000000
alim: 4000000
lpbr: 25
lpba: 25
mfbr: 54
mfba: 54
rlambda: 2.6
alambda: 2.6
q0: 4500000
qintsize: 50000
#q1:4550000

2011-09-10 13:51:07.82 C:\gg\79> type .last_spq0
4505029

2011-09-10 13:51:42.95 C:\gg\79>
[/code]

I've read probably an additional hundred messages here without
being able to answer this question :
Can you tell me what must be done to resume this factorization ?

Walter Nissen 2011-09-11 01:10

While the machine was frustrated ( and frustrating ) , I looked at
whether I should have varied from NUM_THREADS = 1 with
NUM_THREADS = 2 .

<<< with 2 threads first ( 2 ) jobs ran
307 min. Found 234318 relations, 3.8% of the estimated minimum (6105402).

<<< with 1 thread first job ran
380 min. Found 234315 relations, 3.8% of the estimated minimum (6105402).

I intend upon staying with NUM_THREADS = 2 .

( Details on these runs appear in the previous 15 messages or so in this
thread beginning with #589 .
)


All times are UTC. The time now is 22:06.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.