Python Driver for GGNFS and MSIEVE

[Brian Gladman]

The only condition I can see that would lead to the missing '*.ini' and '*.fb' files is when it is intended to run a factorisation from scratch but it has been run before and a spurious '*.resume' file from the previous run still exists. I could not see anything obvious in the log file.

Brian

[Mini-Geek]

As noted here, factmsieve.py chokes when msieve finds factors (at least, factors due to the number being a prime power found during the polynomial selection).
Here's a log from the screen:

Msieve v. 1.44
Fri May 28 06:50:53 2010
random seeds: a84fa7fc 3deedde0
factoring 290064648515653604290850896210557000788223358877016965219834376481563501426743334123375488
518677419822009003459489974639850922949258651900751911554066671289177491607944029 (171 digits)
searching for 15-digit factors
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
elapsed time 00:00:03
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.       This is version 0.44, dated 26th January 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Using 4 threads
-> Working with NAME = test
-> Error: Polynomial file test.poly does not exist!
-> Found n = 290064648515653604290850896210557000788223358877016965219834376481563501426743334123375
488518677419822009003459489974639850922949258651900751911554066671289177491607944029.
-> Running polynomial selection ...
-> msieve -s test.dat -l test.log -i test.ini -nf test.fb -v -np


Msieve v. 1.44
Fri May 28 06:50:56 2010
random seeds: 867cfc78 8f1b8ed8
factoring 290064648515653604290850896210557000788223358877016965219834376481563501426743334123375488
518677419822009003459489974639850922949258651900751911554066671289177491607944029 (171 digits)
searching for 15-digit factors
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
elapsed time 00:00:02
Traceback (most recent call last):
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 1975, in <module>
    run_msieve_poly(fact_p)
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 889, in run_msieve_poly
    fb_to_poly()
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 856, in fb_to_poly
    with open(NAME + '.fb', 'r') as in_f:
IOError: [Errno 2] No such file or directory: 'test.fb'
siever terminated

If you fixed this in a recent version, sorry, I haven't upgraded in a while.

[Brian Gladman]

I am not sure how to handle this. The script is designed to expect a polynomial and process this accordingly.

What msieve output do I need to look for in order to determine that it hasn't produced a polynomial but has instead produced factors? What files should I parse for the factors?

Brian

[jasonp]

I'm not sure it's worth worrying about this case, since the odds of a general input large enough for GNFS but actually a power is essentially zero. msieve gives no indication that it did not run GNFS, other than not printing 'commencing number field sieve' in the logfile.

[Mini-Geek]

Originally Posted by Brian Gladman

I am not sure how to handle this. The script is designed to expect a polynomial and process this accordingly.

What msieve output do I need to look for in order to determine that it hasn't produced a polynomial but has instead produced factors? What files should I parse for the factors?

Brian

Considering the rarity of it, if it's too hard to implement, you could just ignore it.
But if not, here's what msieve writes to msieve.log:

Fri May 28 08:18:28 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 08:18:28 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109

I suppose you could look for prpxx. If there was a very small factor, there can also be something like "p1 factor: 3".

[Brian Gladman]

Is it always the case that there won't be a *.fb file when this happens? Do you know what is written to the normal log file, i.e. test.log in the example you gave above?

Brian

[jasonp]

A *.fb file would not be generated because the factorization would be complete before the NFS code had a chance to run.

[Mini-Geek]

Originally Posted by Brian Gladman

Do you know what is written to the normal log file, i.e. test.log in the example you gave above?

Here it is:

Fri May 28 11:53:42 2010 -> Running polynomial selection ...
Fri May 28 11:53:42 2010 -> msieve -s test.dat -l test.log -i test.ini -nf test.fb -v -np
Fri May 28 11:53:42 2010  
Fri May 28 11:53:42 2010  
Fri May 28 11:53:42 2010  Msieve v. 1.44
Fri May 28 11:53:42 2010  random seeds: 2a73a3ec 23518efe
Fri May 28 11:53:42 2010  factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091850205041234129084385881 (114 digits)
Fri May 28 11:53:43 2010  searching for 15-digit factors
Fri May 28 11:53:44 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 11:53:44 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 11:53:44 2010  elapsed time 00:00:02

And for full information and comparison purposes, here are msieve.log and the screen output:

msieve.log:
Fri May 28 11:53:40 2010  
Fri May 28 11:53:40 2010  
Fri May 28 11:53:40 2010  Msieve v. 1.44
Fri May 28 11:53:40 2010  random seeds: f38a85bc 983ba755
Fri May 28 11:53:40 2010  factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091850205041234129084385881 (114 digits)
Fri May 28 11:53:41 2010  searching for 15-digit factors
Fri May 28 11:53:41 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 11:53:41 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 11:53:41 2010  elapsed time 00:00:01

screen:
Msieve v. 1.44
Fri May 28 11:53:40 2010
random seeds: f38a85bc 983ba755
factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091
850205041234129084385881 (114 digits)
searching for 15-digit factors
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
elapsed time 00:00:01
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.       This is version 0.44, dated 26th January 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Using 4 threads
-> Working with NAME = test
-> Error: Polynomial file test.poly does not exist!
-> Found n = 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192
091850205041234129084385881.
-> Running polynomial selection ...
-> msieve -s test.dat -l test.log -i test.ini -nf test.fb -v -np


Msieve v. 1.44
Fri May 28 11:53:42 2010
random seeds: 2a73a3ec 23518efe
factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091
850205041234129084385881 (114 digits)
searching for 15-digit factors
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
elapsed time 00:00:02
Traceback (most recent call last):
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 1975, in <module>
    run_msieve_poly(fact_p)
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 889, in run_msieve_poly
    fb_to_poly()
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 856, in fb_to_poly
    with open(NAME + '.fb', 'r') as in_f:
IOError: [Errno 2] No such file or directory: 'test.fb'
siever terminated

[Brian Gladman]

Thanks Mini-Geek and Jason.

I can clearly just exit if the *.fb file doesn't exist but I am wondering if I should write anything to the log file and the summary file when this happens. Does msieve write anything to the script's log file?

Looking at the structure of the script it might not be easy to do much.

Brian

[Mini-Geek]

I just noticed that aliqueit had been set to run msieve before calling factmsieve (the option was made and I set it as such with a factmsieve.pl that doesn't use msieve for poly selection). The end result is still the same, of course, but it logs differently (only one msieve call and no msieve.log):

no msieve.log file

test.log:
Fri May 28 12:01:23 2010 -> Running polynomial selection ...
Fri May 28 12:01:23 2010 -> msieve -s test.dat -l test.log -i test.ini -nf test.fb -v -np
Fri May 28 12:01:23 2010  
Fri May 28 12:01:23 2010  
Fri May 28 12:01:23 2010  Msieve v. 1.44
Fri May 28 12:01:23 2010  random seeds: 6b80e878 37aece44
Fri May 28 12:01:23 2010  factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091850205041234129084385881 (114 digits)
Fri May 28 12:01:24 2010  searching for 15-digit factors
Fri May 28 12:01:25 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 12:01:25 2010  prp57 factor: 661959776823384128858607768192010463436677049149657043109
Fri May 28 12:01:25 2010  elapsed time 00:00:02

screen:
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.       This is version 0.44, dated 26th January 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Using 4 threads
-> Working with NAME = test
-> Error: Polynomial file test.poly does not exist!
-> Found n = 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192
091850205041234129084385881.
-> Running polynomial selection ...
-> msieve -s test.dat -l test.log -i test.ini -nf test.fb -v -np


Msieve v. 1.44
Fri May 28 12:01:23 2010
random seeds: 6b80e878 37aece44
factoring 438190746132064523680359691464734744483214502885929539054808778130318905370682196955192091
850205041234129084385881 (114 digits)
searching for 15-digit factors
prp57 factor: 661959776823384128858607768192010463436677049149657043109
prp57 factor: 661959776823384128858607768192010463436677049149657043109
elapsed time 00:00:02
Traceback (most recent call last):
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 1975, in <module>
    run_msieve_poly(fact_p)
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 889, in run_msieve_poly
    fb_to_poly()
  File "C:\Files\Prime\aliquot\ggnfs-374\factmsieve.py", line 856, in fb_to_poly
    with open(NAME + '.fb', 'r') as in_f:
IOError: [Errno 2] No such file or directory: 'test.fb'
siever terminated

So msieve does write some things to the script's log file, including the factors (you might've read my pre-edit post saying there were no factors in test.log even though I showed them). But it doesn't, say, give a special message saying, "message to all people and scripts: factors were found even though you probably weren't expecting them".

[Brian Gladman]

Thats good as all I need to do is exit if no *.fb exists. It would be nice to write a summary file but this might be a bit tricky as I would have to skip a lot of normal processing to get to the end of the script.

Brian

Edit: here is an attachment that attempts to remove this problem.