mersenneforum.org  

Go Back   mersenneforum.org > Factoring Projects > Msieve

Reply
 
Thread Tools
Old 2011-05-01, 06:38   #1
loong
 
Apr 2011

2×3 Posts
Default squre root step problem

Dear all,
I use factmsieve.py to factoring a big N, and it runs very well till the squre root step.
The output is as following:
Code:
xxx@xxx64:/data/corefile/xxx/factortool/test2> /usr/local/bin/python3 factmsieve2.py test2.n
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico.    Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
test2
-> This is client 1 of 1
-> Running on 4 Cores with 1 hyper-thread per Core
-> Working with NAME = test2
-> Selected default factorization parameters for 163 digit level.
-> Selected lattice siever: gnfs-lasieve4I15e
-> No parameter change detected, resuming...
-> Running lattice siever ...
-> entering sieving loop
-> File 'deps' already exists. Proceeding to sqrt step.
-> Running square root step ...
/data/corefile/xxx/factortool
./msieve
-> ./msieve -s test2/test2.dat -l test2/test2.log -i test2/test2.ini -nf test2/test2.fb -t 4 -nc3
-> Computing 1.30423e+09 scale for this machine...
-> procrels -speedtest> PIPE
 __________________________________________________________
|        This is the procrels program for GGNFS.           |
| Version: 0.77.1-20060722-nocona                          |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
b[a%1024] = 00000000
timeunit: 2.063
Traceback (most recent call last):
  File "factmsieve2.py", line 2124, in <module>
    output_summary(NAME, fact_p, pols_p, poly_p, lats_p)
  File "factmsieve2.py", line 1807, in output_summary
    tmp = grep_l('timeunit:', res)
  File "factmsieve2.py", line 190, in grep_l
    for l in lines:
TypeError: 'NoneType' object is not iterable
and test2.log shows:

...
Sun May 1 14:15:58 2011 commencing square root phase
Sun May 1 14:15:58 2011 reading relations for dependency 1
Sun May 1 14:16:01 2011 read 2845889 cycles
Sun May 1 14:16:08 2011 cycles contain 7925904 unique relations
Sun May 1 14:25:03 2011 read 7925904 relations
Sun May 1 14:26:03 2011 multiplying 7925904 relations
Sun May 01 14:35:20 2011 -> Computing 1.30423e+09 scale for this machine...
Sun May 01 14:35:20 2011 -> procrels -speedtest> PIPE


Is there any error with my whole step?
loong is offline   Reply With Quote
Old 2011-05-01, 13:24   #2
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

2·3·19·31 Posts
Default

It looks like you got a reasonable dependency that crashed when you tried to use it. This would sometimes happen if you compiled Msieve or GMP with gcc 4.1.x, for reasons I've never been able to understand. gcc 4.2.x is reported to be safe. The other possibility is that you ran out of memory. If you're running on a 32-bit widows system that could be the problem, give that size of dataset.
jasonp is offline   Reply With Quote
Old 2011-05-01, 15:46   #3
WraithX
 
WraithX's Avatar
 
Mar 2006

23·59 Posts
Default

Quote:
Originally Posted by loong View Post
Code:
-> Running square root step ...
/data/corefile/xxx/factortool
./msieve
-> ./msieve -s test2/test2.dat -l test2/test2.log -i test2/test2.ini -nf test2/test2.fb -t 4 -nc3
-> Computing 1.30423e+09 scale for this machine...
-> procrels -speedtest> PIPE
 __________________________________________________________
|        This is the procrels program for GGNFS.           |
| Version: 0.77.1-20060722-nocona                          |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
b[a%1024] = 00000000
timeunit: 2.063
Traceback (most recent call last):
  File "factmsieve2.py", line 2124, in <module>
    output_summary(NAME, fact_p, pols_p, poly_p, lats_p)
  File "factmsieve2.py", line 1807, in output_summary
    tmp = grep_l('timeunit:', res)
  File "factmsieve2.py", line 190, in grep_l
    for l in lines:
TypeError: 'NoneType' object is not iterable
Also, this error was thrown by Python itself, and is not an error returned by one of the sub-programs called by the python script. This error seems to be due to the fact that the output from "procrels -speedtest> PIPE" was printed to stdout and not captured by factmsieve.py for later grep-ing. Since it wasn't saved internally, there was no info to grep, hence the error. It was looking for the line "timeunit: 2.063" that was printed to stdout.

One way to see if this will complete successfully for you is to modify the script like so:

Find the following two lines (the first is at line 1807 of your factmsieve2.py):
Code:
  tmp = grep_l('timeunit:', res)
  timescale = float(re.sub('timeunit:\s*', '', tmp[0]))
And change them to:
Code:
  # tmp = grep_l('timeunit:', res)
  # timescale = float(re.sub('timeunit:\s*', '', tmp[0]))
  timescale = 2.063
This might be a problem with the version of Python that you are running. Could you run '/usr/local/bin/python3 -V' from the command line and let us know which version of python you are running. And also, which operating system are you on?
WraithX is offline   Reply With Quote
Old 2011-05-02, 11:57   #4
loong
 
Apr 2011

2·3 Posts
Default

Quote:
Originally Posted by jasonp View Post
It looks like you got a reasonable dependency that crashed when you tried to use it. This would sometimes happen if you compiled Msieve or GMP with gcc 4.1.x, for reasons I've never been able to understand. gcc 4.2.x is reported to be safe. The other possibility is that you ran out of memory. If you're running on a 32-bit widows system that could be the problem, give that size of dataset.
Thanks for your reply jasonp.
I've executed msieve with the following command alone:
./msieve -v -s test2/test2.dat -l test2/test2.log -i test2/test2.ini -nf test2/test2.fb -t 4 -nc3

and get the output:

commencing square root phase
reading relations for dependency 1
read 2845889 cycles
cycles contain 7925904 unique relations
read 7925904 relations
multiplying 7925904 relations
Segmentation fault

Note the "Segmentation fault".
I've checked the gcc version, and it is exactly 4.1.x :
gcc version 4.1.2 20070115 (prerelease) (SUSE Linux)

and I compiled msieve under linux x86_64, the memory is 8GB, so it might not the problem of that run out of memory.

I'll update gcc to 4.2.x to see if it will happen again.

Thanks again.
loong is offline   Reply With Quote
Old 2011-05-02, 12:02   #5
loong
 
Apr 2011

2×3 Posts
Default

Quote:
Originally Posted by WraithX View Post
Also, this error was thrown by Python itself, and is not an error returned by one of the sub-programs called by the python script. This error seems to be due to the fact that the output from "procrels -speedtest> PIPE" was printed to stdout and not captured by factmsieve.py for later grep-ing. Since it wasn't saved internally, there was no info to grep, hence the error. It was looking for the line "timeunit: 2.063" that was printed to stdout.

One way to see if this will complete successfully for you is to modify the script like so:

Find the following two lines (the first is at line 1807 of your factmsieve2.py):
Code:
  tmp = grep_l('timeunit:', res)
  timescale = float(re.sub('timeunit:\s*', '', tmp[0]))
And change them to:
Code:
  # tmp = grep_l('timeunit:', res)
  # timescale = float(re.sub('timeunit:\s*', '', tmp[0]))
  timescale = 2.063
This might be a problem with the version of Python that you are running. Could you run '/usr/local/bin/python3 -V' from the command line and let us know which version of python you are running. And also, which operating system are you on?
WraithX, thanks for your help.
I'm running python 3.2, and the operating system is SuSe Linux SP2 x86_64.

Thanks for your help again.
loong is offline   Reply With Quote
Old 2011-05-03, 02:28   #6
loong
 
Apr 2011

2·3 Posts
Default

I've updated gcc to 4.6.0, and have compiled msieve again, but the problem still exists, am I must change gcc to 4.2.x?
loong is offline   Reply With Quote
Old 2011-05-03, 08:18   #7
fivemack
(loop (#_fork))
 
fivemack's Avatar
 
Feb 2006
Cambridge, England

18EE16 Posts
Default

I would suggest

a) build msieve against the newest version of gmp
b) run 'ulimit -s unlimited' before trying the square-root step

I have had this problem in the past and it came from gmp running out of stack space in the very-bignum arithmetic.
fivemack is offline   Reply With Quote
Old 2011-05-03, 08:50   #8
loong
 
Apr 2011

68 Posts
Default

Quote:
Originally Posted by fivemack View Post
I would suggest

a) build msieve against the newest version of gmp
b) run 'ulimit -s unlimited' before trying the square-root step

I have had this problem in the past and it came from gmp running out of stack space in the very-bignum arithmetic.
Thanks for your suggestion.
I've run 'ulimit -s unlimited' before trying the square root step, and msieve successfully passed the 1st dependency.
It looks like follows:

Tue May 3 10:49:46 2011 commencing square root phase
Tue May 3 10:49:46 2011 reading relations for dependency 1
Tue May 3 10:49:49 2011 read 2845889 cycles
Tue May 3 10:49:55 2011 cycles contain 7925904 unique relations
Tue May 3 10:58:42 2011 read 7925904 relations
Tue May 3 10:59:40 2011 multiplying 7925904 relations
Tue May 3 11:58:43 2011 multiply complete, coefficients have about 420.10 million bits
Tue May 3 11:58:50 2011 initial square root is modulo 34510537

till now, 5 hours has been passed, but the output is still shows as above,
I don't know whether such a long time is normal or an exception has occured which caused msieve stop to running?

Last fiddled with by loong on 2011-05-03 at 08:53
loong is offline   Reply With Quote
Old 2011-05-03, 12:31   #9
loong
 
Apr 2011

2·3 Posts
Default

Thanks for all.
After 8 hours of square root phase, msieve successfully got the factors.
loong is offline   Reply With Quote
Old 2011-05-03, 12:47   #10
xilman
Bamboozled!
 
xilman's Avatar
 
"๐’‰บ๐’ŒŒ๐’‡ท๐’†ท๐’€ญ"
May 2003
Down not across

101001011001002 Posts
Default

Quote:
Originally Posted by loong View Post
Thanks for all.
After 8 hours of square root phase, msieve successfully got the factors.
Good!

As a matter of interest, what N were you factoring and what is its factors?

Paul
xilman is offline   Reply With Quote
Old 2011-05-03, 13:14   #11
jasonp
Tribal Bullet
 
jasonp's Avatar
 
Oct 2004

2·3·19·31 Posts
Default

Redacted logfile, matrix of size 5.6M, I'd say somebody's RSA key :)

(8 hours is a little bit long for a dependency of this size, but not unusually so if the leading algebraic polynomial coefficient is at all large)

Last fiddled with by jasonp on 2011-05-03 at 13:16
jasonp is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Donยดt step on my toes... please! lycorn Lone Mersenne Hunters 37 2011-10-09 16:09
GMP-ECM fails on step 2 with B2 > 96M CRGreathouse Factoring 9 2010-10-30 16:38
Optimizing step 2 of ECM on Prime95 alpertron Software 4 2006-01-11 17:27
Wildly differing times for step 2 with ecm-6.0.1 Jushi GMP-ECM 7 2005-09-12 01:30
One-step Hangman Ken_g6 Puzzles 1 2005-01-16 15:03

All times are UTC. The time now is 09:51.

Mon Mar 8 09:51:05 UTC 2021 up 95 days, 6:02, 0 users, load averages: 1.19, 1.11, 1.12

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.