Problem in running msieve with CUDA

[mohamed]

Hi all,

-I have successfully installed msieve on my 64-bit (core i7) machine running windows 7 and it managed to factor up to 80 digits integers.
Now i want to enable cuda to measure the performance improvement but when i set CUDA TRUE i get this error :



C:\ggnfs\example>python factmsieve.py example.n
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico. Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 8 Cores with 1 hyper-thread per Core
-> Working with NAME = example
-> Error: Polynomial file example.poly does not exist!
-> Found n = 100160063.
-> Running polynomial selection ...
-> sieve -s example\example.dat -l example\example.log -i example\example.ini -n
f example\example.fb -g 0 -v -np

Msieve v. 1.51 (SVN 845)

usage: sieve.exe [options] [one_number]

numbers starting with '0' are treated as octal,
numbers starting with '0x' are treated as hexadecimal

options:
-s <name> save intermediate results to <name>
instead of the default msieve.dat
-l <name> append log information to <name>
instead of the default msieve.log
-i <name> read one or more integers to factor from
<name> (default worktodo.ini) instead of
from the command line
-m manual mode: enter numbers via standard input
-q quiet: do not generate any log information,
only print any factors found
-d <min> deadline: if still sieving after <min>
minutes, shut down gracefully (default off)
-r <num> stop sieving after finding <num> relations
-p run at idle priority
-v verbose: write log information to screen
as well as to logfile
-t <num> use at most <num> threads

elliptic curve options:
-e perform 'deep' ECM, seek factors > 15 digits

quadratic sieve options:
-c client: only perform sieving

number field sieve options:

[nfs_phase] "arguments"

where the first part is one or more of:
-n use the number field sieve (80+ digits only;
performs all NFS tasks in order)
-nf <name> read from / write to NFS factor base file
<name> instead of the default msieve.fb
-np perform only NFS polynomial selection
-np1 perform stage 1 of NFS polynomial selection
-nps perform NFS polynomial size optimization
-npr perform NFS polynomial root optimization
-ns perform only NFS sieving
-nc perform only NFS combining (all phases)
-nc1 perform only NFS filtering
-nc2 perform only NFS linear algebra
-ncr perform only NFS linear algebra, restarting
from a previous checkpoint
-nc3 perform only NFS square root

the arguments are a space-delimited list of:
polynomial selection options:
polydegree=X select polynomials with degree X
min_coeff=X minimum leading coefficient to search
in stage 1
max_coeff=X maximum leading coefficient to search
in stage 1
stage1_norm=X the maximum norm value for stage 1
stage2_norm=X the maximum norm value for stage 2
min_evalue=X the minimum score of saved polyomials
poly_deadline=X stop searching after X seconds (0 means
search forever)
X,Y same as 'min_coeff=X max_coeff=Y'
line sieving options:
X,Y handle sieve lines X to Y inclusive
filtering options:
filter_mem_mb=X try to limit filtering memory use to
X megabytes
filter_maxrels=X limit the filtering to using the first
X relations in the data file
filter_lpbound=X have filtering start by only looking
at ideals of size X or larger
target_density=X attempt to produce a matrix with X
entries per column
X,Y same as 'filter_lpbound=X filter_maxrels=Y'
linear algebra options:
skip_matbuild=1 start the linear algebra but skip building
the matrix (assumes it is built already)
square root options:
X,Y use dependencies X through Y, 1<=X<=Y<=64)
Msieve Error: return value -1. Is CUDA enabled? Terminating...

C:\ggnfs\example>


could you please help me ? i am in a deep need of this method to work :(

[jasonp]

You need a special CUDA binary of msieve.exe and a lot of its support files. A 32-bit zip of the package you need is at the current Msieve download page.

[mohamed]

Thank You for the fast reply,
Could you please tell me exactly what to do ?
I have replaced the msieve.exe from version "msieve151_gpu.zip" but this apparently isn't the right method, it gives the same report as the one attached above.
also i am running it on a 64 bit. is there a solution to this problem ?

[jasonp]

Believe it or not, I have never used factmsieve.

Don't just replace the exe, you need all the GPU files from the zip file on sourceforge. Put them all in the same directory as the 64-bit msieve. The gpu release is a 32-bit binary but that will run fine on your 64-bit system. The GPU is used for NFS polynomial selection, which does not need much memory, so being 32-bit doesn't matter. Then try running and if it doesn't work then paste the output here.

Also, the NFS code will never run on a 10-digit input. You will need a test number at least 85 digits in size, and it is more comfortable with 100 digits and up. Try working through Jeff Gilchrist's example from his web page.

[wombatman]

You might also try leaving off the .n like this:

py factmsieve.py example

Sometimes a wrong extension can throw off the program.

[mohamed]

Thanks again MR.Jason,

Actually i did put all the files in the working directory but the output is the same.

*************************************************************


C:\ggnfs\example>python factmsieve.py example
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico. Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 16 Cores with 1 hyper-thread per Core
-> Working with NAME = example
-> Error: Polynomial file example.poly does not exist!
-> Found n = 3521851118865011044136429217528930691441965435121409905222808922963
363310303627.
-> Running polynomial selection ...
-> sieve -s example\example.dat -l example\example.log -i example\example.ini -n
f example\example.fb -g 0 -v -np

Msieve v. 1.51 (SVN 845)

usage: sieve.exe [options] [one_number]

numbers starting with '0' are treated as octal,
numbers starting with '0x' are treated as hexadecimal

options:
-s <name> save intermediate results to <name>
instead of the default msieve.dat
-l <name> append log information to <name>
instead of the default msieve.log
-i <name> read one or more integers to factor from
<name> (default worktodo.ini) instead of
from the command line
-m manual mode: enter numbers via standard input
-q quiet: do not generate any log information,
only print any factors found
-d <min> deadline: if still sieving after <min>
minutes, shut down gracefully (default off)
-r <num> stop sieving after finding <num> relations
-p run at idle priority
-v verbose: write log information to screen
as well as to logfile
-t <num> use at most <num> threads

elliptic curve options:
-e perform 'deep' ECM, seek factors > 15 digits

quadratic sieve options:
-c client: only perform sieving

number field sieve options:

[nfs_phase] "arguments"

where the first part is one or more of:
-n use the number field sieve (80+ digits only;
performs all NFS tasks in order)
-nf <name> read from / write to NFS factor base file
<name> instead of the default msieve.fb
-np perform only NFS polynomial selection
-np1 perform stage 1 of NFS polynomial selection
-nps perform NFS polynomial size optimization
-npr perform NFS polynomial root optimization
-ns perform only NFS sieving
-nc perform only NFS combining (all phases)
-nc1 perform only NFS filtering
-nc2 perform only NFS linear algebra
-ncr perform only NFS linear algebra, restarting
from a previous checkpoint
-nc3 perform only NFS square root

the arguments are a space-delimited list of:
polynomial selection options:
polydegree=X select polynomials with degree X
min_coeff=X minimum leading coefficient to search
in stage 1
max_coeff=X maximum leading coefficient to search
in stage 1
stage1_norm=X the maximum norm value for stage 1
stage2_norm=X the maximum norm value for stage 2
min_evalue=X the minimum score of saved polyomials
poly_deadline=X stop searching after X seconds (0 means
search forever)
X,Y same as 'min_coeff=X max_coeff=Y'
line sieving options:
X,Y handle sieve lines X to Y inclusive
filtering options:
filter_mem_mb=X try to limit filtering memory use to
X megabytes
filter_maxrels=X limit the filtering to using the first
X relations in the data file
filter_lpbound=X have filtering start by only looking
at ideals of size X or larger
target_density=X attempt to produce a matrix with X
entries per column
X,Y same as 'filter_lpbound=X filter_maxrels=Y'
linear algebra options:
skip_matbuild=1 start the linear algebra but skip building
the matrix (assumes it is built already)
square root options:
X,Y use dependencies X through Y, 1<=X<=Y<=64)
Msieve Error: return value -1. Is CUDA enabled? Terminating...

C:\ggnfs\example>



*************************************************************

I tried the version 1.52dev SVN 927 (compiled by Brian Gladman) from http://gilchrist.ca/jeff/factoring/

and again when setting CUDA to true all i get is :

*****************************************************


C:\ggnfs\example>python factmsieve.py example
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico. Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 16 Cores with 1 hyper-thread per Core
-> Working with NAME = example
-> Error: Polynomial file example.poly does not exist!
-> Found n = 3521851118865011044136429217528930691441965435121409905222808922
363310303627.
-> Running polynomial selection ...
-> sieve -s example\example.dat -l example\example.log -i example\example.ini
f example\example.fb -g 0 -v -np

Msieve v. 1.51 (SVN 845)

usage: sieve.exe [options] [one_number]

numbers starting with '0' are treated as octal,
numbers starting with '0x' are treated as hexadecimal

options:
-s <name> save intermediate results to <name>
instead of the default msieve.dat
-l <name> append log information to <name>
instead of the default msieve.log
-i <name> read one or more integers to factor from
<name> (default worktodo.ini) instead of
from the command line
-m manual mode: enter numbers via standard input
-q quiet: do not generate any log information,
only print any factors found
-d <min> deadline: if still sieving after <min>
minutes, shut down gracefully (default off)
-r <num> stop sieving after finding <num> relations
-p run at idle priority
-v verbose: write log information to screen
as well as to logfile
-t <num> use at most <num> threads

elliptic curve options:
-e perform 'deep' ECM, seek factors > 15 digits

quadratic sieve options:
-c client: only perform sieving

number field sieve options:

[nfs_phase] "arguments"

where the first part is one or more of:
-n use the number field sieve (80+ digits only;
performs all NFS tasks in order)
-nf <name> read from / write to NFS factor base file
<name> instead of the default msieve.fb
-np perform only NFS polynomial selection
-np1 perform stage 1 of NFS polynomial selection
-nps perform NFS polynomial size optimization
-npr perform NFS polynomial root optimization
-ns perform only NFS sieving
-nc perform only NFS combining (all phases)
-nc1 perform only NFS filtering
-nc2 perform only NFS linear algebra
-ncr perform only NFS linear algebra, restarting
from a previous checkpoint
-nc3 perform only NFS square root

the arguments are a space-delimited list of:
polynomial selection options:
polydegree=X select polynomials with degree X
min_coeff=X minimum leading coefficient to search
in stage 1
max_coeff=X maximum leading coefficient to search
in stage 1
stage1_norm=X the maximum norm value for stage 1
stage2_norm=X the maximum norm value for stage 2
min_evalue=X the minimum score of saved polyomials
poly_deadline=X stop searching after X seconds (0 means
search forever)
X,Y same as 'min_coeff=X max_coeff=Y'
line sieving options:
X,Y handle sieve lines X to Y inclusive
filtering options:
filter_mem_mb=X try to limit filtering memory use to
X megabytes
filter_maxrels=X limit the filtering to using the first
X relations in the data file
filter_lpbound=X have filtering start by only looking
at ideals of size X or larger
target_density=X attempt to produce a matrix with X
entries per column
X,Y same as 'filter_lpbound=X filter_maxrels=Y'
linear algebra options:
skip_matbuild=1 start the linear algebra but skip building
the matrix (assumes it is built already)
square root options:
X,Y use dependencies X through Y, 1<=X<=Y<=64)
Msieve Error: return value -1. Is CUDA enabled? Terminating...

C:\ggnfs\example>

*****************************************************

this 80 digit example takes about 7 seconds to be solved when setting CUDA to false,
I will try 100+ numbers but first i need the program to run when CUDA is enabled.

excuse me for my ignorance and i really appreciate your efforts.Thanks

[Brian Gladman]

SVN 927 is a normal binary and does not support CUDA. Right now I cannot produce a new CUDA binary because NVIDIA has been absurdly slow in getting thier tools to work with Visual Studio 2012. They said they would do this in Q2 2013 but they missed this deadline. They are now claiming that they will do this by the end of July but I wouldn't bet on it.

[jasonp]

Mohamed, is there more than one copy of msieve.exe on your machine? You can run 'msieve -h' on whichever binary you have and look for a '-g' choice in the list of command line options. That's the sign that the binary is GPU enabled (it's what I'm looking for in the output above).

If I do that with the binary from sourceforge, I get

options:
   -s <name> save intermediate results to <name>
             instead of the default msieve.dat
   -l <name> append log information to <name>
             instead of the default msieve.log
   -i <name> read one or more integers to factor from
             <name> (default worktodo.ini) instead of
             from the command line
   -m        manual mode: enter numbers via standard input
   -q        quiet: do not generate any log information,
             only print any factors found
   -d <min>  deadline: if still sieving after <min>
             minutes, shut down gracefully (default off)
   -r <num>  stop sieving after finding <num> relations
   -p        run at idle priority
   -v        verbose: write log information to screen
             as well as to logfile
   -g <num>  use GPU <num>, 0 <= num < (# graphics cards)>      <-----
   -t <num>  use at most <num> threads
...

[mohamed]

Thanks Mr.Brian Gladman, i hope they manage to make their tools compatible soon.

Mr.Jason,

yeah i have several versions of msieve including this one from sourceforge

I've tried msieve -h for all of them , but i got either the same output above (without -g)

or it gives system error :

This program can't start because VCOMP100.DLL is missing from your computer.

I've downloaded and installed Microsoft Visual C++ 2010 Redistributable Package (x64) but nothing changed, I've also downloaded VCOMP100.DLL but i don't know where to put it in order to make the program runs.

..

[wombatman]

Mohamed, are you changing the MSIEVE variable in the python script to point to the correct msieve you want to use? It looks like (based on the output from when you tried SVN 927) that you are pointing to SVN 845. In my script, the variable is defined on Line 94:

# ggnfs and msieve executable flie names

MSIEVE = 'msieve152_svn922_amd64_32bit_CUDA_ECM'

[mohamed]

sorry my fault ,
this is what i get from the version 1.52dev SVN 927 (compiled by Brian Gladman) : (CUDA set to false )



C:\ggnfs\example>python factmsieve.py example
-> ________________________________________________________________
-> | Running factmsieve.py, a Python driver for MSIEVE with GGNFS |
-> | sieving support. It is Copyright, 2010, Brian Gladman and is |
-> | a conversion of factmsieve.pl that is Copyright, 2004, Chris |
-> | Monico. Version 0.76 (Python 2.6 or later) 10th Nov 2010. |
-> |______________________________________________________________|
-> This is client 1 of 1
-> Running on 8 Cores with 1 hyper-thread per Core
-> Working with NAME = example
-> Error: Polynomial file example.poly does not exist!
-> Found n = 100160063.
-> Running polynomial selection ...
-> sieve -s example\example.dat -l example\example.log -i example\example.ini -n
f example\example.fb -v -np

__________________________________________________________
| This is the sieve program for GGNFS. |
| Version: 0.77.1-VC8(UTE) |
| This program is copyright 2004, Chris Monico, and subject|
| to the terms of the GNU General Public License version 2.|
|__________________________________________________________|
USAGE: sieve.exe [OPTIONS]
--help : show this help and exit
-fb <filename> : use factor base in file <filename>
-j <filename> : use job file <filename>
-s : short output format (only a,b)

-> Computing 1.37492e+09 scale for this machine...
-> procrels -speedtest> PIPE
Scaled time: 6793464631.16 units (timescale= 4.941).
Traceback (most recent call last):
File "factmsieve.py", line 2035, in <module>
output_summary(NAME, fact_p, pols_p, poly_p, lats_p)
File "factmsieve.py", line 1893, in output_summary
with open(NAME + '.poly', 'r') as in_f:
IOError: [Errno 2] No such file or directory: 'example.poly'

C:\ggnfs\example>

it doesnt even solve the number without setting CUDA


any help with a version that runs CUDA differs than the one i use or any other approach ?

I am stuck in here :((