GMP-ECM GPU for WSL problems - Page 2

VBCurtis · 2022-09-30, 04:38

Try smaller bounds for B1 and/or a smaller input.

Or, tell us what input number (in digits or an expression) and B1/B2 you're using- you may be asking the program to do something well outside the norm, in a way that requires a hopeless amount of memory.

Rubiksmath · 2022-09-30, 05:16

Okay, B1 of 1e10 fails in stage 2, B1 of 2e10 fails in stage 1. I know that prime95 can cope with those bounds just fine (i guess it uses swap space or something that doesnt require as much memory idk), but yeah on prime95 I have ran curves at higher B1 levels (1e11) for far bigger numbers (~400 digits), the one I am testing rn has 248 digits.

Rubiksmath · 2022-09-30, 10:42

Argh! this never ends! I tried to install CGBN but to no avail. It says you need gootletest framework, okay, it says you can use make download-gtest so I do that and it fails because of some weird file match error, like warning: filename not matched: googletest-master/googletest/*

So I go fine, I will install googletest manually. So I go and do that, set all the environment variables, and then run make volta in CGBN directory and... oh my god ive never seen so many errors in my life. just line after line of error after error, I counted at least 15 different errors before the makefile exited with error status. Anyone know why and what I can do to fix it??? Looks like there will be a day #6 to this... oof.

EdH · 2022-09-30, 12:50

I haven't been able to get GMP-ECM to compile anymore with CGBN. I've been told you don't need to compile googletest (but on my working machine, I did). You only need the cgbn.h file available, but GMP-ECM won't find it in plain sight, right where it is looking. Further digging showed me that the reason it claims it can't find it is actually because it tries to compile something with it and the compile fails. This is reported as not finding cgbn.h. You might review this thread:

Faster GPU-ECM with CGBN

for more info. If you can contact Seth, it may also help.

Currently, I haven't been able to get one of my GPU machines to compile GMP-ECM for the GPU, at all, even though I have it running on another GPU machine fine with CGBN. Perhaps I need to check if I'm trying 7.0.6 (which is probably the case).

chris2be8 · 2022-09-30, 15:36

Quote:

Originally Posted by Rubiksmath

Okay, B1 of 1e10 fails in stage 2, B1 of 2e10 fails in stage 1. I know that prime95 can cope with those bounds just fine (i guess it uses swap space or something that doesnt require as much memory idk), but yeah on prime95 I have ran curves at higher B1 levels (1e11) for far bigger numbers (~400 digits), the one I am testing rn has 248 digits.

Does make test work? If it fails please post output here.

Try B1=1e5 and B2=1e6 on that number (small enough they should work on any GPU). Then try larger B1 and B2 until you find the limit.

What make of GPU do you have? And how much memory does it have?

ECM is not prim95, it's designed to work on general numbers, not just Mersenne and Fermat numbers. But that means it needs more memory to handle them.

And CUDA is about typical for commercial software, ie rushed out as soon as possible in order to start earning money. I spent 18 years working as a systems programmer on a mainframe installing and supporting commercial software and you would not want to believe some of the things I've seen sent out to paying customers.

VBCurtis · 2022-09-30, 15:41

Quote:

Originally Posted by Rubiksmath

Okay, B1 of 1e10 fails in stage 2, B1 of 2e10 fails in stage 1. I know that prime95 can cope with those bounds just fine (i guess it uses swap space or something that doesnt require as much memory idk), but yeah on prime95 I have ran curves at higher B1 levels (1e11) for far bigger numbers (~400 digits), the one I am testing rn has 248 digits.

Try B1 of like 1e5 to test program function. Work your way up to see where your card's memory limit is. I'd expect 5e8 to work on most cards, 1e9 is a "probably". Is stage 2 still run on CPU? The stage 2 bound for B1 = 1e10 is quite large, and will require something like 20-40GB ram per curve.

Bounds of, say, B1 = 12e8 and B2 = 38e12 are big enough to find factors in the 63-68 digit range while stage 1 should fit on your GPU and stage 2 should take ~5GB per curve. I'm doubtful that B1 = 3e9 for T70-sized curves will fit.

P95 (at least before 30.9's beta ECM) uses an entirely different algorithm for stage 2 which uses much less memory but is vastly slower for large B2.

wombatman · 2022-09-30, 19:41

I got GPU-enabled and CGBN-enabled GMP-ECM working on my WSL2 installation of Ubuntu 22.04.1 LTS.

I'm using the git repository here: https://gitlab.inria.fr/zimmerma/ecm

My GPU is a 3090, and I'm using CUDA 11.7, GCC 11.2, and GMP 6.2.1, compiling for CC8.6. I don't know if that has anything to with my lack of issues compiling for CGBN or not.

The biggest thing is making sure you install CUDA via NVIDIA's instructions here: https://docs.nvidia.com/cuda/wsl-use...pport-for-wsl2

The configure line I used was

Code:

./configure --enable-gpu --with-cuda=/usr/local/cuda --with-cgbn-include=/home/wombat/CGBN/include/cgbn

It seems like you've worked through most of these issues, but I figured I would share what worked for me to see if it could give you any insight into the remaining issues.

Rubiksmath · 2022-09-30, 21:55

Quote:

Originally Posted by wombatman

I got GPU-enabled and CGBN-enabled GMP-ECM working on my WSL2 installation of Ubuntu 22.04.1 LTS.

How did you go about using the CGBN? do you actually have to compile the CGBN or can you just use the source?
Also, dang I mean I couldnt get it working on WSL at all despite days of trying, do you know what you did differently to me? I used like 10 clean installs and installed the CUDA exactly as required a ridiculous amount of times and then tried doing it other ways, to no avail.

I also dont really know how to get specific versions of GCC and whatnot i just did sudo apt-get install gcc and g++ and all that, is that a problem?

wombatman · 2022-10-01, 03:23

Quote:

Originally Posted by Rubiksmath

How did you go about using the CGBN? do you actually have to compile the CGBN or can you just use the source?
Also, dang I mean I couldnt get it working on WSL at all despite days of trying, do you know what you did differently to me? I used like 10 clean installs and installed the CUDA exactly as required a ridiculous amount of times and then tried doing it other ways, to no avail.

I also dont really know how to get specific versions of GCC and whatnot i just did sudo apt-get install gcc and g++ and all that, is that a problem?

Assuming a clean install of WSL2 (it *MUST* be WSL2--WSL1 doesn't support GPU at all), I would start with installing the build-essential package, then installing CUDA as directed by NVIDIA. Confirm CUDA is installed properly by downloading the CUDA samples and compiling and running a few of them (again, instructions on NVIDIA's site).

Then, compile gmp. You can also make sure ecm without GPU support builds and tests correctly.

I didn't have to build/compile anything for CGBN before compiling GMP-ECM with GPU/CGBN, just have the directory as-is.

Rubiksmath · 2022-10-01, 04:18

Okay interesting, I never compiled gmp I just ran sudo apt-get install libgmp3-dev, could that be the problem? Also, just to eliminate any kind of user error on my side could you point me to the exact files and versions you used for gimp and Cgbn. My internet just died today and we dunno why so it may be a good while before I can try it out. Thanks for the help tho.

wombatman · 2022-10-01, 16:44

Quote:

Originally Posted by Rubiksmath

Okay interesting, I never compiled gmp I just ran sudo apt-get install libgmp3-dev, could that be the problem? Also, just to eliminate any kind of user error on my side could you point me to the exact files and versions you used for gimp and Cgbn. My internet just died today and we dunno why so it may be a good while before I can try it out. Thanks for the help tho.

GMP 6.2.1 is here: https://gmplib.org/

CGBN came from here: https://github.com/NVlabs/CGBN

2022-09-30, 19:41	#18
wombatman I moo ablest echo power! May 2013 2²×463 Posts	I got GPU-enabled and CGBN-enabled GMP-ECM working on my WSL2 installation of Ubuntu 22.04.1 LTS. I'm using the git repository here: https://gitlab.inria.fr/zimmerma/ecm My GPU is a 3090, and I'm using CUDA 11.7, GCC 11.2, and GMP 6.2.1, compiling for CC8.6. I don't know if that has anything to with my lack of issues compiling for CGBN or not. The biggest thing is making sure you install CUDA via NVIDIA's instructions here: https://docs.nvidia.com/cuda/wsl-use...pport-for-wsl2 The configure line I used was Code: ./configure --enable-gpu --with-cuda=/usr/local/cuda --with-cgbn-include=/home/wombat/CGBN/include/cgbn It seems like you've worked through most of these issues, but I figured I would share what worked for me to see if it could give you any insight into the remaining issues.

2022-10-01, 04:18	#21
Rubiksmath Sep 2022 3²·7 Posts	Okay interesting, I never compiled gmp I just ran sudo apt-get install libgmp3-dev, could that be the problem? Also, just to eliminate any kind of user error on my side could you point me to the exact files and versions you used for gimp and Cgbn. My internet just died today and we dunno why so it may be a good while before I can try it out. Thanks for the help tho. Last fiddled with by Rubiksmath on 2022-10-01 at 04:37

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Just a few problems.	storm5510	YAFU	3	2019-10-21 22:25
PC problems	Nimras	Information & Answers	6	2009-12-15 21:24
Readline problems	CRGreathouse	Software	11	2009-07-07 05:18
Need help with few problems	Laserjet	Hardware	1	2007-10-13 10:59
Two problems	gribozavr	Puzzles	11	2007-02-05 05:46

2022-09-30, 04:38	#12
VBCurtis "Curtis" Feb 2005 Riverside, CA 3×1,879 Posts	Try smaller bounds for B1 and/or a smaller input. Or, tell us what input number (in digits or an expression) and B1/B2 you're using- you may be asking the program to do something well outside the norm, in a way that requires a hopeless amount of memory.

2022-09-30, 05:16	#13
Rubiksmath Sep 2022 3²·7 Posts	Okay, B1 of 1e10 fails in stage 2, B1 of 2e10 fails in stage 1. I know that prime95 can cope with those bounds just fine (i guess it uses swap space or something that doesnt require as much memory idk), but yeah on prime95 I have ran curves at higher B1 levels (1e11) for far bigger numbers (~400 digits), the one I am testing rn has 248 digits.

2022-09-30, 10:42	#14
Rubiksmath Sep 2022 3²×7 Posts	Argh! this never ends! I tried to install CGBN but to no avail. It says you need gootletest framework, okay, it says you can use make download-gtest so I do that and it fails because of some weird file match error, like warning: filename not matched: googletest-master/googletest/* So I go fine, I will install googletest manually. So I go and do that, set all the environment variables, and then run make volta in CGBN directory and... oh my god ive never seen so many errors in my life. just line after line of error after error, I counted at least 15 different errors before the makefile exited with error status. Anyone know why and what I can do to fix it??? Looks like there will be a day #6 to this... oof.

2022-09-30, 12:50	#15
EdH "Ed Hall" Dec 2009 Adirondack Mtns 2³·5·131 Posts	I haven't been able to get GMP-ECM to compile anymore with CGBN. I've been told you don't need to compile googletest (but on my working machine, I did). You only need the cgbn.h file available, but GMP-ECM won't find it in plain sight, right where it is looking. Further digging showed me that the reason it claims it can't find it is actually because it tries to compile something with it and the compile fails. This is reported as not finding cgbn.h. You might review this thread: Faster GPU-ECM with CGBN for more info. If you can contact Seth, it may also help. Currently, I haven't been able to get one of my GPU machines to compile GMP-ECM for the GPU, at all, even though I have it running on another GPU machine fine with CGBN. Perhaps I need to check if I'm trying 7.0.6 (which is probably the case).