mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Cloud Computing (https://www.mersenneforum.org/forumdisplay.php?f=134)
-   -   Google Diet Colab Notebook (https://www.mersenneforum.org/showthread.php?t=24646)

petrw1 2020-03-23 01:25

[QUOTE=chalsall;540538]Taking a quick stab in the dark... Your Subprocess() call appears to be passing all the parameters to CUDAPm1 as the file name.

Also, wrt your previous message, was your attempt to run "./CUDAPm1" within a Notebook section? If so, you need to do "!./CUDAPm1" instead, since it's within the Python shell.[/QUOTE]

OK making progress. Running this:

%cd '/content/drive/My Drive/cudapm1//'

# List files here
for x in os.listdir('.'):
print (x)

%cd '/content/drive/My Drive/cudapm1/cudapm1-0.20/'

# List files here too.
for x in os.listdir('.'):
print (x)

# libcufft.so.5.5 is in the list above but in just in case I also copied it to the parent
import shutil
shutil.copyfile('libcufft.so.5.5', '/content/drive/My Drive/cudapm1/libcufft.so.5.5')

# RUN it!!!
!./CUDAPm1 61408363 -b1 60000 -b2 1200000 -f 3360k

[CODE]/content/drive/My Drive/cudapm1
cudapm1-0.20
cudapm1-0.20.tar.gz
libcufft.so.5.5

/content/drive/My Drive/cudapm1/cudapm1-0.20
readme.txt
CUDAPm1.ini
libcufft.so.5.5
CUDAPm1
cudapm1-setup.txt

./CUDAPm1: error while loading shared libraries: libcufft.so.5.5: cannot open shared object file: No such file or directory[/CODE]

Dylan14 2020-03-23 14:14

[QUOTE=petrw1;540534]
---> 20 subprocess.call("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1 61408363 -b1 600000 -b2 12000000 -f 3360k'")[/QUOTE]

When you use subprocess.call (or subprocess.run), it expects each parameter to be in its own string, separated by a comma. The way you have it, it is trying to run the whole thing as an executable, but that is clearly not a valid file. Try this instead:

[code]subprocess.run(“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”)[/code]

petrw1 2020-03-23 17:09

subprocess.run ... ERROR
 
[QUOTE=Dylan14;540603]When you use subprocess.call (or subprocess.run), it expects each parameter to be in its own string, separated by a comma. The way you have it, it is trying to run the whole thing as an executable, but that is clearly not a valid file. Try this instead:

[code]subprocess.run(“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”)[/code][/QUOTE]

[CODE]import subprocess
subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k")[/CODE]

[CODE]TypeError Traceback (most recent call last)
<ipython-input-17-2cf284193cd6> in <module>()
26
27 #subprocess.run("/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1")
---> 28 subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k")
29 #subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363 -b1 600000 -b2 12000000 -f 3360k")
30 # run ./CUDAPm1 -cufftbench 1k to 65536k 2 times

1 frames
/usr/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
627 bufsize = -1 # Restore default
628 if not isinstance(bufsize, int):
--> 629 raise TypeError("bufsize must be an integer")
630
631 if _mswindows:

TypeError: bufsize must be an integer[/CODE]

Here is the section of code from subprocess.py
bufsize is set to -1 then checked later for integer and fails???

[CODE]def __init__(self, args, bufsize=-1, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0,
restore_signals=True, start_new_session=False,
pass_fds=(), *, encoding=None, errors=None):
"""Create new Popen instance."""
_cleanup()

self._waitpid_lock = threading.Lock()

self._input = None
self._communication_started = False
if bufsize is None:
bufsize = -1 # Restore default
if not isinstance(bufsize, int):
raise TypeError("bufsize must be an integer")[/CODE]

Dylan14 2020-03-23 17:45

[QUOTE=petrw1;540620]Here is the section of code from subprocess.py
bufsize is set to -1 then checked later for integer and fails???

[CODE]def __init__(self, args, bufsize=-1, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0,
restore_signals=True, start_new_session=False,
pass_fds=(), *, encoding=None, errors=None):
"""Create new Popen instance."""
_cleanup()

self._waitpid_lock = threading.Lock()

self._input = None
self._communication_started = False
if bufsize is None:
bufsize = -1 # Restore default
if not isinstance(bufsize, int):
raise TypeError("bufsize must be an integer")[/CODE][/QUOTE]

I missed a key thing: the list of arguments have to be enclosed in brackets ([]).
So it should read
[code]subprocess.run([“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”])[/code]Without it, it's treating "61408363" as the argument for bufsize, which is a string, not an integer. And that's why it's breaking.

petrw1 2020-03-23 20:01

Returncode=127
 
I tried these 3 variations (One at a time all with the same error):
[CODE]subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363 -b1 600000 -b2 12000000 -f 3360k"])
[/CODE]
[CODE]CompletedProcess(args=['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'], returncode=127)[/CODE]

How do I know if the 127 was from subprocess.run
OR
from CUDAPm1?

It it was from CUDAPm1 can someone familiar with it tell me what that code means,

Thx

chalsall 2020-03-23 20:54

[QUOTE=petrw1;540655]It it was from CUDAPm1 can someone familiar with it tell me what that code means,[/QUOTE]

Juggling a few balls at the moment, but I suspect that code is not good...

Your above "error while loading shared libraries: libcufft.so.5.5: cannot open shared object" suggests the executable doesn't like it's home. (Read: perhaps try compiling the code in it's native environment.)

Dylan14 2020-03-24 01:39

[QUOTE=petrw1;540655]I tried these 3 variations (One at a time all with the same error):
[CODE]subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363 -b1 600000 -b2 12000000 -f 3360k"])
[/CODE]
[CODE]CompletedProcess(args=['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'], returncode=127)[/CODE]

How do I know if the 127 was from subprocess.run
OR
from CUDAPm1?

It it was from CUDAPm1 can someone familiar with it tell me what that code means,

Thx[/QUOTE]

The return code comes from subprocess.run. Ideally, it should be 0, which means the program ran with no problems. Here, we got a non zero code, which means something went wrong when python tried to execute the program. In this case, something went wrong in executing cudapm1, and thus the process ended, and python returns the error code in a CompletedProcess object.
You might get more information if you add the keyword argument “check=True” to the run command (without the quotes).

petrw1 2020-03-24 04:05

Yup...from CUDAPm1 ...I think Chris is right; I need to recompile
 
[QUOTE=Dylan14;540714]The return code comes from subprocess.run. Ideally, it should be 0, which means the program ran with no problems. Here, we got a non zero code, which means something went wrong when python tried to execute the program. In this case, something went wrong in executing cudapm1, and thus the process ended, and python returns the error code in a CompletedProcess object.
You might get more information if you add the keyword argument “check=True” to the run command (without the quotes).[/QUOTE]

[CODE]CalledProcessError Traceback (most recent call last)
<ipython-input-28-d81f224b82fc> in <module>()
25 #import subprocess
26
---> 27 subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"],check=True)

/usr/lib/python3.6/subprocess.py in run(input, timeout, check, *popenargs, **kwargs)
436 if check and retcode:
437 raise CalledProcessError(retcode, process.args,
--> 438 output=stdout, stderr=stderr)
439 return CompletedProcess(process.args, retcode, stdout, stderr)
440

CalledProcessError: Command '['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1']' returned non-zero exit status 127.
[/CODE]

Uncwilly 2020-03-24 13:53

Anyone else think that Colab and the rest of google and amazon, M$, etc should through all their spare cycles at folding@home.

kriesel 2020-03-24 14:06

[QUOTE=Uncwilly;540748]Anyone else think that Colab and the rest of google and amazon, M$, etc should through all their spare cycles at folding@home.[/QUOTE]
Folding@home supposedly has 470 Petaflops as is. To put that in perspective, it's over [B]1200 times[/B] the stated throughput of GIMPS at [URL]https://www.mersenne.org/primenet/[/URL] (384 Tflops)

reference: [URL]https://www.techspot.com/news/84492-foldinghome-now-has-access-over-470-petaflops-distributed.html[/URL]
Which, I suppose, suggests there's room for GIMPS to grow, after the current COVID19 unpleasantness is behind us.

petrw1 2020-03-24 19:36

I see I have an old-experimental version. Is there a better one here?
 
[url]https://download.mersenne.ca/CUDAPm1/[/url]

Can anyone tell me which one will work at colab?


All times are UTC. The time now is 22:43.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.