Google Diet Colab Notebook

[petrw1]

Originally Posted by chalsall

Taking a quick stab in the dark... Your Subprocess() call appears to be passing all the parameters to CUDAPm1 as the file name.

Also, wrt your previous message, was your attempt to run "./CUDAPm1" within a Notebook section? If so, you need to do "!./CUDAPm1" instead, since it's within the Python shell.

OK making progress. Running this:

%cd '/content/drive/My Drive/cudapm1//'

# List files here
for x in os.listdir('.'):
print (x)

%cd '/content/drive/My Drive/cudapm1/cudapm1-0.20/'

# List files here too.
for x in os.listdir('.'):
print (x)

# libcufft.so.5.5 is in the list above but in just in case I also copied it to the parent
import shutil
shutil.copyfile('libcufft.so.5.5', '/content/drive/My Drive/cudapm1/libcufft.so.5.5')

# RUN it!!!
!./CUDAPm1 61408363 -b1 60000 -b2 1200000 -f 3360k

/content/drive/My Drive/cudapm1
cudapm1-0.20
cudapm1-0.20.tar.gz
libcufft.so.5.5

/content/drive/My Drive/cudapm1/cudapm1-0.20
readme.txt
CUDAPm1.ini
libcufft.so.5.5
CUDAPm1
cudapm1-setup.txt

./CUDAPm1: error while loading shared libraries: libcufft.so.5.5: cannot open shared object file: No such file or directory

[Dylan14]

Originally Posted by petrw1

---> 20 subprocess.call("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1 61408363 -b1 600000 -b2 12000000 -f 3360k'")

When you use subprocess.call (or subprocess.run), it expects each parameter to be in its own string, separated by a comma. The way you have it, it is trying to run the whole thing as an executable, but that is clearly not a valid file. Try this instead:

subprocess.run(“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”)

[petrw1]

Originally Posted by Dylan14

When you use subprocess.call (or subprocess.run), it expects each parameter to be in its own string, separated by a comma. The way you have it, it is trying to run the whole thing as an executable, but that is clearly not a valid file. Try this instead:

subprocess.run(“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”)

import subprocess
subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k")

TypeError                                 Traceback (most recent call last)
<ipython-input-17-2cf284193cd6> in <module>()
     26 
     27 #subprocess.run("/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1")
---> 28 subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k")
     29 #subprocess.run("'/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'", "61408363 -b1 600000 -b2 12000000 -f 3360k")
     30 # run ./CUDAPm1 -cufftbench 1k to 65536k 2 times

1 frames
/usr/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
    627             bufsize = -1  # Restore default
    628         if not isinstance(bufsize, int):
--> 629             raise TypeError("bufsize must be an integer")
    630 
    631         if _mswindows:

TypeError: bufsize must be an integer

Here is the section of code from subprocess.py
bufsize is set to -1 then checked later for integer and fails???

def __init__(self, args, bufsize=-1, executable=None,
                 stdin=None, stdout=None, stderr=None,
                 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
                 shell=False, cwd=None, env=None, universal_newlines=False,
                 startupinfo=None, creationflags=0,
                 restore_signals=True, start_new_session=False,
                 pass_fds=(), *, encoding=None, errors=None):
        """Create new Popen instance."""
        _cleanup()
      
        self._waitpid_lock = threading.Lock()

        self._input = None
        self._communication_started = False
        if bufsize is None:
            bufsize = -1  # Restore default
        if not isinstance(bufsize, int):
            raise TypeError("bufsize must be an integer")

[Dylan14]

Originally Posted by petrw1

Here is the section of code from subprocess.py
bufsize is set to -1 then checked later for integer and fails???

def __init__(self, args, bufsize=-1, executable=None,
                 stdin=None, stdout=None, stderr=None,
                 preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS,
                 shell=False, cwd=None, env=None, universal_newlines=False,
                 startupinfo=None, creationflags=0,
                 restore_signals=True, start_new_session=False,
                 pass_fds=(), *, encoding=None, errors=None):
        """Create new Popen instance."""
        _cleanup()
      
        self._waitpid_lock = threading.Lock()

        self._input = None
        self._communication_started = False
        if bufsize is None:
            bufsize = -1  # Restore default
        if not isinstance(bufsize, int):
            raise TypeError("bufsize must be an integer")

I missed a key thing: the list of arguments have to be enclosed in brackets ([]).
So it should read

subprocess.run([“content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1”, “61408363”, “-b1”, “600000”, “-b2”, “120000000”, “-f”, “3360k”])

Without it, it's treating "61408363" as the argument for bufsize, which is a string, not an integer. And that's why it's breaking.

[petrw1]

I tried these 3 variations (One at a time all with the same error):

subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363 -b1 600000 -b2 12000000 -f 3360k"])

CompletedProcess(args=['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'], returncode=127)

How do I know if the 127 was from subprocess.run
OR
from CUDAPm1?

It it was from CUDAPm1 can someone familiar with it tell me what that code means,

Thx

[chalsall]

Originally Posted by petrw1

It it was from CUDAPm1 can someone familiar with it tell me what that code means,

Juggling a few balls at the moment, but I suspect that code is not good...

Your above "error while loading shared libraries: libcufft.so.5.5: cannot open shared object" suggests the executable doesn't like it's home. (Read: perhaps try compiling the code in it's native environment.)

[Dylan14]

Originally Posted by petrw1

I tried these 3 variations (One at a time all with the same error):

subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363", "-b1", "600000", "-b2", "12000000", "-f", "3360k"])
subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1", "61408363 -b1 600000 -b2 12000000 -f 3360k"])

CompletedProcess(args=['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1'], returncode=127)

How do I know if the 127 was from subprocess.run
OR
from CUDAPm1?

It it was from CUDAPm1 can someone familiar with it tell me what that code means,

Thx

The return code comes from subprocess.run. Ideally, it should be 0, which means the program ran with no problems. Here, we got a non zero code, which means something went wrong when python tried to execute the program. In this case, something went wrong in executing cudapm1, and thus the process ended, and python returns the error code in a CompletedProcess object.
You might get more information if you add the keyword argument “check=True” to the run command (without the quotes).

[petrw1]

Originally Posted by Dylan14

The return code comes from subprocess.run. Ideally, it should be 0, which means the program ran with no problems. Here, we got a non zero code, which means something went wrong when python tried to execute the program. In this case, something went wrong in executing cudapm1, and thus the process ended, and python returns the error code in a CompletedProcess object.
You might get more information if you add the keyword argument “check=True” to the run command (without the quotes).

CalledProcessError                        Traceback (most recent call last)
<ipython-input-28-d81f224b82fc> in <module>()
     25 #import subprocess
     26 
---> 27 subprocess.run(["/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1"],check=True)
     
/usr/lib/python3.6/subprocess.py in run(input, timeout, check, *popenargs, **kwargs)
    436         if check and retcode:
    437             raise CalledProcessError(retcode, process.args,
--> 438                                      output=stdout, stderr=stderr)
    439     return CompletedProcess(process.args, retcode, stdout, stderr)
    440 

CalledProcessError: Command '['/content/drive/My Drive/cudapm1/cudapm1-0.20/CUDAPm1']' returned non-zero exit status 127.

[Uncwilly]

Anyone else think that Colab and the rest of google and amazon, M$, etc should through all their spare cycles at folding@home.

[kriesel]

Originally Posted by Uncwilly

Anyone else think that Colab and the rest of google and amazon, M$, etc should through all their spare cycles at folding@home.

Folding@home supposedly has 470 Petaflops as is. To put that in perspective, it's over 1200 times the stated throughput of GIMPS at https://www.mersenne.org/primenet/ (384 Tflops)

reference: https://www.techspot.com/news/84492-...stributed.html
Which, I suppose, suggests there's room for GIMPS to grow, after the current COVID19 unpleasantness is behind us.

[petrw1]

https://download.mersenne.ca/CUDAPm1/

Can anyone tell me which one will work at colab?