mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Cloud Computing (https://www.mersenneforum.org/forumdisplay.php?f=134)
-   -   Google Diet Colab Notebook (https://www.mersenneforum.org/showthread.php?t=24646)

kriesel 2019-10-11 17:59

[QUOTE=SimonVall00;525036]I have successfully run mfaktc in Google Colaboratory. I have created a Github repository with the Notebook i used.

Here is a link to the repository.
[URL]https://github.com/SimonVall00/mfaktc-google-colab[/URL][/QUOTE]

Step 5 in the readme.md says "Open mfaktc.ipynb in Google Colab"

Uh, how?
For that matter how does one open their own google drive in Colab?
From what URL in Colab, the welcome page?
Step by step in detail please.

chalsall 2019-10-11 17:59

[QUOTE=PhilF;527750]But I don't need any GPU resources, so maybe that is the difference.[/QUOTE]

Without question! GPUs are a highly coveted but scarce resource.

This is something that I don't think has been explored enough -- the CPUs which are available are fairly powerful, and readily available.

Heck, Kaggle will let you configure up and then batch run up to ten (10#) CPU instances for up to nine hours each. You can even close your browser, and they "just keep on going" (TM).

So long as they are able to autonomously report back the results to whatever server is appropriate, you're good-to-go.

chalsall 2019-10-11 18:04

[QUOTE=kriesel;527751]From what URL in Colab, the welcome page? Step by step in detail please.[/QUOTE]

Absolutely no disrespect towards Simon on this. This Notebook is what started this whole adventure! :tu:

But... You might find [URL="https://github.com/chalsall/GPU72_CoLab/blob/master/gpu72_tf.ipynb"]my Notebook on Github[/URL] a little easier to start with.

If Github is playing nice, you should see a "Open in Colaboratory" link, which automatically imports the Notebook into Colab. Then get an AKey [URL="https://www.gpu72.com/account/instances/new/"]here[/URL] (after signing up), and you're off to the races.

Uncwilly 2019-10-11 18:21

I have tried to close down my session and browser and restart before I leave work to avoid the 12 hour limit. But it does not seem to give me a full new 12. It seems to shut off about 12 hours after my morning spin up.

I did get my phone to spin up a session (used the "Hacker's Keyboard" app to get the cut and paste to work.)

kriesel 2019-10-11 18:38

2 Attachment(s)
[QUOTE=Dylan14;525400]Using the same type of script as chalsall did with mfaktc, I set up a script to handle CUDAPm1. Here's the procedure:
[LIST][*]In your Google Drive, create a folder called cudapm1[*]Download CUDAPm1-0.22-cuda10-linux from the download mirror at [URL]https://download.mersenne.ca/CUDAPm1[/URL], rename it to cudapm1-linux.exe and upload it to the cudapm1 folder you made[*](Optional) Provide a CudaPm1.ini file in the same folder[*]Run the following code in the notebook:[/LIST][CODE]import os.path
from google.colab import drive

if not os.path.exists('/content/drive/My Drive'):
drive.mount('/content/drive')

%cd '/content/drive/My Drive/cudapm1//'

#CUDAPm1 executable from https://download.mersenne.ca/CUDAPm1/CUDAPm1-0.22-cuda10-linux
!cp 'cudapm1-linux.exe' /usr/local/bin/
!chmod 755 '/usr/local/bin/cudapm1-linux.exe'

!mkdir /usr/local/bin/lib/
!cp 'lib/libcudart.so.6.5' /usr/local/bin/lib/

!cd '.' && LD_LIBRARY_PATH="lib:${LD_LIBRARY_PATH}" /usr/local/bin/cudapm1-linux.exe -selftest[/CODE]and the self test will run.
I got one of the T4's, and it is fast! Attached is the results from the cufft benchmark.[/QUOTE]
Hmmm. It's not clear where that libcudart.so.6.5 is supposed to come from since the web browser's host system is Windows 7. But there are other issues. The colab session can't see the files on the google drive that are there.

It's quite likely that I'm missing something, perhaps something really obvious to others. I've read this thread twice but there are clearly holes in my understanding.

kriesel 2019-10-11 18:50

1 Attachment(s)
[QUOTE=chalsall;527753]You might find [URL="https://github.com/chalsall/GPU72_CoLab/blob/master/gpu72_tf.ipynb"]my Notebook on Github[/URL] a little easier to start with.

If Github is playing nice, you should see a "Open in Colaboratory" link, which automatically imports the Notebook into Colab. Then get an AKey [URL="https://www.gpu72.com/account/instances/new/"]here[/URL] (after signing up), and you're off to the races.[/QUOTE]Nope. Apparently it is not playing nice, for me. "Open" does not appear as part of the GitHub page; Firefox Edit, Find in this page does not find it.
Note that I am trying to do Colab for direct manual primenet operation, not through GPU72.


Edit: "Sorry, something went wrong. [URL="https://github.com/chalsall/GPU72_CoLab/blob/master/gpu72_tf.ipynb"]Reload?[/URL]". Six times in six attempts. And my internet connection speed tests ok, and the recalcitrant file is indicated as a mere 2.5KB. I guess this means I should go paint my house window frames for a while.

chalsall 2019-10-11 18:55

[QUOTE=kriesel;527758]Nope. Apparently it is not playing nice, for me. "Open" does not appear as part of the GitHub page; Firefox Edit, Find in this page does not find it.[/QUOTE]

Hmmm... Yeah, I would sometimes see that kinda thing as well, even with Javascript enabled for that site. I was wondering if it was just me.

[QUOTE=kriesel;527758]Note that I am trying to do Colab for direct manual primenet operation, not through GPU72.[/QUOTE]

Copy that. Dylan14 has presented many really good blocks of Python which can be directly cut-and-pasted into a running Notebook (Colab and/or Kaggle). Note that you have to be sure all needed libraries are in the instance -- apt-get them if not.

There are some differences in what is available by default between Colab and Kaggle.

chalsall 2019-10-11 19:00

[QUOTE=Uncwilly;527754]I did get my phone to spin up a session (used the "Hacker's Keyboard" app to get the cut and paste to work.)[/QUOTE]

:bow:

May we live in interesting times... :smile:

chalsall 2019-10-11 20:04

[QUOTE=chalsall;527748]Today I'm running an experiment wrt possible daily quotas per account... One account is running one instance, the other two.[/QUOTE]

OK, I just noticed this... The second account's two instances were both killed at the same time -- within a minute of each other.

First had run 6:06. The second 5:46. Can't relaunch GPU attached instances.

The first account's singular instance continues nominally.

kriesel 2019-10-11 21:16

[QUOTE=Dylan14;525400]Using the same type of script as chalsall did with mfaktc, I set up a script to handle CUDAPm1. Here's the procedure:
[LIST][*]In your Google Drive, create a folder called cudapm1[*]Download CUDAPm1-0.22-cuda10-linux from the download mirror at [URL]https://download.mersenne.ca/CUDAPm1[/URL], rename it to cudapm1-linux.exe and upload it to the cudapm1 folder you made[*](Optional) Provide a CudaPm1.ini file in the same folder[*]Run the following code in the notebook:[/LIST][CODE]import os.path
from google.colab import drive

if not os.path.exists('/content/drive/My Drive'):
drive.mount('/content/drive')

%cd '/content/drive/My Drive/cudapm1//'

#CUDAPm1 executable from https://download.mersenne.ca/CUDAPm1/CUDAPm1-0.22-cuda10-linux
!cp 'cudapm1-linux.exe' /usr/local/bin/
!chmod 755 '/usr/local/bin/cudapm1-linux.exe'

!mkdir /usr/local/bin/lib/
!cp 'lib/libcudart.so.6.5' /usr/local/bin/lib/

!cd '.' && LD_LIBRARY_PATH="lib:${LD_LIBRARY_PATH}" /usr/local/bin/cudapm1-linux.exe -selftest[/CODE]and the self test will run.
I got one of the T4's, and it is fast! Attached is the results from the cufft benchmark.[/QUOTE]
Wouldn't it also need libcufft.so.6.5.14?

chalsall 2019-10-11 21:42

Just got official word back from Kaggle. We're good!
 
So, way back in mid-September, I reached out to both Colab and Kaggle, trying to get some clarity on "fair use". I pointed to the beginning of this thread; just got the official word back:

[QUOTE]Thank you for your patience. The linked thread provided doesn't describe in adequate detail what your team intends to do. The discussion is mainly from a collaboration aspect, and the link is to a private GitHub repo.

However, if your team is trying to identify Mersenne prime numbers by using Kaggle GPUs, that is totally fine. [/QUOTE]

:smile:


All times are UTC. The time now is 22:30.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.