View Single Post
Old 2019-10-13, 15:18   #2
kriesel
 
kriesel's Avatar
 
"TF79LL86GIMPS96gpu17"
Mar 2017
US midwest

5,807 Posts
Default How to

(Derived beginning from chalsall's directions at https://www.mersenneforum.org/showpo...&postcount=194 and reading lots of other posts in that thread, followed by a lot of experimentation and head-scratching plus some help from PhilF)

Step by step:
  1. If the application script you plan to use makes use of files to be stored on your Google drive, and does not build the necessary content structure for you as the mprime script originated by Dylan14 does, put into a suitably named folder on your Google drive, all the files required to run the application on Ubuntu Linux:
    application file (Mfaktc, CUDAPm1, Gpuowl, CUDALucas, Mprime, whatever)
    a suitable cudart if applicable (Mfaktc, CUAPm1, CUDALucas)
    a suitable cufft if applicable (CUDALucas, CUDAPm1)
    ini file if needed
    worktodo.txt
    config.txt (Gpuowl optional), etc.
  2. In a modern web browser, such as the recommended Chrome, Firefox or Safari, or Microsoft Edge which worked in a test here (but not Internet Explorer) go to https://colab.research.google.com/no.../welcome.ipynb (Internet Explorer results in a message "This site may not work in your browser. Please use a supported browser. More info") It shouldn't matter what the web browser's host OS is. I've successfully run from Windows 7 x64 Pro, and Windows 10. If I recall correctly, others have run from Linux or a smartphone.
  3. On that same Welcome page, click on the "Connect" button in the upper right-hand corner.
  4. You will need to sign-in with your Google credentials. This connects your browser with a running Virtual Machine (VM) somewhere "in the cloud".
  5. Next, click on the "+ Code" button in the upper left-hand corner. This inserts a new "Code Section" into the "Notebook". You should see a "Play" (">") button (don't click it yet), and then to the immediate right a blinking vertical cursor inviting you to type in code.
  6. Paste in a notebook script for the application(s) you want to run. These may require some alteration. Make sure it includes connection to your google drive, and error detection. (Unless you don't mind everything staying on the VM and being lost after the run ends.)
    See either following posts in this thread, or https://www.mersenneforum.org/showthread.php?t=24646, post numbers by application:
    Mfaktc #2; 19
    perl 11
    misfit 25
    CUDAPm1 26; 158
    Gpuowl 29
    boinc 39
    cado-nfs 131; 137
    CUDALucas 178
    Mprime 208 (or use the build it from scratch script below)
    step by step howto with some system commands 194
    Mlucas 347
    GMP-ECM gpu flavor 729
    Note, for Mfaktc, due to a lot of console output, there are reports of increased browser memory consumption. It may be helpful to redirect its console output to a file on the VM drive or Google drive, or to nul. It has also been suggested to avoid low bit levels, which generate more frequent output.
  7. If it's a gpu application, at the upper left of the web page, select Edit, Notebook Settings, hardware accelerator, GPU, save.
  8. Now, you can click that "Play" button, and either provide information it requests (such as connecting to Google drive and providing the authorization code as in https://www.mersenneforum.org/showpo...4&postcount=12), watch it run, or read error messages and debug.
  9. Keep the browser tab around for the planned duration of the run. If you close the tab, the VM detects the disconnection and shuts down early. It's ok to minimize the whole browser while the notebook script runs on the Colab VM.
  10. Repeat the above, with appropriate application scripts, from step 5 onward, to occupy both the GPU and the cpu cores available in the Colab VM, to make maximal use of the expected 12 hour run time. This is done by lirst reconnecting to the Google drive if applicable, then launching the resume of one as a background task, and finally a section launching the other application, since it appears only one Code block can run on the notebook page at a time; not even !top or !nvidia-smi in another. See examples at https://www.mersenneforum.org/showpo...73&postcount=8
  11. Repeat as often as you like and can get resources allocated by Colaboratory. Note, one can run multiple Colab free sessions in separate VMs simultaneously with multiple google credential sets. As of last check, there is no limit to number of Google email accounts per person.
  12. To end a session before it times out, try stopping the code by clicking the "Play" button again, then at the upper left of the web page, Runtime, Manage Sessions, Terminate, and confirm
Currently, June 2021, run times I am seeing in Colab free are ~7 hours for CPU-only, ~4 hours for GPU&CPU, with occasional variation down to ~1.5 hours GPU&CPU or rarely up to 12 hours for CPU-only.

Beginning ~June 10 2021, I'm sometimes getting the following error. "Failed to execute cell. Could not send execute message to runtime: SyntaxError: An invalid or illegal string was specified." This occurs, 0, 1, 2, or several times per successful resumption of work with an existing notebook section. The notebook section, app, app configuration, and sometimes worktodo file have not changed between the error occurring or not. It would seem to be an intermittent error in Google Colaboratory.

Caveat: this is gleaned from two sources. The forum thread about Google Colaboratory https://www.mersenneforum.org/showthread.php?t=24646, and my own attempts to use it from Windows 7 or 10 systems. So it is only as right as I know how to make it.


Top of reference tree: https://www.mersenneforum.org/showpo...22&postcount=1

Last fiddled with by kriesel on 2021-09-26 at 13:48 Reason: added paragraph about occosional error message appearing recently
kriesel is offline