[QUOTE=xx005fs;526913]I suspect that it's a gcc version issue but i can't find gcc8 in the package list no matter if I add ubuntu-toolchain-r/test repository or not.[/QUOTE]

Wetware can be annoying. It will sometimes keep working on a problem, even though you've asked it to work on something different... :wink:

This suddenly rang a bell -- I encountered this natively building hansl's mfaktc package provided [URL="https://www.mersenneforum.org/showpost.php?p=525178&postcount=3189"]here[/URL] on Colab.

To solve this build issue, I simply edited the Makefile, changing CC = gcc-8 to be CC = gcc -- I also had to change the cuda path to be /usr/local/cuda (generic rather than specific path).

I have no idea for the reason for the version-specific gcc request. Perhaps a simple symlink before beginning builds would solve the problem?

@hansl: Any thoughts?

Checkpoint question
 

Are checkpoints picked up only from the platform where the work was started? I am asking if a Kaggle TF checkpoint file would ever be restarted on a Colab instance or vice versa.

[QUOTE=Chuck;526928]Are checkpoints picked up only from the platform where the work was started? I am asking if a Kaggle TF checkpoint file would ever be restarted on a Colab instance or vice versa.[/QUOTE]

Yup. They are completely cross-compatible.

As currently coded, twelve hours after the last update they will be handed out to the next requesting instance, to finish off.

I've updated the CudaPm1 script that I wrote a while back. Now you can interactively add work to worktodo.txt before starting the program.


[CODE]#@title Default title text
import os.path
from google.colab import drive

if not os.path.exists('/content/drive/My Drive'):
drive.mount('/content/drive')

%cd '/content/drive/My Drive/cudapm1//'
#look for a worktodo.txt
if os.path.exists('/content/drive/My Drive/cudapm1/worktodo.txt'):
print("Worktodo.txt found.")
f = open("worktodo.txt","a+")
no_more_work = 0
while(no_more_work == 0):
work = input("Enter a valid assignment: ")
if work == "": #empty string
#assume the user doesn't want to add work and break
break
else:
f.write(work)
ans = input("Do you want to add more work? (press 0 or 1)")
if ans == "1":
continue
elif ans == "0":
no_more_work = 1
else:
raise ValueError("Expected 0 or 1, got " + str(ans))
else: #no worktodo.txt
print("Worktodo.txt not found, creating it...")
f = open("worktodo.txt","a+")
no_more_work == 0
while(no_more_work == 0):
work = input("Enter a valid assignment: ")
ans = input("Do you want to add more work? (press 0 or 1)")
if ans == "1":
continue
elif ans == "0":
no_more_work = 1
else:
raise ValueError("Expected 0 or 1, got " + str(ans))

#Execute CudaPm1 once we have finished gathering work
print("Starting work...")
#CUDAPm1 executable from https://download.mersenne.ca/CUDAPm1/CUDAPm1-0.22-cuda10-linux
!cp 'cudapm1-linux.exe' /usr/local/bin/
!chmod 755 '/usr/local/bin/cudapm1-linux.exe'

!mkdir /usr/local/bin/lib/
!cp 'lib/libcudart.so.6.5' /usr/local/bin/lib/
!cp '/content/drive/My Drive/cudapm1/worktodo.txt'

!cd '.' && LD_LIBRARY_PATH="lib:${LD_LIBRARY_PATH}" /usr/local/bin/cudapm1-linux.exe
!cat 'results.txt'
!rm 'results.txt'[/CODE]

[QUOTE=Dylan14;527031]I've updated the CudaPm1 script that I wrote a while back. Now you can interactively add work to worktodo.txt before starting the program.[/QUOTE]

Cool addition! :tu:

Where are the checkpoint files?
 

Where on my Google drive are the Colab checkpoint files?

[QUOTE=Chuck;527090]Where on my Google drive are the Colab checkpoint files?[/QUOTE]See attached image:

And a little suggestion:
Maybe add a direct link to the Colab results page in addition to the main link to GPUto72

[QUOTE=Chuck;527090]Where on my Google drive are the Colab checkpoint files?[/QUOTE]

Hey gang! Sorry -- just got back at the console. I was at a conference all day today.

One of the design decisions I made early on was to not require the use of a participant's Google Drive for this to work. The reasons are:

1. It's an additional step (and trust level) that the participant must do ("Click this link... Copy and paste this code... Confirm that you understand the risks... Etc...") to "spin up" an instance.

2. Secondly, and I consider this probably more important: there is no way to constrain what part of Drive is "attached" to any instance requesting access. You either give access to your "root" ("/" directory in *nix parlance), or you give access to nothing. As in, you simply don't "attach".

2.1. IMO, it would be a very useful extension of the Google Drive "attaching functionality" would be to be able to give access to only subdirectories (AKA "folders") of a Drive.

2.2. As it is, ***any*** "Notebook" could _theoretically_ run something like "!rm -rf /" somewhere in its code, wiping out a user's entire Drive.

2.2.1. I haven't actually tried this myself, to see if it would work. I don't currently have a Drive which is "throw-away" at the moment to test this.

[QUOTE=Uncwilly;527103]See attached image:[/QUOTE]

That is so cool!

This is sincere: you've just taught me something about the Colab interface: I didn't even realize that left side-panel (nominally hidden) was there, and that you could drill down on the instances' file-system graphically through the web-interface! In real-time!!!

Sweat!

[QUOTE=Uncwilly;527103]And a little suggestion: Maybe add a direct link to the Colab results page in addition to the main link to GPUto72[/QUOTE]

Umm... Where? On the output from the initial Bootstrap.pl startup?

Please bear with me at the moment -- I'm juggling a few balls at once at the moment (this is only one of several currently "in the air").

To be perfectly honest with everyone, this has kinda gotten away from me. When I started this experiment I had no idea just how much potential it truly has. In all kinds of spaces!

I have a plan of next steps in my workbook (Yes, I still actually manage my projects with pen and paper (and whiteboards -- many whiteboards)...) but I don't have the time to transcript them at the moment.

But, if I may please share (with some excitement) -- I met a very important person from the UWI (Barbados) campus today. We discussed this Colab / Kaggle thing for a bit...

It was quite wonderful watching ***his*** brain explode in front of me, as he internalized the educational opportunities of this!!! :tu: :smile:

[QUOTE=chalsall;527110]This is sincere: you've just taught me something about the Colab interface: I didn't even realize that left side-panel (nominally hidden) was there, and that you could drill down on the instances' file-system graphically through the web-interface![/quote]Glad to be of help.[quote]Umm... Where? On the output from the initial Bootstrap.pl startup?

Please bear with me at the moment -- I'm juggling a few balls at once at the moment (this is only one of several currently "in the air").[/quote]Yeah, there. But I know you are busy and it is not much of an effort for the user to do the work. Don't worry about. I tried to word it as just a passing idea.
:chalsall::george::farley:

[QUOTE=Chuck;527090]Where on my Google drive are the Colab checkpoint files?[/QUOTE]

One quick thing I was able to do just now on my "to do list" was to expose this data, at an individual account level.

For any GPU72'ers currently experimenting with the Colab / Kaggle thing, please see [URL="https://www.gpu72.com/account/cpoint/"]this new report[/URL].

Please note that if you're going to manually feed these checkpoint records into an instance for completion, mfaktc is a little finicky about the format of the M\d+.ckp files.

Specifically, single string; no CR/LF. When doing this kind of thing by hand, I had to use the "truncate" command to generate legal files after a "vi" or "echo".

YMMV.