mersenneforum.org

mersenneforum.org (https://www.mersenneforum.org/index.php)
-   Cloud Computing (https://www.mersenneforum.org/forumdisplay.php?f=134)
-   -   Google Diet Colab Notebook (https://www.mersenneforum.org/showthread.php?t=24646)

bayanne 2020-07-02 14:19

[QUOTE=bayanne;549592]Completed exponents are not being reported from Colab ...

Can you advise?[/QUOTE]

Seems to be sorted now :smile:

chalsall 2020-07-02 22:06

[QUOTE=bayanne;549613]Seems to be sorted now :smile:[/QUOTE]

Yeah; sorry. Error between keyboard and chair...

Stupidly busy at the moment. I check in on GPU72 several times a day to make sure things are sane, so if you see something strange please wait until 1200 UTC, and if things don't magically sort themselves out, please PM me.

chalsall 2020-07-02 22:12

[QUOTE=moebius;549602]Hope they stay there persistent. Maybe something has been changed in the configuration of colab.[/QUOTE]

My ***very*** early prototyping with Colab (only last year did we discovered this resource!) suggests they will (or, at least, should).

My recollection is copying files from Google Drive to the local Colab filesystem (read "/") can have quite a bit of latency. But writing to the Drive from executing code in the VM seemed to be relatively quick and reliable.

Note that I haven't actually mounted a Drive in a Colab instance for many months (it doesn't scale for what I do with them (and other instance environments)). I do know that there are many here who *do* regularly do this now without complaint.

moebius 2020-07-04 17:09

[QUOTE=moebius;549602]I have now logged in with a different Google account and lo and behold, the save....[/QUOTE]

note: it is better to change the directory with chdir(), because in some cases, a cd leaves the current working directory unchanged....

[SIZE="1"]import os.path
from google.colab import drive
if not os.path.exists('/content/drive/My Drive'):
drive.mount('/content/drive')
%cd '/content/drive/My Drive/gpuowl-master/'
[COLOR="Lime"]os.chdir ('/content/drive/My Drive/gpuowl-master/')[/COLOR]
!chmod 755 '/content/drive/My Drive/gpuowl-master/gpuowl.exe'
!ls -la
!cd '.' && /content/drive/My\ Drive/gpuowl-master/gpuowl.exe -prp 333XXXXXX[/SIZE]

chalsall 2020-07-04 17:47

[QUOTE=moebius;549759]note: it is better to change the directory with chdir(), because in some cases, a cd leaves the current working directory unchanged....[/QUOTE]

So long as you use "%cd" it should be persistent (within the context of the Python script). Using only "cd", on the other hand, does effectively nothing.

There's no downside in using chdir() though.

lycorn 2020-07-17 12:21

Is it just my impression or Colab is tighthning the assignment rules for GPUs?
It´s getting more and more difficult to get a GPU for a decent (say, more than 2-3 hours, sometimes much less, very seldom more than six) period of time, and most of the times I get K80s, even after "Factory Resettting" the runtime. In the rare situation I get a P100 or a T4, the runtimes are normally very short.
Are more people experiencing this "shortage"?

kriesel 2020-07-17 13:50

[QUOTE=lycorn;550835]Is it just my impression or Colab is tightening the assignment rules for GPUs?
It´s getting more and more difficult to get a GPU for a decent (say, more than 2-3 hours, sometimes much less, very seldom more than six) period of time, and most of the times I get K80s, even after "Factory Resetting" the runtime. In the rare situation I get a P100 or a T4, the runtimes are normally very short.
Are more people experiencing this "shortage"?[/QUOTE]Paid or free?
As a Colab free user, I'm seeing gpus usually unavailable entirely, and when, slightly less than daily, I get one per user account, they last 90 minutes or as little as 20 minutes. Sometimes even cpu-only sessions are a few hours instead of the nominal 12. I think free users can expect to get preempted whenever there's paid use demand sufficient to do so, and we're seeing the effect of increased demand. It's really messing with the latency of GIMPS assignments in TF and P-1. I have different working directories with worktodo files for the different gpu models so that -maxAlloc is different. It can be weeks between appearances of a given gpu model on a given account.
At times I've resorted to removing assignments from Colab to run them on a local gpu, to keep assignments from expiring or wavefronts from stalling.

lycorn 2020-07-17 15:07

Ok, so it´s definitely not "my" problem...
I am also a free Colab user. Have 5 different google accounts, and the issues I´m having are very similar to yours. The main difference is the availabilty of CPUs, which is 100% (every time I want one I get one, and the sessions last the full 12 hours).
I think you are right in that paid users have higher priority access to resources, (which makes a whole lot of sense, absolutely! :smile:)
As I only use Colab for work that doesn´t impact in any way the GIMPS workflow, I don´t care too much. I take what I am given and feel grateful for it (isn´t this a nice sentence, by the way?...:cool:)

LaurV 2020-07-18 03:12

Same here. For a while, I ignored the "factory reset" trick because any time I can get a T4 or P100, they last from 20-40 minutes to 2-3 hours, while the k80s tend to last 6-10 hours. Especially before going to bed, if I am lucky enough to get a k80, then I won't make any effort to "reset" it and just let it run. It may produce more work till morning than a T4 in 30 minutes or one hour.

To be clear, I only run one account (my main), and that only when I remember. I fully understand that this is a resource which is supplied for free, and any time they need to sell it, i.e. there is a buyer, they can take it back and give it to that guy. I have no reason to complain.

moebius 2020-07-18 14:24

[QUOTE=kriesel;550839]..... I'm seeing gpus usually unavailable entirely, and when, slightly less than daily, I get one per user account, they last 90 minutes or as little as 20 minutes. [/QUOTE]

[SIZE="2"][B]If you use 3 google accounts one after the other (not at the same time) you can definitely calculate a 100M digit exponent continuously with Tesla P100's without a break. I use 3 different notebooks for this purpose. The running time is approximately 12 hours each. However, I've only been using colab since 3 weeks, maybe that's the cause. [/B][/SIZE]

[URL="https://www.mersenne.org/report_exponent/?exp_lo=333698863&full=1"]333698863[/URL]

lycorn 2020-07-18 15:42

[QUOTE=moebius;550942]However, I've only been using colab since 3 weeks, maybe that's the cause. [/QUOTE]

Yes, that should be the reason. Get it while you can... :smile:


All times are UTC. The time now is 22:02.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.