Google Diet Colab Notebook

[kriesel]

Originally Posted by petrw1

9 days later and I still cannot get these 2 sessions to start ... they still say I have reached my compute limit.

I created a new email address 5 days ago and by day I've been given:
10 hours
7.5
6
4.5
3.5 hours today

A few thoughts:
Colab Pro is allegedly available in US and Canada, perhaps a workaround for you if you're continually stymied in Colab free.
Posting here a screen shot of what is blocking you might be helpful.
Try one of the approaches to report an issue or get help here

[chalsall]

Originally Posted by petrw1

9 days later and I still cannot get these 2 sessions to start ... they still say I have reached my compute limit.

Quick datapoint from my Instances PoV...

I haven't received a single GPU instance in over a week. I do continue to get CPU, but they tend to run for about 20 minutes. They will sometimes last up to seven hours though; no idea what variables influence this.

I /do/ continue to see some people get GPU resources for the GPU72 Notebook. I think the paid-tier still gets some reasonable access, but even some free-tier who haven't done any work recently can also get some compute (at least, for a while).

[petrw1]

https://github.com/googlecolab/colabtools/issues/1964

[Uncwilly]

Originally Posted by chalsall

I haven't received a single GPU instance in over a week. I do continue to get CPU, but they tend to run for about 20 minutes. They will sometimes last up to seven hours though; no idea what variables influence this.

Same here.

[LaurV]

Hey Chris, can you change your comms.pl to have "$NumberToHold" be 2 or 3 instead of 1?

The story is that since you implemented the "worktodo.add" feature, and limited the "max assignments to hold at a time" to 1, the things go like that:

1. we get one assignment, and start working at it.
2. comms check and we have one assignment, it does nothing.
3. we finish the assignment and worktodo is empty, nothing to do for mfaktc, which exits.
4. comms see that after a while (seconds) and bring a new assignment, saved into worktodo.add file (?!?!?)
5. boot starts a new copy of mfaktc (showing again "107 tests passed" etc, this is how we discover the issue) and the .add is moved to worktodo
6. repeat from 2.

This wastes a lot of time, especially for "fast" (low bitlevel) assignments (and lucky users who get fast GPUs). The steps 4 and 5 are not needed, and they waste anything between few % and up to 30% of the time! (no joke, for a good card and low-bitlvel assignments, you go through them very fast, one assignment for every 3-4 minutes, and if your mfaktc is stopped one minute in between... - see my last reports to know I am not joking!)

I understand the fear people can get a lot of assignments then block them because they get bored or Colab cuts their resources, but you should allow us to hold at least 3 assignments at a time, for every instance.

That is because when you allow more than 2 assignments at a time, the process goes like that:

1. we get one assignment, and start working at it.
2. comms check and see we don't have 3 lines in worktodo, therefore it brings another 1 or 2 assignments that are saved in "worktodo.add"
3. at the next screen output mfaktc sees the .add file and adds the assignments to worktodo.txt file
4. we finish the assignment and it is deleted from the worktodo, but the file is not empty, mfaktc continues with the assignments and does NOT exit. (therefore does NOT start again next time, doing again the detection, running the 107 tests again, etc.)
5. repeat from 2.

At the current assignments we get, we need about 12-15 minutes for one, and we waste one minute to do the restart, detection, run 107 tests again. If we modified "$NumberToHold" to 3, the assignments are turned-over about 7% faster. This works brilliantly, until we restart the instance, every few hours, and then the "bootstrap" tar/zip file is downloaded again, in the hope you have new updates (which you never have ), and that unpacks a new comms.pl, and brings back the variable to 1. And we need to edit it again to 3... Grrr...

Please change it to 3, so we can continue to be friends

[petrw1]

Up until a couple weeks ago my 2 mail addresses were each getting 6 - 8 hours per day.
I think I was found guilty of:

What are the usage limits of Colab?
Colab is able to provide free resources in part by having dynamic usage limits that 
sometimes fluctuate, and by not providing guaranteed or unlimited resources. This 
means that overall usage limits as well as idle timeout periods, maximum VM 
lifetime, GPU types available, and other factors vary over time. Colab does not 
publish these limits, in part because they can (and sometimes do) vary quickly.

GPUs and TPUs are sometimes prioritized for users who use Colab interactively rather 
than for long-running computations, or for users who have recently used
 less resources in Colab. As a result, users who use Colab for long-running 
computations, or users who have recently used more resources in Colab, are more likely 
to run into usage limits and have their access to GPUs and TPUs temporarily restricted.
 Users with high computational needs may be interested in using Colab’s UI with 
a local runtime running on their own hardware. Users interested in having higher and 
more stable usage limits may be interested in Colab Pro.

[petrw1]

My first 2 emails were finally allowed back in yesterday...but less than 2 hours per day each (had been 6-8 hours).
My new email is getting about 4 hours per day.

[moebius]

runs good for me, because I couldn't use colab in the last 6 months. Three 6 to 8 hours runtimes after the other with the same google account

2021-05-07 18:02:41 gpuowl v6.11-380-g79ea0cc
2021-05-07 18:02:41 Note: not found 'config.txt'
2021-05-07 18:02:42 config: -carry short -use CARRY32,ORIG_SLOWTRIG,IN_WG=128,IN_SIZEX=16,IN_SPACING=4,OUT_WG=128,OUT_SIZEX=16,OUT_SPACING=4 -nospin -block 100 -maxAlloc 10000 -B1 750000 -rB2 20 
2021-05-07 18:02:42 device 0, unique id ''
2021-05-07 18:02:42 Tesla P100-PCIE-16GB-0 110427151 FFT: 6M 1K:12:256 (17.55 bpw)
2021-05-07 18:02:42 Tesla P100-PCIE-16GB-0 Expected maximum carry32: 36300000
2021-05-07 18:02:43 Tesla P100-PCIE-16GB-0 OpenCL args "-DEXP=110427151u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=12u -DPM1=0 -DWEIGHT_STEP_MINUS_1=0x1.74f6446ccaf7ep-2 -DIWEIGHT_STEP_MINUS_1=-0x1.116367ff04dc1p-2 -DCARRY32=1 -DIN_SIZEX=16 -DIN_SPACING=4 -DIN_WG=128 -DORIG_SLOWTRIG=1 -DOUT_SIZEX=16 -DOUT_SPACING=4 -DOUT_WG=128  -cl-unsafe-math-optimizations -cl-std=CL2.0 -cl-finite-math-only "
2021-05-07 18:02:46 Tesla P100-PCIE-16GB-0 

2021-05-07 18:02:46 Tesla P100-PCIE-16GB-0 OpenCL compilation in 2.62 s
2021-05-07 18:02:49 Tesla P100-PCIE-16GB-0 110427151 OK 29000000 loaded: blockSize 100, eedbba2c5d82d2ee
2021-05-07 18:02:49 Tesla P100-PCIE-16GB-0 validating proof residues for power 8
2021-05-07 18:05:14 Tesla P100-PCIE-16GB-0 Proof using power 8
2021-05-07 18:05:15 Tesla P100-PCIE-16GB-0 110427151 OK 29000200  26.26%; 1058 us/it; ETA 0d 23:56; e957c29f2567ba06 (check 0.26s)
2021-05-07 18:08:40 Tesla P100-PCIE-16GB-0 110427151 OK 29200000  26.44%; 1027 us/it; ETA 0d 23:10; f654a8428a3ae07a (check 0.27s)
2021-05-07 18:12:06 Tesla P100-PCIE-16GB-0 110427151 OK 29400000  26.62%; 1028 us/it; ETA 0d 23:08; a65813102ea4d1e6 (check 0.28s)
2021-05-07 18:15:32 Tesla P100-PCIE-16GB-0 110427151 OK 29600000  26.80%; 1026 us/it; ETA 0d 23:03; 85fd670853e6cce6 (check 0.26s)
2021-05-07 18:18:57 Tesla P100-PCIE-16GB-0 110427151 OK 29800000  26.99%; 1028 us/it; ETA 0d 23:01; 5d75874b5ba97778 (check 0.26s)
2021-05-07 18:22:23 Tesla P100-PCIE-16GB-0 110427151 OK 30000000  27.17%; 1026 us/it; ETA 0d 22:56; df49346508d9cb02 (check 0.26s)
2021-05-07 18:25:49 Tesla P100-PCIE-16GB-0 110427151 OK 30200000  27.35%; 1027 us/it; ETA 0d 22:54; 923e3e695e9a0902 (check 0.27s)
2021-05-07 18:29:14 Tesla P100-PCIE-16GB-0 110427151 OK 30400000  27.53%; 1026 us/it; ETA 0d 22:49; a9d43bf0f2bcc966 (check 0.26s)
2021-05-07 18:32:40 Tesla P100-PCIE-16GB-0 110427151 OK 30600000  27.71%; 1026 us/it; ETA 0d 22:46; 681c2891ad235acb (check 0.28s)
2021-05-07 18:36:06 Tesla P100-PCIE-16GB-0 110427151 OK 30800000  27.89%; 1028 us/it; ETA 0d 22:44; 8fabc62a76ffb67a (check 0.27s)
2021-05-07 18:39:31 Tesla P100-PCIE-16GB-0 110427151 OK 31000000  28.07%; 1026 us/it; ETA 0d 22:39; 49db68c7e227f4c0 (check 0.27s)
2021-05-07 18:42:57 Tesla P100-PCIE-16GB-0 110427151 OK 31200000  28.25%; 1028 us/it; ETA 0d 22:37; 665d57ca3fa08b94 (check 0.26s)
2021-05-07 18:46:23 Tesla P100-PCIE-16GB-0 110427151 OK 31400000  28.43%; 1026 us/it; ETA 0d 22:32; 2dc3302c66dbb4b6 (check 0.27s)
2021-05-07 18:49:48 Tesla P100-PCIE-16GB-0 110427151 OK 31600000  28.62%; 1028 us/it; ETA 0d 22:30; 102e2095af5ad30c (check 0.27s)
2021-05-07 18:53:14 Tesla P100-PCIE-16GB-0 110427151 OK 31800000  28.80%; 1026 us/it; ETA 0d 22:25; 865295b42f267796 (check 0.27s)
2021-05-07 18:56:40 Tesla P100-PCIE-16GB-0 110427151 OK 32000000  28.98%; 1028 us/it; ETA 0d 22:23; 070c4357172a9865 (check 0.27s)
2021-05-07 19:00:05 Tesla P100-PCIE-16GB-0 110427151 OK 32200000  29.16%; 1026 us/it; ETA 0d 22:18; b0d66ec071632079 (check 0.26s)
2021-05-07 19:03:31 Tesla P100-PCIE-16GB-0 110427151 OK 32400000  29.34%; 1027 us/it; ETA 0d 22:16; 529d3092dae824d2 (check 0.27s)
2021-05-07 19:06:56 Tesla P100-PCIE-16GB-0 110427151 OK 32600000  29.52%; 1026 us/it; ETA 0d 22:11; 60410928d70964a4 (check 0.27s)
2021-05-07 19:10:22 Tesla P100-PCIE-16GB-0 110427151 OK 32800000  29.70%; 1028 us/it; ETA 0d 22:10; 4544b3f5a17b12a5 (check 0.27s)
2021-05-07 19:13:48 Tesla P100-PCIE-16GB-0 110427151 OK 33000000  29.88%; 1026 us/it; ETA 0d 22:05; 1a5a32fdf2003aa1 (check 0.26s)
2021-05-07 19:17:13 Tesla P100-PCIE-16GB-0 110427151 OK 33200000  30.07%; 1026 us/it; ETA 0d 22:01; 66b96c7c073d1dd1 (check 0.26s)
2021-05-07 19:20:39 Tesla P100-PCIE-16GB-0 110427151 OK 33400000  30.25%; 1027 us/it; ETA 0d 21:59; 4b20455ae34e0cff (check 0.26s)
2021-05-07 19:24:05 Tesla P100-PCIE-16GB-0 110427151 OK 33600000  30.43%; 1026 us/it; ETA 0d 21:54; caffbd4152c7aedb (check 0.27s)
2021-05-07 19:27:30 Tesla P100-PCIE-16GB-0 110427151 OK 33800000  30.61%; 1027 us/it; ETA 0d 21:52; 47a99fbb962744b2 (check 0.26s)
2021-05-07 19:30:56 Tesla P100-PCIE-16GB-0 110427151 OK 34000000  30.79%; 1026 us/it; ETA 0d 21:47; 89c6ee652cbc1c37 (check 0.28s)
2021-05-07 19:34:22 Tesla P100-PCIE-16GB-0 110427151 OK 34200000  30.97%; 1027 us/it; ETA 0d 21:45; 00aedc9b2a8de5a5 (check 0.26s)
2021-05-07 19:37:47 Tesla P100-PCIE-16GB-0 110427151 OK 34400000  31.15%; 1026 us/it; ETA 0d 21:41; 27a86e46ee74a3ce (check 0.27s)
2021-05-07 19:41:13 Tesla P100-PCIE-16GB-0 110427151 OK 34600000  31.33%; 1028 us/it; ETA 0d 21:39; 9d143f743537d6e7 (check 0.27s)
2021-05-07 19:44:39 Tesla P100-PCIE-16GB-0 110427151 OK 34800000  31.51%; 1026 us/it; ETA 0d 21:34; d6f3a304f69067ad (check 0.27s)
2021-05-07 19:48:04 Tesla P100-PCIE-16GB-0 110427151 OK 35000000  31.70%; 1027 us/it; ETA 0d 21:32; 550228e87f5eb803 (check 0.26s)
2021-05-07 19:51:30 Tesla P100-PCIE-16GB-0 110427151 OK 35200000  31.88%; 1026 us/it; ETA 0d 21:27; f0c4e46d7109934b (check 0.28s)
2021-05-07 19:54:56 Tesla P100-PCIE-16GB-0 110427151 OK 35400000  32.06%; 1028 us/it; ETA 0d 21:25; 0cc94360b78d03dc (check 0.26s)
2021-05-07 19:58:21 Tesla P100-PCIE-16GB-0 110427151 OK 35600000  32.24%; 1026 us/it; ETA 0d 21:20; b80056a683a47c06 (check 0.27s)
2021-05-07 20:01:47 Tesla P100-PCIE-16GB-0 110427151 OK 35800000  32.42%; 1026 us/it; ETA 0d 21:17; 286d37c508a095b0 (check 0.27s)
2021-05-07 20:05:12 Tesla P100-PCIE-16GB-0 110427151 OK 36000000  32.60%; 1027 us/it; ETA 0d 21:15; 766527cf4e285517 (check 0.27s)
2021-05-07 20:08:38 Tesla P100-PCIE-16GB-0 110427151 OK 36200000  32.78%; 1026 us/it; ETA 0d 21:10; 46e83faf57aed4cf (check 0.27s)
2021-05-07 20:12:04 Tesla P100-PCIE-16GB-0 110427151 OK 36400000  32.96%; 1027 us/it; ETA 0d 21:08; a4de528c39e1988f (check 0.27s)
2021-05-07 20:15:29 Tesla P100-PCIE-16GB-0 110427151 OK 36600000  33.14%; 1026 us/it; ETA 0d 21:03; ec3c990d3a95a21d (check 0.27s)

[moebius]

Originally Posted by kriesel

A few thoughts:
Colab Pro is allegedly available in US and Canada, perhaps a workaround for you if you're continually stymied in Colab free.

ColabPro is available in Germany, UK, France, Brazil, Japan, India and Thailand too

[kriesel]

Originally Posted by moebius

ColabPro is available in Germany, UK, France, Brazil, Japan, India and Thailand too

Apparently that changed in March 2021, for Germany at least. Enjoy! https://github.com/googlecolab/colabtools/issues/1039
It would be good if Google advertised a page which listed nations in which Colab Pro is available. Estimated rollout dates would be a plus for "coming soon" listings.

[moebius]

This list provide the information.( in german at me)
https://colab.research.google.com/pro/terms