|
[QUOTE=kriesel;529484][CODE]g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [/CODE]So, anticipating maybe the next recommendation would be, !apt install g++-8 did that, got [CODE]Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libstdc++-8-dev Suggested packages: g++-8-multilib gcc-8-doc libstdc++6-8-dbg libstdc++-8-doc The following NEW packages will be installed: g++-8 libstdc++-8-dev 0 upgraded, 2 newly installed, 0 to remove and 28 not upgraded. Need to get 11.6 MB of archives. After this operation, 45.2 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libstdc++-8-dev amd64 8.3.0-6ubuntu1~18.04.1 [1,543 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 g++-8 amd64 8.3.0-6ubuntu1~18.04.1 [10.1 MB] Fetched 11.6 MB in 3s (3,461 kB/s) Selecting previously unselected package libstdc++-8-dev:amd64. (Reading database ... 132925 files and directories currently installed.) Preparing to unpack .../libstdc++-8-dev_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking libstdc++-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package g++-8. Preparing to unpack .../g++-8_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking g++-8 (8.3.0-6ubuntu1~18.04.1) ... Setting up libstdc++-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Setting up g++-8 (8.3.0-6ubuntu1~18.04.1) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. echo \"`git describe --long --dirty --always`\" > version.new diff -q -N version.new version.inc >/dev/null || mv version.new version.inc echo Version: `cat version.inc` Version: "v6.11-11-gfaaa2f2-dirty" g++ -MT common.o -MMD -MP -MF .d/common.Td -Wall -O2 -std=c++17 -c -o common.o common.cpp common.cpp:14:10: fatal error: filesystem: No such file or directory #include <filesystem> ^~~~~~~~~~~~ compilation terminated. Makefile:30: recipe for target 'common.o' failed make: *** [common.o] Error 1[/CODE][/QUOTE] use g++-8 instead of gcc when compiling |
[QUOTE=kracker;529488]use g++-8 instead of gcc when compiling[/QUOTE]How?
Gpuowl's makefile is [CODE]CXXFLAGS = -Wall -O2 -std=c++17 LIBPATH = -L/opt/rocm/opencl/lib/x86_64 -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -L/c/Windows/System32 -L. LDFLAGS = -lstdc++fs -lOpenCL -lgmp -pthread ${LIBPATH} LINK = $(CXX) -o $@ ${OBJS} ${LDFLAGS} SRCS = Pm1Plan.cpp GmpUtil.cpp Worktodo.cpp common.cpp main.cpp Gpu.cpp clwrap.cpp Task.cpp checkpoint.cpp timeutil.cpp Args.cpp state.cpp Signal.cpp FFTConfig.cpp AllocTrac.cpp gpuowl-wrap.cpp OBJS = $(SRCS:%.cpp=%.o) DEPDIR := .d $(shell mkdir -p $(DEPDIR) >/dev/null) DEPFLAGS = -MT $@ -MMD -MP -MF $(DEPDIR)/$*.Td COMPILE.cc = $(CXX) $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c POSTCOMPILE = @mv -f $(DEPDIR)/$*.Td $(DEPDIR)/$*.d && touch $@ gpuowl: ${OBJS} ${LINK} gpuowl-win.exe: ${OBJS} ${LINK} -static strip $@ clean: rm -f ${OBJS} gpuowl gpuowl-win %.o : %.cpp %.o : %.cpp $(DEPDIR)/%.d gpuowl-wrap.cpp version.inc $(COMPILE.cc) $(OUTPUT_OPTION) $< $(POSTCOMPILE) $(DEPDIR)/%.d: ; .PRECIOUS: $(DEPDIR)/%.d version.inc: FORCE echo \"`git describe --long --dirty --always`\" > version.new diff -q -N version.new version.inc >/dev/null || mv version.new version.inc echo Version: `cat version.inc` gpuowl-wrap.cpp: gpuowl.cl head.txt tail.txt cat head.txt gpuowl.cl tail.txt > gpuowl-wrap.cpp FORCE: include $(wildcard $(patsubst %,$(DEPDIR)/%.d,$(basename $(SRCS)))) [/CODE] It's invoked by this bit of Colab script:[CODE]%cd '/content/drive/My Drive/gpuowl//' !git clone https://github.com/preda/gpuowl %cd '/content/drive/My Drive/gpuowl/gpuowl//' !apt install libgmp-dev !apt install gcc-8 !apt install g++-8 !g++ --version !make gpuowl[/CODE] |
1 Attachment(s)
[QUOTE=kriesel;529492]How?
Gpuowl's makefile is [CODE]CXXFLAGS = -Wall -O2 -std=c++17 LIBPATH = -L/opt/rocm/opencl/lib/x86_64 -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -L/c/Windows/System32 -L. LDFLAGS = -lstdc++fs -lOpenCL -lgmp -pthread ${LIBPATH} LINK = $(CXX) -o $@ ${OBJS} ${LDFLAGS} SRCS = Pm1Plan.cpp GmpUtil.cpp Worktodo.cpp common.cpp main.cpp Gpu.cpp clwrap.cpp Task.cpp checkpoint.cpp timeutil.cpp Args.cpp state.cpp Signal.cpp FFTConfig.cpp AllocTrac.cpp gpuowl-wrap.cpp OBJS = $(SRCS:%.cpp=%.o) DEPDIR := .d $(shell mkdir -p $(DEPDIR) >/dev/null) DEPFLAGS = -MT $@ -MMD -MP -MF $(DEPDIR)/$*.Td COMPILE.cc = $(CXX) $(DEPFLAGS) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c POSTCOMPILE = @mv -f $(DEPDIR)/$*.Td $(DEPDIR)/$*.d && touch $@ gpuowl: ${OBJS} ${LINK} gpuowl-win.exe: ${OBJS} ${LINK} -static strip $@ clean: rm -f ${OBJS} gpuowl gpuowl-win %.o : %.cpp %.o : %.cpp $(DEPDIR)/%.d gpuowl-wrap.cpp version.inc $(COMPILE.cc) $(OUTPUT_OPTION) $< $(POSTCOMPILE) $(DEPDIR)/%.d: ; .PRECIOUS: $(DEPDIR)/%.d version.inc: FORCE echo \"`git describe --long --dirty --always`\" > version.new diff -q -N version.new version.inc >/dev/null || mv version.new version.inc echo Version: `cat version.inc` gpuowl-wrap.cpp: gpuowl.cl head.txt tail.txt cat head.txt gpuowl.cl tail.txt > gpuowl-wrap.cpp FORCE: include $(wildcard $(patsubst %,$(DEPDIR)/%.d,$(basename $(SRCS)))) [/CODE] It's invoked by this bit of Colab script:[CODE]%cd '/content/drive/My Drive/gpuowl//' !git clone https://github.com/preda/gpuowl %cd '/content/drive/My Drive/gpuowl/gpuowl//' !apt install libgmp-dev !apt install gcc-8 !apt install g++-8 !g++ --version !make gpuowl[/CODE][/QUOTE] You have to make g++-8 and gcc-8 the default compiler for C/C++ programs for Linux. Instructions on how to do this can be found in [URL="https://askubuntu.com/questions/26498/how-to-choose-the-default-gcc-and-g-version"]the first answer for this question on Stack Exchange[/URL]. Alternatively, I have attached an executable for gpuowl that I compiled on a colab instance with the help of the ssh tunnel, if this doesn't work. |
[QUOTE=Dylan14;529495]You have to make g++-8 and gcc-8 the default compiler for C/C++ programs for Linux. Instructions on how to do this can be found in [URL="https://askubuntu.com/questions/26498/how-to-choose-the-default-gcc-and-g-version"]the first answer for this question on Stack Exchange[/URL].
Alternatively, I have attached an executable for gpuowl that I compiled on a colab instance with the help of the ssh tunnel, if this doesn't work.[/QUOTE]Thanks, I take that to mean in Colab context, [CODE]... !sudo update-alternatives --remove-all gcc !sudo update-alternatives --remove-all g++ !sudo apt-get install gcc-8 g++-8 !make gpuowl [/CODE]or rather, since Colab is always as root,[CODE]... !update-alternatives --remove-all gcc !update-alternatives --remove-all g++ !apt-get install gcc-8 g++-8 !make gpuowl[/CODE]Will give it a try at the next opportunity. I'm trying to arrive at the complete setup script for anyone to use on Colab, to build the latest gpuowl as needed. [URL]https://www.mersenneforum.org/showpost.php?p=527930&postcount=7[/URL] |
[CODE]!update-alternatives --remove-all gcc
!update-alternatives --remove-all g++ !apt-get install gcc-8 g++-8 !g++ --version !make gpuowl[/CODE]Somehow still keeps 7.4.0 around as default on Colab. [CODE]update-alternatives: error: no alternatives for gcc update-alternatives: error: no alternatives for g++ Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: cpp-8 libasan5 libgcc-8-dev libstdc++-8-dev libubsan1 Suggested packages: gcc-8-locales g++-8-multilib gcc-8-doc libstdc++6-8-dbg gcc-8-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg libquadmath0-dbg libstdc++-8-doc The following NEW packages will be installed: cpp-8 g++-8 gcc-8 libasan5 libgcc-8-dev libstdc++-8-dev libubsan1 0 upgraded, 7 newly installed, 0 to remove and 28 not upgraded. Need to get 33.1 MB of archives. After this operation, 117 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 cpp-8 amd64 8.3.0-6ubuntu1~18.04.1 [8,914 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libasan5 amd64 8.3.0-6ubuntu1~18.04.1 [366 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libubsan1 amd64 8.3.0-6ubuntu1~18.04.1 [122 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgcc-8-dev amd64 8.3.0-6ubuntu1~18.04.1 [2,307 kB] Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 gcc-8 amd64 8.3.0-6ubuntu1~18.04.1 [9,787 kB] Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libstdc++-8-dev amd64 8.3.0-6ubuntu1~18.04.1 [1,543 kB] Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 g++-8 amd64 8.3.0-6ubuntu1~18.04.1 [10.1 MB] Fetched 33.1 MB in 1s (34.2 MB/s) Selecting previously unselected package cpp-8. (Reading database ... 132699 files and directories currently installed.) Preparing to unpack .../0-cpp-8_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking cpp-8 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package libasan5:amd64. Preparing to unpack .../1-libasan5_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking libasan5:amd64 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package libubsan1:amd64. Preparing to unpack .../2-libubsan1_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking libubsan1:amd64 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package libgcc-8-dev:amd64. Preparing to unpack .../3-libgcc-8-dev_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking libgcc-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package gcc-8. Preparing to unpack .../4-gcc-8_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking gcc-8 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package libstdc++-8-dev:amd64. Preparing to unpack .../5-libstdc++-8-dev_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking libstdc++-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Selecting previously unselected package g++-8. Preparing to unpack .../6-g++-8_8.3.0-6ubuntu1~18.04.1_amd64.deb ... Unpacking g++-8 (8.3.0-6ubuntu1~18.04.1) ... Setting up cpp-8 (8.3.0-6ubuntu1~18.04.1) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Setting up libasan5:amd64 (8.3.0-6ubuntu1~18.04.1) ... Setting up libubsan1:amd64 (8.3.0-6ubuntu1~18.04.1) ... Setting up libgcc-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Setting up libstdc++-8-dev:amd64 (8.3.0-6ubuntu1~18.04.1) ... Setting up gcc-8 (8.3.0-6ubuntu1~18.04.1) ... Setting up g++-8 (8.3.0-6ubuntu1~18.04.1) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. echo \"`git describe --long --dirty --always`\" > version.new diff -q -N version.new version.inc >/dev/null || mv version.new version.inc echo Version: `cat version.inc` Version: "v6.11-11-gfaaa2f2-dirty" g++ -MT common.o -MMD -MP -MF .d/common.Td -Wall -O2 -std=c++17 -c -o common.o common.cpp common.cpp:14:10: fatal error: filesystem: No such file or directory #include <filesystem> ^~~~~~~~~~~~ compilation terminated. Makefile:30: recipe for target 'common.o' failed make: *** [common.o] Error 1[/CODE] |
Colab availability observations...
So, just to further document the "one hour kill" behavior I mentioned earlier...
For the last two or three days, I've been finding that sometimes a new Colab instance (issued after something like a five to eight hour period of non-usage) will do the greyed-out "Instance Shutdown" thing almost exactly after one hour. But... If I happen to notice this within about five minutes and click on the "Connect" button presented, the usual "Connecting... Initializing..." thing happens, but instead of showing the usual "CPU / RAM / HD" values, it instead shows "Busy", and the Section's output updates for the time it was greyed out, and then continues to update nominally. Often for the remaining 12 hours. As in, the instance wasn't actually killed (yet). I have observed this behavior on all my Colab environments -- two here in Barbados (same browser context, different Google account) and two SOCKS tunneled environments (appearing to be running in the "States -- different browsers, different Google accounts). One of those is a CentOS VM on Intel, the other a Rasberry Pi on ARM. So now my procedure in the morning is to spin up as many instances as I can get, and then say to my phone "OK, Google. Set an alarm for one hour from now...". |
[QUOTE=chalsall;529576]But... If I happen to notice this within about five minutes and click on the "Connect" button presented, the usual "Connecting... Initializing..." thing happens, but instead of showing the usual "CPU / RAM / HD" values, it instead shows "Busy", and the Section's output updates for the time it was greyed out, and then continues to update nominally. Often for the remaining 12 hours. As in, the instance wasn't actually killed (yet).[/QUOTE]
I have noticed this too, although I have been able to recover the sessions without having to catch it within 5 minutes. The key is to [U]not[/U] click on the grayed out banner, but instead note which notebook it is, then close the window. Then open a new Colab window, navigate to that notebook, and open it. 100% of the time (so far anyway), I get the "Busy" message, and the code continues to run (although from all outward appearances it isn't running, but I know it is because the save files on my Google Drive continues to get updated normally). |
Build gpuowl on colab script
1 Attachment(s)
[CODE]!update-alternatives --remove-all gcc
!update-alternatives --remove-all g++ !apt-get install gcc-8 g++-8 !update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 10 !update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 10 !update-alternatives --config gcc !update-alternatives --config g++ !g++ --version !make gpuowl[/CODE]Seems to do the trick, although ubuntu repeatedly protests that alternatives are unnecessary. [CODE]g++-8 is already the newest version (8.3.0-6ubuntu1~18.04.1). gcc-8 is already the newest version (8.3.0-6ubuntu1~18.04.1). 0 upgraded, 0 newly installed, 0 to remove and 28 not upgraded. update-alternatives: using /usr/bin/gcc-8 to provide /usr/bin/gcc (gcc) in auto mode update-alternatives: using /usr/bin/g++-8 to provide /usr/bin/g++ (g++) in auto mode There is only one alternative in link group gcc (providing /usr/bin/gcc): /usr/bin/gcc-8 Nothing to configure. There is only one alternative in link group g++ (providing /usr/bin/g++): /usr/bin/g++-8 Nothing to configure. g++ (Ubuntu 8.3.0-6ubuntu1~18.04.1) 8.3.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. echo \"`git describe --long --dirty --always`\" > version.new diff -q -N version.new version.inc >/dev/null || mv version.new version.inc echo Version: `cat version.inc` Version: "v6.11-11-gfaaa2f2-dirty" g++ -MT common.o -MMD -MP -MF .d/common.Td -Wall -O2 -std=c++17 -c -o common.o common.cpp g++ -MT main.o -MMD -MP -MF .d/main.Td -Wall -O2 -std=c++17 -c -o main.o main.cpp g++ -MT Gpu.o -MMD -MP -MF .d/Gpu.Td -Wall -O2 -std=c++17 -c -o Gpu.o Gpu.cpp g++ -MT clwrap.o -MMD -MP -MF .d/clwrap.Td -Wall -O2 -std=c++17 -c -o clwrap.o clwrap.cpp g++ -MT Task.o -MMD -MP -MF .d/Task.Td -Wall -O2 -std=c++17 -c -o Task.o Task.cpp g++ -MT checkpoint.o -MMD -MP -MF .d/checkpoint.Td -Wall -O2 -std=c++17 -c -o checkpoint.o checkpoint.cpp g++ -MT timeutil.o -MMD -MP -MF .d/timeutil.Td -Wall -O2 -std=c++17 -c -o timeutil.o timeutil.cpp g++ -MT Args.o -MMD -MP -MF .d/Args.Td -Wall -O2 -std=c++17 -c -o Args.o Args.cpp g++ -MT state.o -MMD -MP -MF .d/state.Td -Wall -O2 -std=c++17 -c -o state.o state.cpp g++ -MT Signal.o -MMD -MP -MF .d/Signal.Td -Wall -O2 -std=c++17 -c -o Signal.o Signal.cpp g++ -MT FFTConfig.o -MMD -MP -MF .d/FFTConfig.Td -Wall -O2 -std=c++17 -c -o FFTConfig.o FFTConfig.cpp g++ -MT AllocTrac.o -MMD -MP -MF .d/AllocTrac.Td -Wall -O2 -std=c++17 -c -o AllocTrac.o AllocTrac.cpp g++ -MT gpuowl-wrap.o -MMD -MP -MF .d/gpuowl-wrap.Td -Wall -O2 -std=c++17 -c -o gpuowl-wrap.o gpuowl-wrap.cpp g++ -o gpuowl Pm1Plan.o GmpUtil.o Worktodo.o common.o main.o Gpu.o clwrap.o Task.o checkpoint.o timeutil.o Args.o state.o Signal.o FFTConfig.o AllocTrac.o gpuowl-wrap.o -lstdc++fs -lOpenCL -lgmp -pthread -L/opt/rocm/opencl/lib/x86_64 -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -L/c/Windows/System32 -L. [/CODE]This executable is as yet untested. |
New GPUs on Colab
Today I surprisingly received 2 Tesla P100s on my 3 accounts that are currently running Colab. They are absolutely amazing cards for P-1 and PRP, ranging anywhere from 1160us/it to 1175us/it on 5120K FFT. So happy to receive something other than a boring Tesla K80 or a slow Tesla T4 (from a PRP tester perspective) that I've been receiving for the past 1-2 months.
[CODE]Tue Nov 5 02:51:54 2019 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 430.50 Driver Version: 418.67 CUDA Version: 10.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 | | N/A 44C P0 29W / 250W | 0MiB / 16280MiB | 0% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+[/CODE] |
Wow, congrats!
We still didn't see a T4 from the last time we were boasting here about getting one. We think it was a jinxy decision to boast about it, hehe. |
[QUOTE=LaurV;529680]Wow, congrats!
We still didn't see a T4 from the last time we were boasting here about getting one. We think it was a jinxy decision to boast about it, hehe.[/QUOTE] Neither have I seen a T4 in months, maybe Google is thinking about reallocating T4s to kaggle and leaving the P100s to Colab, or maybe they are just removing T4 instances in general since they are in extremely high demand? If it's indeed the case that kaggle is switching to T4s, then they will be completely useless to me since I won't be doing TF works on them. |
| All times are UTC. The time now is 23:01. |
Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.