![]() |
1 Attachment(s)
Make a "cudalucas" folder on your Google Drive with the 4 files from the zip as well as a "worktodo.txt".
Make an "mprime" folder with mprime and in local.txt: WorkerThreads=1 CoresPerTest=1 and whatever prime.txt you want, it is fairly slow so I recommend ECM or PRP CF or PRP CF DC. Then I use this script to start it: [CODE]import os.path from google.colab import drive if not os.path.exists('/content/drive/My Drive'): drive.mount('/content/drive') %cd '/content/drive/My Drive' !chmod 700 mprime %cd '/content/drive/My Drive/mprime' !chmod 700 mprime !./mprime -d >> outputmprime.txt 2>&1 & %cd '/content/drive/My Drive/cudalucas/' !cp 'CUDALucas' /usr/local/bin/ !chmod 755 '/usr/local/bin/CUDALucas' !cd '.' && LD_LIBRARY_PATH="lib:${LD_LIBRARY_PATH}" /usr/local/bin/CUDALucas >> outputcudalucas.txt[/CODE] If you do not want output from the programs to files then change ">> outputmprime.txt" and ">> outputcudalucas.txt" to "> /dev/null" I have not gotten a T4 yet, so not been able to create the benchmark files for a T4. |
[QUOTE=bayanne;528480]OK, is there a set of instructions for getting LL running on Colab or Kaggle?
[/QUOTE]I'm gathering and tweaking scripts and trying the various apps, summarizing at [URL]https://www.mersenneforum.org/showthread.php?t=24839[/URL]. Luckily, since I hadn't got to cudalucas yet, ATH has covered it above. I would change one thing from that regular run script, to put both tasks in the background and run top occasionally. Another is that CoresPerTest=2 was reported to provide better throughput. Note also, that recommended practice is to do fft benchmarking and threads benchmarking to tune for the specific gpu model. See [URL]https://www.mersenneforum.org/showthread.php?t=23387[/URL] post 5. |
[QUOTE=ATH;528481]I have not gotten a T4 yet, so not been able to create the benchmark files for a T4.[/QUOTE]
If T4s are anything like RTX, don't bother; they're going to suck at DP. |
[QUOTE=axn;528493]If T4s are anything like RTX, don't bother; they're going to suck at DP.[/QUOTE]If I recall correctly, T4 is no longer being offered. One account I'm using reliably gets a VM with a K80 via Colaboratory and goes 12 hours, while a second has become unpredictable whether I can even get a cpu-only VM, often taking hours to reestablish, and also terminating early. They're running the same apps.
|
Has anyone gotten the GPU branch of GMP-ECM working in a Notebook yet? I've been trying to learn the Colab process and have tried to compile the above, but I run into cuda.h not being found. I have a script that runs all the way through the gmp/ECM process successfully without the GPU.
It's been a long time since I did any CUDA programming, so I'm very rusty. "nvcc --version" works fine, so CUDA is installed, but I'd like to do some basic programming again and will need cuda.h. Am I missing a "dev" library, or maybe the toolkit? |
@EdH: I haven’t tried to get a CUDA version of ECM working, as I am usually fine with the repo version of the program, which is retrieved by running
[CODE]!apt install gmp-ecm[/CODE] What you could try is to install the locate package (which for some reason isn’t available by default) and then try [CODE]!locate cuda.h[/CODE] and then edit the make file to link the place where that file is located. |
[QUOTE=kriesel;528483]I would change one thing from that regular run script, to put both tasks in the background and run top occasionally.[/QUOTE]
Top doesn't work in the Python shell -- that's an interactive command. "!vmstat -w 5" would probably be a better fit for purpose in this context. |
[QUOTE=Dylan14;528501]What you could try is to install the locate package (which for some reason isn’t available by default) and then try[/QUOTE]
I don't know if the install will do this, but be sure to first run "!updatedb" -- otherwise the locate database might not be populated. BTW, I often point to this as an example of "namespace pollution". A bit presumptuous of the locate package to take "updatedb" as its own. What DB is being updated? But since it was here first, it got to claim it. A bit like M$ and "Word"... |
@chalsall: Just ran apt to install "locate" and, no, it doesn't automatically build the database. "updatedb" has to called after the install and then one can use locate.
@EdH: issuing locate cuda.h yields the following results: [CODE]/tensorflow-2.0.0/python2.7/tensorflow_core/include/tensorflow/core/platform/cuda.h /tensorflow-2.0.0/python2.7/tensorflow_core/include/tensorflow/core/platform/stream_executor_no_cuda.h /tensorflow-2.0.0/python3.6/tensorflow_core/include/tensorflow/core/platform/cuda.h /tensorflow-2.0.0/python3.6/tensorflow_core/include/tensorflow/core/platform/stream_executor_no_cuda.h /usr/include/hwloc/cuda.h /usr/include/linux/cuda.h /usr/include/opencv2/core/cuda.hpp /usr/include/opencv2/core/private.cuda.hpp /usr/include/opencv2/photo/cuda.hpp /usr/include/x86_64-linux-gnu/libavutil/hwcontext_cuda.h /usr/lib/R/site-library/BH/include/boost/predef/language/cuda.h /usr/local/cuda-10.0/doc/man/man3/cuda.h.3 /usr/local/cuda-10.0/libnsight/plugins/com.nvidia.cuda.help_10.0.0.201808260220.jar /usr/local/cuda-10.0/samples/5_Simulations/nbody/bodysystemcuda.h /usr/local/cuda-10.0/samples/5_Simulations/nbody_opengles/bodysystemcuda.h /usr/local/cuda-10.0/samples/5_Simulations/nbody_screen/bodysystemcuda.h /usr/local/cuda-10.0/samples/6_Advanced/matrixMulDynlinkJIT/cuda_drvapi_dynlink_cuda.h /usr/local/cuda-10.0/samples/common/inc/helper_cuda.h /usr/local/cuda-10.0/targets/x86_64-linux/include/cuda.h /usr/local/lib/python2.7/dist-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/predef/language/cuda.h /usr/local/lib/python2.7/dist-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include/nvector/nvector_cuda.h /usr/local/lib/python2.7/dist-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include/nvector/nvector_mpicuda.h /usr/local/lib/python2.7/dist-packages/tensorflow_core/include/tensorflow/core/platform/cuda.h /usr/local/lib/python2.7/dist-packages/tensorflow_core/include/tensorflow/core/platform/stream_executor_no_cuda.h /usr/local/lib/python2.7/dist-packages/torch/include/torch/csrc/api/include/torch/cuda.h /usr/local/lib/python3.6/dist-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/predef/language/cuda.h /usr/local/lib/python3.6/dist-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include/nvector/nvector_cuda.h /usr/local/lib/python3.6/dist-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include/nvector/nvector_mpicuda.h /usr/local/lib/python3.6/dist-packages/tensorflow_core/include/tensorflow/core/platform/cuda.h /usr/local/lib/python3.6/dist-packages/tensorflow_core/include/tensorflow/core/platform/stream_executor_no_cuda.h /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/cuda.h /usr/src/linux-headers-4.15.0-58/include/linux/cuda.h /usr/src/linux-headers-4.15.0-58/include/uapi/linux/cuda.h [/CODE] so maybe try adding /usr/include/linux/ to your makefile (where you define where the cuda library is located). |
[QUOTE=Dylan14;528508]@EdH: issuing locate cuda.h yields the following results:[/QUOTE]
One of the wonderful things about Linux, much like the maths, is there is often more than one way of acheiving something... :smile: [CODE]root@Colab_RPi:~# find / -name cuda.h /usr/local/lib/python2.7/dist-packages/tensorflow_core/include/tensorflow/core/platform/cuda.h /usr/local/lib/python2.7/dist-packages/torch/include/torch/csrc/api/include/torch/cuda.h /usr/local/lib/python2.7/dist-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/predef/language/cuda.h /usr/local/lib/python3.6/dist-packages/tensorflow_core/include/tensorflow/core/platform/cuda.h /usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include/torch/cuda.h /usr/local/lib/python3.6/dist-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0/boost/predef/language/cuda.h /usr/local/cuda-10.0/targets/x86_64-linux/include/cuda.h /usr/include/hwloc/cuda.h /usr/include/linux/cuda.h /usr/src/linux-headers-4.15.0-58/include/uapi/linux/cuda.h /usr/src/linux-headers-4.15.0-58/include/linux/cuda.h /usr/lib/R/site-library/BH/include/boost/predef/language/cuda.h /tensorflow-2.0.0/python2.7/tensorflow_core/include/tensorflow/core/platform/cuda.h /tensorflow-2.0.0/python3.6/tensorflow_core/include/tensorflow/core/platform/cuda.h[/CODE] |
[QUOTE=chalsall;528502]Top doesn't work in the Python shell -- that's an interactive command.
"!vmstat -w 5" would probably be a better fit for purpose in this context.[/QUOTE] !top -d n sorta does. Here n=120. I just want to see the cpu and gpu apps are running. Will try vmstat. But from examples I've found online, it's not showing individual user process status. -w 5 seems much too frequent for this basically batch use over long periods. [CODE][B] Htop - 16:44:06 up 2:03, 0 users, load average: 1.00, 1.01, 1.00 Tasks: 15 total, 2 running, 13 sleeping, 0 stopped, 0 zombie %Cpu(s): 1.0 us, 1.1 sy, 50.1 ni, 47.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 13335184 total, 9829276 free, 906316 used, 2599592 buff/cache KiB Swap: 0 total, 0 free, 0 used. 12136692 avail Mem 342 root 30 10 353720 224716 6884 S 99.9 1.7 119:47.84 mprime 353 root 20 0 36.335g 94088 82888 S 0.7 0.7 0:51.24 mfaktc.e+ 121 root 20 0 639292 150864 62188 R 0.6 1.1 0:44.30 python3 290 root 20 0 1374064 65640 22160 S 0.1 0.5 0:05.03 drive 19 root 20 0 402904 98528 25664 S 0.0 0.7 0:04.85 jupyter-+ 9 root 20 0 682640 53228 24880 S 0.0 0.4 0:02.68 node 337 root 20 0 4568 824 764 S 0.0 0.0 0:00.26 tail 1 root 20 0 39192 6456 4948 S 0.0 0.0 0:00.04 run.sh 112 root 20 0 35884 4856 3752 S 0.0 0.0 0:00.30 tail 160 root 20 0 18376 1560 1256 S 0.0 0.0 0:00.00 bash 161 root 20 0 1117648 14408 12808 S 0.0 0.1 0:00.02 drive 162 root 20 0 11596 2148 1872 S 0.0 0.0 0:00.00 grep 336 root 20 0 18376 2984 2740 S 0.0 0.0 0:00.00 bash 338 root 20 0 11464 1036 944 S 0.0 0.0 0:00.00 grep 360 root 20 0 63180 6788 5060 R 0.0 0.1 0:00.11 top Htop - 16:46:06 up 2:05, 0 users, load average: 1.02, 1.01, 1.00 %Cpu(s): 1.1 us, 1.2 sy, 50.0 ni, 47.6 id, 0.1 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 13335184 total, 9823216 free, 910508 used, 2601460 buff/cache KiB Swap: 0 total, 0 free, 0 used. 12132116 avail Mem [/B]...[/CODE] |
| All times are UTC. The time now is 22:55. |
Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.