mersenneforum.org  

Go Back   mersenneforum.org > New To GIMPS? Start Here! > Information & Answers

Reply
 
Thread Tools
Old 2021-03-30, 17:42   #1
azhad
 
Oct 2004

24 Posts
Default How to avoid the dirty flag in gpuowl builds?

I am trying to compile v6.11-380-g79ea0cc in Google Colab. Build is successful but the version is displayed as "Version: "v6.11-380-g79ea0cc-dirty".

How to get rid of that dirty flag suffix?

Code:
#draft Notebook to set up a gpuowl Google drive folder for a future Colab session
import os.path
from google.colab import drive
import sys
if not os.path.exists('/content/drive/My Drive'):
  drive.mount('/content/drive')
%cd '/content/drive/My Drive//'
!chmod +w '/content/drive/My Drive'

if not os.path.exists('/content/drive/My Drive/gpuowl/gpuowlbuilded'):
  !mkdir '/content/drive/My Drive/gpuowl/gpuowlbuilded'

%cd '/content/drive/My Drive/gpuowl/gpuowlbuilded/'

!git clone https://github.com/preda/gpuowl.git

%cd gpuowl/
!git checkout -f v6.11-380-g79ea0cc

%cd '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl'
!chmod 777 '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py'
!apt install libgmp-dev
!update-alternatives --remove-all gcc 
!update-alternatives --remove-all g++
!apt-get install gcc-8 g++-8
!update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 10
!update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 10
!update-alternatives --config gcc
!update-alternatives --config g++
!g++ --version
!make gpuowl

!echo create config.txt, worktodo.txt before continuing

Last fiddled with by azhad on 2021-03-30 at 17:50 Reason: Missed !git clone <repo>
azhad is offline   Reply With Quote
Old 2021-03-31, 11:55   #2
preda
 
preda's Avatar
 
"Mihai Preda"
Apr 2015

32×151 Posts
Default

I guess it's because of the "chmod 777 expand.py". Do you need that? (expand.py already has rights 775)
preda is offline   Reply With Quote
Old 2021-04-01, 15:28   #3
azhad
 
Oct 2004

100002 Posts
Default

This is what happens without that chmod 777:

Code:
./tools/expand.py < gpuowl.cl > gpuowl-expanded.cl
/bin/sh: 1: ./tools/expand.py: Permission denied
Makefile:42: recipe for target 'gpuowl-expanded.cl' failed
make: *** [gpuowl-expanded.cl] Error 126
create config.txt, worktodo.txt before continuing
.

So reverted to run that chmod 777 command - then it works.

Below is a !git status result:

Code:
/content/drive/My Drive
/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl
HEAD detached at 79ea0cc
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   conv.py
	modified:   pm1/pm1.py
	modified:   test-pm1/makeWork.py
	modified:   tools/counts.py
	modified:   tools/delta.sh
	modified:   tools/device.py
	modified:   tools/monitor.py
	modified:   tools/primenet.py
	modified:   tools/upload.py

no changes added to commit (use "git add" and/or "git commit -a")
azhad is offline   Reply With Quote
Old 2021-04-02, 12:29   #4
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

3·5·53 Posts
Default

If editing the Makefile is fine you can run this before make:
Code:
sed -i s/--dirty//g Makefile
M344587487 is offline   Reply With Quote
Old 2021-04-02, 13:21   #5
PhilF
 
PhilF's Avatar
 
Feb 2005
Colorado

617 Posts
Default

Quote:
Originally Posted by M344587487 View Post
If editing the Makefile is fine you can run this before make:
Code:
sed -i s/--dirty//g Makefile
What's the point of that? Isn't that just covering up the symptom rather than fixing the cause?
PhilF is online now   Reply With Quote
Old 2021-04-02, 15:24   #6
chris2be8
 
chris2be8's Avatar
 
Sep 2009

111111110102 Posts
Default

Try !ls -l '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py' before the chmod 777. That should tell you what it's permissions really are.

And run !ls -l '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py' again at the end of the build to see if the last update time has changed. I'm wondering if part of the build process tries to update the script. If not 775 (or even 555) should be OK.

Chris
chris2be8 is offline   Reply With Quote
Old 2021-04-02, 15:43   #7
M344587487
 
M344587487's Avatar
 
"Composite as Heck"
Oct 2017

3·5·53 Posts
Default

Editing is simple, and if changing a permission really is a fix for the underlying problem it'll work. But now that you've forced me to look again, how can giving "other" write permissions for expand.py be a fix when we don't write to expand.py? git and other commands appear to be able to modify files no problem so it doesn't look like they're running under "other", why should expand.py be? As it's the first redirect encountered it could be something along the lines of the shell not having write permissions somehow so the redirect fails. Still doesn't explain the chmod fix. If it is a permission issue then there's nothing that can be done on the repo end as apparently git only tracks the execute bit anyway.

Alternatively the whole permission nonsense could be a red herring, it's the first time a python script is executed and it could just be that we're doing it indirectly without preceding it with python3. I have zero experience of colab or why every command is preceded with !, but not specifying python3 in the Makefile could be tripping up whatever weirdo environment google has going on with colab. Anyone else tried gpuowl with colab?
M344587487 is offline   Reply With Quote
Old 2021-04-03, 01:53   #8
azhad
 
Oct 2004

24 Posts
Default

Quote:
Originally Posted by chris2be8 View Post
Try !ls -l '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py' before the chmod 777. That should tell you what it's permissions really are.

And run !ls -l '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py' again at the end of the build to see if the last update time has changed. I'm wondering if part of the build process tries to update the script. If not 775 (or even 555) should be OK.

Chris
It appears that the execute bit is not set for the file before the first chmod command. Hence attempts to run the expand.py fail?

I have tried chmod 700 and the compile works too.

Here are the full results (of 777 compile) for the benefit of others:
Code:
Mounted at /content/drive
/content/drive/My Drive
/content/drive/My Drive/gpuowl/gpuowlbuilded
Cloning into 'gpuowl'...
remote: Enumerating objects: 306, done.
remote: Counting objects: 100% (306/306), done.
remote: Compressing objects: 100% (167/167), done.
remote: Total 6571 (delta 194), reused 222 (delta 136), pack-reused 6265
Receiving objects: 100% (6571/6571), 13.10 MiB | 11.67 MiB/s, done.
Resolving deltas: 100% (4732/4732), done.
/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl
Note: checking out 'v6.11-380-g79ea0cc'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 79ea0cc Fix P-1 integral
/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl
-rw------- 1 root root 1161 Apr  3 01:29 '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py'
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libgmpxx4ldbl
Suggested packages:
  gmp-doc libgmp10-doc libmpfr-dev
The following NEW packages will be installed:
  libgmp-dev libgmpxx4ldbl
0 upgraded, 2 newly installed, 0 to remove and 30 not upgraded.
Need to get 325 kB of archives.
After this operation, 1,667 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 libgmpxx4ldbl amd64 2:6.1.2+dfsg-2 [8,964 B]
Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libgmp-dev amd64 2:6.1.2+dfsg-2 [316 kB]
Fetched 325 kB in 1s (323 kB/s)
Selecting previously unselected package libgmpxx4ldbl:amd64.
(Reading database ... 160980 files and directories currently installed.)
Preparing to unpack .../libgmpxx4ldbl_2%3a6.1.2+dfsg-2_amd64.deb ...
Unpacking libgmpxx4ldbl:amd64 (2:6.1.2+dfsg-2) ...
Selecting previously unselected package libgmp-dev:amd64.
Preparing to unpack .../libgmp-dev_2%3a6.1.2+dfsg-2_amd64.deb ...
Unpacking libgmp-dev:amd64 (2:6.1.2+dfsg-2) ...
Setting up libgmpxx4ldbl:amd64 (2:6.1.2+dfsg-2) ...
Setting up libgmp-dev:amd64 (2:6.1.2+dfsg-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
/sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link

update-alternatives: error: no alternatives for gcc
update-alternatives: error: no alternatives for g++
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  cpp-8 libasan5 libgcc-8-dev libstdc++-8-dev libubsan1
Suggested packages:
  gcc-8-locales g++-8-multilib gcc-8-doc libstdc++6-8-dbg gcc-8-multilib
  libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg
  liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg libquadmath0-dbg
  libstdc++-8-doc
The following NEW packages will be installed:
  cpp-8 g++-8 gcc-8 libasan5 libgcc-8-dev libstdc++-8-dev libubsan1
0 upgraded, 7 newly installed, 0 to remove and 30 not upgraded.
Need to get 27.7 MB of archives.
After this operation, 113 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 cpp-8 amd64 8.4.0-1ubuntu1~18.04 [7,225 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libasan5 amd64 8.4.0-1ubuntu1~18.04 [366 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libubsan1 amd64 8.4.0-1ubuntu1~18.04 [122 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgcc-8-dev amd64 8.4.0-1ubuntu1~18.04 [2,305 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 gcc-8 amd64 8.4.0-1ubuntu1~18.04 [8,044 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libstdc++-8-dev amd64 8.4.0-1ubuntu1~18.04 [1,534 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 g++-8 amd64 8.4.0-1ubuntu1~18.04 [8,122 kB]
Fetched 27.7 MB in 3s (9,273 kB/s)
Selecting previously unselected package cpp-8.
(Reading database ... 160998 files and directories currently installed.)
Preparing to unpack .../0-cpp-8_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking cpp-8 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libasan5:amd64.
Preparing to unpack .../1-libasan5_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libasan5:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../2-libubsan1_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libubsan1:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libgcc-8-dev:amd64.
Preparing to unpack .../3-libgcc-8-dev_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libgcc-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package gcc-8.
Preparing to unpack .../4-gcc-8_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking gcc-8 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libstdc++-8-dev:amd64.
Preparing to unpack .../5-libstdc++-8-dev_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libstdc++-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package g++-8.
Preparing to unpack .../6-g++-8_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking g++-8 (8.4.0-1ubuntu1~18.04) ...
Setting up cpp-8 (8.4.0-1ubuntu1~18.04) ...
Setting up libasan5:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up libubsan1:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up libgcc-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up libstdc++-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up gcc-8 (8.4.0-1ubuntu1~18.04) ...
Setting up g++-8 (8.4.0-1ubuntu1~18.04) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
/sbin/ldconfig.real: /usr/local/lib/python3.7/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link

update-alternatives: using /usr/bin/gcc-8 to provide /usr/bin/gcc (gcc) in auto mode
update-alternatives: using /usr/bin/g++-8 to provide /usr/bin/g++ (g++) in auto mode
There is only one alternative in link group gcc (providing /usr/bin/gcc): /usr/bin/gcc-8
Nothing to configure.
There is only one alternative in link group g++ (providing /usr/bin/g++): /usr/bin/g++-8
Nothing to configure.
g++ (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

./tools/expand.py < gpuowl.cl > gpuowl-expanded.cl
cat head.txt gpuowl-expanded.cl tail.txt > gpuowl-wrap.cpp
echo \"`git describe --long --dirty --always`\" > version.new
diff -q -N version.new version.inc >/dev/null || mv version.new version.inc
echo Version: `cat version.inc`
Version: "v6.11-380-g79ea0cc-dirty"
g++ -MT Pm1Plan.o -MMD -MP -MF .d/Pm1Plan.Td -Wall -O2 -std=c++17   -c -o Pm1Plan.o Pm1Plan.cpp
g++ -MT GmpUtil.o -MMD -MP -MF .d/GmpUtil.Td -Wall -O2 -std=c++17   -c -o GmpUtil.o GmpUtil.cpp
g++ -MT Worktodo.o -MMD -MP -MF .d/Worktodo.Td -Wall -O2 -std=c++17   -c -o Worktodo.o Worktodo.cpp
g++ -MT common.o -MMD -MP -MF .d/common.Td -Wall -O2 -std=c++17   -c -o common.o common.cpp
g++ -MT main.o -MMD -MP -MF .d/main.Td -Wall -O2 -std=c++17   -c -o main.o main.cpp
g++ -MT Gpu.o -MMD -MP -MF .d/Gpu.Td -Wall -O2 -std=c++17   -c -o Gpu.o Gpu.cpp
g++ -MT clwrap.o -MMD -MP -MF .d/clwrap.Td -Wall -O2 -std=c++17   -c -o clwrap.o clwrap.cpp
g++ -MT Task.o -MMD -MP -MF .d/Task.Td -Wall -O2 -std=c++17   -c -o Task.o Task.cpp
g++ -MT checkpoint.o -MMD -MP -MF .d/checkpoint.Td -Wall -O2 -std=c++17   -c -o checkpoint.o checkpoint.cpp
g++ -MT timeutil.o -MMD -MP -MF .d/timeutil.Td -Wall -O2 -std=c++17   -c -o timeutil.o timeutil.cpp
g++ -MT Args.o -MMD -MP -MF .d/Args.Td -Wall -O2 -std=c++17   -c -o Args.o Args.cpp
g++ -MT state.o -MMD -MP -MF .d/state.Td -Wall -O2 -std=c++17   -c -o state.o state.cpp
g++ -MT Signal.o -MMD -MP -MF .d/Signal.Td -Wall -O2 -std=c++17   -c -o Signal.o Signal.cpp
g++ -MT FFTConfig.o -MMD -MP -MF .d/FFTConfig.Td -Wall -O2 -std=c++17   -c -o FFTConfig.o FFTConfig.cpp
g++ -MT AllocTrac.o -MMD -MP -MF .d/AllocTrac.Td -Wall -O2 -std=c++17   -c -o AllocTrac.o AllocTrac.cpp
g++ -MT gpuowl-wrap.o -MMD -MP -MF .d/gpuowl-wrap.Td -Wall -O2 -std=c++17   -c -o gpuowl-wrap.o gpuowl-wrap.cpp
g++ -MT sha3.o -MMD -MP -MF .d/sha3.Td -Wall -O2 -std=c++17   -c -o sha3.o sha3.cpp
g++ -MT md5.o -MMD -MP -MF .d/md5.Td -Wall -O2 -std=c++17   -c -o md5.o md5.cpp
g++ -o gpuowl Pm1Plan.o GmpUtil.o Worktodo.o common.o main.o Gpu.o clwrap.o Task.o checkpoint.o timeutil.o Args.o state.o Signal.o FFTConfig.o AllocTrac.o gpuowl-wrap.o sha3.o md5.o -lstdc++fs -lOpenCL -lgmp -pthread -L/opt/rocm-3.3.0/opencl/lib/x86_64 -L/opt/rocm-3.1.0/opencl/lib/x86_64 -L/opt/rocm/opencl/lib/x86_64 -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -L.
create config.txt, worktodo.txt before continuing
-rwx------ 1 root root 1161 Apr  3 01:29 '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py'
Changes made to code:
Code:
!chmod 700 '/content/drive/My Drive/gpuowl/gpuowlbuilded/gpuowl/tools/expand.py'
azhad is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Gpuowl Windows builds kriesel GpuOwl 27 2021-03-03 01:11
ARM builds and SIMD-assembler prospects ewmayer Mlucas 183 2019-02-25 08:17
Running 32-bit builds on a Win7 system ewmayer Programming 34 2010-10-18 22:36
Quick & Dirty storm5510 Programming 37 2009-09-08 06:52
Dirty Primes Cyclamen Persicum Math 23 2004-05-13 13:59

All times are UTC. The time now is 02:15.

Tue May 11 02:15:25 UTC 2021 up 32 days, 20:56, 1 user, load averages: 2.82, 2.79, 2.89

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.