Radeon VII @ newegg for 500 dollars US on 11-27

[M344587487]

You're probably on 3.0. Looks like to try reverting to 2.10 you'll need to add rocm-dev to Paul's apt autoremove line as that's the package you used, one or both of rocm-dkms and rock-dkms shouldn't be installed but it doesn't matter if you leave them in the remove command. Similarly if you want to try the 2.10 upstream drivers install rocm-dev instead of rocm-dkms.

clinfo should be in

/opt/rocm/opencl/bin/x86_64/

[ewmayer]

Originally Posted by M344587487

You're probably on 3.0. Looks like to try reverting to 2.10 you'll need to add rocm-dev to Paul's apt autoremove line as that's the package you used, one or both of rocm-dkms and rock-dkms shouldn't be installed but it doesn't matter if you leave them in the remove command. Similarly if you want to try the 2.10 upstream drivers install rocm-dev instead of rocm-dkms.

clinfo should be in

/opt/rocm/opencl/bin/x86_64/

OK, clinfo gives

ewmayer@ewmayer-haswell:~/gpuowl/run0$ /opt/rocm/opencl/bin/x86_64/clinfo
Number of platforms:				 1
  Platform Profile:				 FULL_PROFILE
  Platform Version:				 OpenCL 2.1 AMD-APP (3052.0)
  Platform Name:				 AMD Accelerated Parallel Processing
  Platform Vendor:				 Advanced Micro Devices, Inc.
  Platform Extensions:				 cl_khr_icd cl_amd_event_callback cl_amd_offline_devices 


  Platform Name:				 AMD Accelerated Parallel Processing
ERROR: clGetDeviceIDs(-1)

Querying the installed packages with 'apt list | grep roc', I see the following ROC-related ones:

rocm-bandwidth-test/Ubuntu 16.04 1.4.0.9-rocm-rel-3.0-6-g8c2ce31 amd64
rocm-clang-ocl/Ubuntu 16.04,now 0.5.0.47-rocm-rel-3.0-6-cfddddb amd64 [installed,automatic]
rocm-cmake/Ubuntu 16.04,now 0.3.0.134-rocm-rel-3.0-6-e6d1ef3 amd64 [installed,automatic]
rocm-debug-agent/Ubuntu 16.04,now 1.0.0 amd64 [installed,automatic]
rocm-dev/Ubuntu 16.04,now 3.0.6 amd64 [installed]
rocm-device-libs/Ubuntu 16.04,now 1.0.0.559-rocm-rel-3.0-6-628eea4 amd64 [installed,automatic]
rocm-dkms/Ubuntu 16.04 3.0.6 amd64
rocm-libs/Ubuntu 16.04 3.0.6 amd64
rocm-opencl-dev/Ubuntu 16.04,now 2.0.0-rocm-rel-3.0-6-9a4afec amd64 [installed,automatic]
rocm-opencl/Ubuntu 16.04,now 2.0.0-rocm-rel-3.0-6-9a4afec amd64 [installed,automatic]
rocm-profiler/Ubuntu 16.04 5.6.7262 amd64
rocm-smi-lib64/Ubuntu 16.04,now 2.2.0.8.rocm-rel-3.0-6-8ffe1bc amd64 [installed,automatic]
rocm-smi/Ubuntu 16.04,now 1.0.0-192-rocm-rel-3.0-6-g01752f2 amd64 [installed,automatic]
rocm-utils/Ubuntu 16.04,now 3.0.6 amd64 [installed,automatic]
rocm-validation-suite/Ubuntu 16.04 0.0.33 amd64
rocminfo/Ubuntu 16.04,now 1.0.0 amd64 [installed,automatic]
rocprim/Ubuntu 16.04 2.9.0.950-rocm-rel-3.0-6-b85751b amd64
rocprofiler-dev/Ubuntu 16.04,now 1.0.0 amd64 [installed,automatic]
rocrand/Ubuntu 16.04 2.10.0.656-rocm-rel-3.0-6-b9f838b amd64
rocs/eoan 4:19.04.3-0ubuntu1 amd64
rocs/eoan 4:19.04.3-0ubuntu1 i386
rocsolver/Ubuntu 16.04 2.7.0.57-rocm-rel-3.0-6-7983da3 amd64
rocsparse/Ubuntu 16.04 1.5.15.691-rocm-rel-3.0-6-aee785e amd64
rocthrust/Ubuntu 16.04 2.9.0.413-rocm-rel-3.0-6-957b1e9 amd64

So as you note, -dev is the one I want, -dkms is not installed. Did the autoremove, but for the next file-entry-edit step per Smelov, I don't see an 'apt' subdir in my /etc dir - is that likely a Debian-specific thing, or is the needed file perhaps somewhere else in Ubuntu?

[M344587487]

It should be there, as you followed my guide you added the rocm repo to the sources list with this:

echo 'deb [arch=amd64] http://repo.radeon.com/rocm/apt/debian/ xenial main' | sudo tee /etc/apt/sources.list.d/rocm.list

If it didn't exist you wouldn't have been able to install rocm.

[ewmayer]

Originally Posted by M344587487

It should be there, as you followed my guide you added the rocm repo to the sources list with this:

echo 'deb [arch=amd64] http://repo.radeon.com/rocm/apt/debian/ xenial main' | sudo tee /etc/apt/sources.list.d/rocm.list

If it didn't exist you wouldn't have been able to install rocm.

Ah ... I hit ctrl-o in my edit window on the system, it default-pointed me to my last location, which was a subdir of /etc ... as root, did the file-entry debian->2.10.0 edit, 'apt update' and 'apt install rocm-dev' were successful, and from one of the 2 run subdirs I created in ~/gpuowl, fired up one job, success at last!

ewmayer@ewmayer-haswell:~/gpuowl/run0$ sudo ../gpuowl -user ewmayer
[sudo] password for ewmayer: 
2020-02-04 13:58:31 gpuowl v6.11-142-gf54af2e
2020-02-04 13:58:31 Note: not found 'config.txt'
2020-02-04 13:58:31 config: -user ewmayer 
2020-02-04 13:58:31 device 0, unique id ''
2020-02-04 13:58:32 gfx906+sram-ecc-0 103984877 FFT 5632K: Width 256x4, Height 64x4, Middle 11; 18.03 bits/word
2020-02-04 13:58:34 gfx906+sram-ecc-0 OpenCL args "-DEXP=103984877u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=11u -DWEIGHT_STEP=0x1.f54acc23489eep+0 -DIWEIGHT_STEP=0x1.0577e0c0e09e4p-1 -DWEIGHT_BIGSTEP=0x1.ae89f995ad3adp+0 -DIWEIGHT_BIGSTEP=0x1.306fe0a31b715p-1 -DAMDGPU=1  -I. -cl-fast-relaxed-math -cl-std=CL2.0"
1 warning generated.
2020-02-04 13:58:44 gfx906+sram-ecc-0 warning: argument unused during compilation: '-I .'

2020-02-04 13:58:44 gfx906+sram-ecc-0 OpenCL compilation in 10.22 s
2020-02-04 13:58:45 gfx906+sram-ecc-0 103984877 P1 B1=1000000, B2=30000000; 1442134 bits; starting at 0
2020-02-04 13:58:53 gfx906+sram-ecc-0 103984877 P1    10000   0.69%;  758 us/it; ETA 0d 00:18; 7011c7174679e5dd
2020-02-04 13:59:00 gfx906+sram-ecc-0 103984877 P1    20000   1.39%;  753 us/it; ETA 0d 00:18; f066604ab63196d0
2020-02-04 13:59:08 gfx906+sram-ecc-0 103984877 P1    30000   2.08%;  760 us/it; ETA 0d 00:18; 6e54df44e09f831d
2020-02-04 13:59:15 gfx906+sram-ecc-0 103984877 P1    40000   2.77%;  755 us/it; ETA 0d 00:18; 306d220bd3f66b99
2020-02-04 13:59:23 gfx906+sram-ecc-0 103984877 P1    50000   3.47%;  753 us/it; ETA 0d 00:17; 18faa6b7b06be852
2020-02-04 13:59:30 gfx906+sram-ecc-0 103984877 P1    60000   4.16%;  754 us/it; ETA 0d 00:17; b499eb4c155b7ed4
2020-02-04 13:59:38 gfx906+sram-ecc-0 103984877 P1    70000   4.85%;  758 us/it; ETA 0d 00:17; b26087c1e503d5f6
2020-02-04 13:59:46 gfx906+sram-ecc-0 103984877 P1    80000   5.55%;  762 us/it; ETA 0d 00:17; 3a4debdafd61495c
2020-02-04 13:59:53 gfx906+sram-ecc-0 103984877 P1    90000   6.24%;  756 us/it; ETA 0d 00:17; 928441b2e23adf31

But, ctrl-z/bg didn't stop those screen outputs ... how do I divert those to a file?

I left the smi fan control setting at 10, the fan has automatically kicked into turbo-blast mode. After several minutes of running, per-iter times have stabilized at ~800 us, which suggests that I may want to manually up the fan speed (and/or downclock the card). rocm-smi shows

PU  Temp   AvgPwr  SCLK     MCLK     Fan     Perf  PwrCap  VRAM%  GPU%  
1    80.0c  248.0W  1684Mhz  1001Mhz  56.86%  auto  250.0W    2%   100%

...and my wall wattmeter jumped from 120W to 400W. So I think I need to downclock the system a bit, to gpuowl drawing maybe ~200W instead of 280W. How do I find the max stock voltage of my card, so I can tweak it downward per your instructions? [I'll do the setting-up-for-2-jobs later, think I'll quit while I'm ahead today. :]

[preda]

Originally Posted by ewmayer

Ah ... I hit ctrl-o in my edit window on the system, it default-pointed me to my last location, which was a subdir of /etc ... as root, did the file-entry debian->2.10.0 edit, 'apt update' and 'apt install rocm-dev' were successful, and from one of the 2 run subdirs I created in ~/gpuowl, fired up one job, success at last!
But, ctrl-z/bg didn't stop those screen outputs ... how do I divert those to a file?

I has left the smi fan control setting at 10, the fan has automatically kicked into turbo-blast mode. After several minutes of running, rocm-smi shows
...and my wall wattmeter jumped from 120W to 400W. So I think I need to downclock the system a bit, to gpuowl drawing maybe ~200W instead of 280W. How do I find the max stock voltage of my card, so I can tweak it downward per your instructions? [I'll do the setting-up-for-2-jobs later, think I'll quit while I'm ehead today. :]

rocm-smi --setsclk 3
or 4
now you only need one job per GPU for optimal throughput.

[kriesel]

Originally Posted by ewmayer

But, ctrl-z/bg didn't stop those screen outputs ... how do I divert those to a file?

I think you don't. Gpuowl prints to both gpuowl.log and to console. On Windows the console output is not redirectable in my experience. Just dedicate a (virtual) terminal to it and move on.

[paulunderwood]

Originally Posted by ewmayer

...success at last!...

Welcome to the Radeon VII club. You will never look back

[ewmayer]

Originally Posted by preda

rocm-smi --setsclk 3
or 4
now you only need one job per GPU for optimal throughput.

Thanks - nice and simple. In the meantime I upped the fan setting to 150, then tried --setsclk with setting 3,4,5 - looks like 5 is the default, is that right?

--setsclk 5: 757 us/iter, temp = 70C, watts = 400 [~120 of those are baseline, including an ongoing 4-thread Mlucas job on the CPU]
--setsclk 4: 792 us/iter, temp = 65C, watts = 350
--setsclk 3: 848 us/iter, temp = 63C, watts = 300

So without fiddling the clocking, simply upping fanspeed to 150 dropped the temp from 80C to 70C. Downclocking cuts the wattage nicely, but it's hard to see what the effect on runtime is because the job I started is in p-1 stage 2. I'll update with effect of the above setting on per-iteration times once the job gets into PRP-test mode. [Edit: added per-iter to above table.]

Based on the results, I'll use '--setsclk 4' for now. Preda, can I expect any total-throughput boost from running 2 jobs per Matt's instructions, at the same settings?

[paulunderwood]

Originally Posted by ewmayer

But, ctrl-z/bg didn't stop those screen outputs ... how do I divert those to a file?

There will be a way to use the command "screen" (from a crontab -- but you will need root not sudo). That way you can open up a terminal and screen the output. See https://www.mersenneforum.org/showpo...91&postcount=7

[ewmayer]

Originally Posted by kriesel

I think you don't. Gpuowl prints to both gpuowl.log and to console. On Windows the console output is not redirectable in my experience. Just dedicate a (virtual) terminal to it and move on.

Yeah, that's what I did while awaiting an answer from one of the old hands.

Originally Posted by paulunderwood

Welcome to the Radeon VII club. You will never look back

Seeing those actual per-iter times on what was until an hour ago an aged, clunky 6-y.o. Haswell system is something else, that's for sure. Thanks, Mihai, for such a great program! It was nice to be able to upgrade the aforementioned aging system this way, got a lot of added-throughput bang for my hardware-purchase $.

So it looks like p-1 stage 2 finished, no factor found ... I will update my previous post with the per-iter times at each of the 3 clock settings I tried.

[xx005fs]

Originally Posted by preda

If it's ROCm 3.0, it may have broken OpenCL, see https://github.com/RadeonOpenCompute/ROCm/issues/977

I am honestly really disappointed on how AMD is handling OpenCL right now, since they have basically neglected support of it on Windows machines, now ROCm 3.0 breaks OpenCL. I think what they need to do is work out something similar to CUDA or somehow convert CUDA codes automatically while maintaining good performance. I hope in the future with stronger hardware, OpenCL won't be neglected to the degree that they can't be utilized to run GPUOWL.