New GPU computing system (experimental)

I got a new mainboard to test. Unfortunately the board has Gen2 PCIe slots (full-width), as such it does not work with ROCm.


Installed Ubuntu 18.04.2 and now trying to install amdgpu-pro.


Installation of amdgpu-pro fails, but this appears to be a common bug (by reading through the forums).


So I am digging documentation and forums to attempt to understand where is the bug and how to workaround.


If someone has the same problem, lets unite forces and fight the bug.

Originally Posted by SELROC

I got a new mainboard to test. Unfortunately the board has Gen2 PCIe slots (full-width), as such it does not work with ROCm.


Installed Ubuntu 18.04.2 and now trying to install amdgpu-pro.


Installation of amdgpu-pro fails, but this appears to be a common bug (by reading through the forums).


So I am digging documentation and forums to attempt to understand where is the bug and how to workaround.


If someone has the same problem, lets unite forces and fight the bug.

It turns out that Ubuntu live-server does not have the multiverse repository by default, it must be added, after that amdgpu-pro installs successfully.

Originally Posted by SELROC

It turns out that Ubuntu live-server does not have the multiverse repository by default, it must be added, after that amdgpu-pro installs successfully.

Validated both gpuowl and mfakto on the main-board with Ubuntu live-server 18.04.2 and amdgpu-pro 18.50.


1. GpuOwl was validated against prime 44497.
2. Mfakto was validated with self-tests and perftest.

Originally Posted by SELROC

Validated both gpuowl and mfakto on the main-board with Ubuntu live-server 18.04.2 and amdgpu-pro 18.50.


1. GpuOwl was validated against prime 44497.
2. Mfakto was validated with self-tests and perftest.

I'm working hard here but I won't tell you which mobo it is until I start real computation :-P

Originally Posted by SELROC

It turns out that Ubuntu live-server does not have the multiverse repository by default, it must be added, after that amdgpu-pro installs successfully.

I must add that amdgpu-pro 18.50 has gained much stability.

Originally Posted by SELROC

I'm working hard here but I won't tell you which mobo it is until I start real computation :-P

OK this main board is just over the top.


Name: BTC-HM65


https://www.aliexpress.com/item/B250...7-b6e73995c8fc

Originally Posted by SELROC

OK this main board is just over the top.


Name: BTC-HM65


https://www.aliexpress.com/item/B250...7-b6e73995c8fc

I have downgraded the kernel from 5.0.0 to 4.18 in order to install the new driver amdgpu-pro 19.10. It seems to carry a tiny performance increase on the RX580.

[M344587487]

Originally Posted by SELROC

OK this main board is just over the top.


Name: BTC-HM65


https://www.aliexpress.com/item/B250...7-b6e73995c8fc

That looks ideal apart from gen 2 slots, a mobo that can run 8 cards nicely spaced with CPU included. It's like a giant riser with the gubbins you're rising from integrated underneath. What's the postage cost like from ali express or did you source it from elsewhere? Can it run Radeon VII with ROCm?


edit: The last question is because of this from ROCm:

GFX8 GPUs require PCI Express 3.0 (PCIe 3.0) with support for PCIe atomics. This requires both CPU and motherboard support. GFX9 GPUs, by default, also require PCIe 3.0 with support for PCIe atomics, but they can operate in most cases without this capability.

Originally Posted by M344587487

That looks ideal apart from gen 2 slots, a mobo that can run 8 cards nicely spaced with CPU included. It's like a giant riser with the gubbins you're rising from integrated underneath. What's the postage cost like from ali express or did you source it from elsewhere? Can it run Radeon VII with ROCm?


edit: The last question is because of this from ROCm:

The specification of the main board says it supports gfx8, and it has 8 power connectors, that doesn't fit gfx9 gpus that have 2 power connectors (the main board would need 16).
I run it with amdgpu-pro, the cost is approx. 105 Euro + 4 Euro of shipping cost.

Originally Posted by M344587487

That looks ideal apart from gen 2 slots, a mobo that can run 8 cards nicely spaced with CPU included. It's like a giant riser with the gubbins you're rising from integrated underneath. What's the postage cost like from ali express or did you source it from elsewhere? Can it run Radeon VII with ROCm?


edit: The last question is because of this from ROCm:

Even with Gen2 slots the overall performance is better than the classical RIG with risers. GpuOwl timing for wave-front 89M exponents is 4.08~4.20 ms/sq.


The board does not support ROCm, but the new amdgpu-pro version performs very well.

[M344587487]

Originally Posted by SELROC

The specification of the main board says it supports gfx8, and it has 8 power connectors, that doesn't fit gfx9 gpus that have 2 power connectors (the main board would need 16).
I run it with amdgpu-pro, the cost is approx. 105 Euro + 4 Euro of shipping cost.

Why would the board need 16 connectors? The 8 pin power cables go directly from PSU to GPU, the power connectors on the board next to the slots are to supply 75W from the PCIe slot. I don't get why the 8 connectors lining the bottom of the board from your link exist.