Radeon VII @ newegg for 500 dollars US on 11-27

[ewmayer]

Originally Posted by kriesel

If I recall correctly, per Mihai, PCIe 3 vs 2 and x16 vs x1 impact is minor. The cooling advantage may be more significant than the occasional data transfer rate.

On my system, I went with the higher cost i7-4790K because I intended to work the cpu hard with a base load of prime95. And initially with a single 8Gb DIMM, and got ~15 ms/iter prime95 for 4.5M fft; now with two DIMMs, it's ~8.3ms/iter, quite a speedup for the little I spent on the last DIMM. And it will let me do some P-1 on occasion. Your gpus with 16GB on each Radeon VII will handle that too.

Ernst, do you plan to remove the bracket from one of your gpus to mechanically allow use of the x16 slot?

Re. cooling, recall your note about having extra PCI slots for debug purposes - those could also be handy by way of spacing the 3 GPUs apart for better airflow.

Your first note re. this mobo didn't mention bracket incompatibility with Radeon VII - could you be more specific, and how did you deal with that when you had your Radeon VII mounted? I had to do some Dremeling and end-of-bracket right-angle-bending to get my current Radeon VII mounted in my Haswell-system ATX case (see pic below), but would greatly prefer to avoid the need for such (literal) hackery in the multi-GPU build.

[kriesel]

Originally Posted by ewmayer

Re. cooling, recall your note about having extra PCI slots for debug purposes - those could also be handy by way of spacing the 3 GPUs apart for better airflow.

Your first note re. this mobo didn't mention bracket incompatibility with Radeon VII - could you be more specific, and how did you deal with that when you had your Radeon VII mounted? I had to do some Dremeling and end-of-bracket right-angle-bending to get my current Radeon VII mounted in my Haswell-system ATX case (see pic below), but would greatly prefer to avoid the need for such (literal) hackery in the multi-GPU build.

I have mentioned it, perhaps another thread.

The cheap sheet metal mining rig frame I bought has a bottom pan on which the motherboard mounts with metal standoffs. That pan extended past where the chassis-back bracket of the gpu would pass lower. So there was interference. I chose to mount the Radeon VII and all other gpus on the upper rails and connect by extenders, rather than modify a gpu or the pan. Fortunately so, since the Radeon VII was returned for refund after it failed within 3 weeks.
The 6 PCIe slots are a single slot width apart on that mobo, as on most motherboards. Most gpus are two slots wide (and some even thicker). So mounting three gpus directly onto the motherboard covers 3 other slots. And puts them tightly back-to-front with their neighbor(s). This is a typical problem with motherboards, including workstation-bespoke designs. Two gpus in an HP Z600 workstation tower can be internal; going to 3, requires two of them are outside the case or are single-slot-width. Since the last PCIe slot is generally crowding the side/bottom of the motherboard and case, the only way to get a gpu per slot is usually for all to be single-wide, or all extender-connected.

That's why I was fond of Quadro 2000s and 4000s early on; they're one slot wide.
Single-slot gpu models are rare and getting rarer.
The constraints increase from there, as some motherboard designs put tall components or drive cages in places that will interfere with full population of the PCIe space, especially with a full-2-slot-volume brick design like the Radeon VII.
Some cards are enough taller than what the case designers anticipated that I've had to remove various items; over-the-top clamp-in-place plastic piece in a Lenovo;
metal-and-foam clamp attached to the side cover on the HP Z600s. These interferences included NVIDIA GTX and AMD RX gpus.

Extenders on a mining frame let one increase the lateral spacing and air flow and evade the interference issues by using the top rails, where the row of fans get mounted. I've had ten different gpus mounted on my little mining frame, and modded none of them; they were all extender-connected.

I've never needed to mod a Radeon VII or other gpu bracket as you had done, and I've had two different Radeon VIIs installed in a total of 3 systems; the mining frame and two conventional workstation towers (Lenovo). Large gpus can be finicky about getting the bracket and the card edge to go in but I've always found a way in a commercial case to maneuver it in. Sometimes it takes several tries.

[ewmayer]

@Ken: So since used mining-rig mobos are cheap in no small part due to the cryptocyurrency-bubble implosion of the last few years, should I look for a big honking mobo with, say, 12 pci slots, allowing me to space the 3 dual-width cards apart better?

[M344587487]

If you're going for a larger expandable solution you can get mining mobos with many 1xPCIe slots that you use risers with. The nicer mining mobos have a USB plug on the board in place of a PCIe slot which means you don't need to use the often loose fitting PCIe adapter that comes with a riser ( https://cryptomining-blog.com/tag/gpu-riser/ ). The mobos are also small as they don't use physical x16 slots. If you don't mind a bit of DIY it's easy to make an open air rig out of 2x2's with mobo+psu underneath, or there are cheap off the shelf open-air cases.



Unless you're going for >4 GPUs per system you can do the same with a non-mining mobo. Saying that normal hardware is starting to get expensive, a mining mobo might be the way to go.

[kriesel]

Originally Posted by ewmayer

@Ken: So since used mining-rig mobos are cheap in no small part due to the cryptocyurrency-bubble implosion of the last few years, should I look for a big honking mobo with, say, 12 pci slots, allowing me to space the 3 dual-width cards apart better?

No. The extenders work fine. I have 5 gpus on a frame well spaced and have had 6 on it, using extenders. And one of those is a 2.75-slot gpu; most are 2-wide.
If you used a 13-slot board, which would cost more, that would give you 3 slots of space between double-slot-width cards mounted direct to the board. And maybe bring bracket complications into the mix.
A big board requires a bigger frame.
If you haven't already, you may find it useful to draw the alternatives. Lay them out spatially.

[ewmayer]

Originally Posted by kriesel

No. The extenders work fine. I have 5 gpus on a frame well spaced and have had 6 on it, using extenders. And one of those is a 2.75-slot gpu; most are 2-wide.
If you used a 13-slot board, which would cost more, that would give you 3 slots of space between double-slot-width cards mounted direct to the board. And maybe bring bracket complications into the mix.
A big board requires a bigger frame.
If you haven't already, you may find it useful to draw the alternatives. Lay them out spatially.

Just to be clear, you're saying that a cheap combo like this would work fine for hosting three (or more) 2-slot-wide GPUs, just with extenders on one or more of the PCI slots in order to space the GPUs apart for better airflow?

Edit: Just bought this bundle - that has just 3 pci3 slots, but they are well-spaced, and seller is including the stock cooler, so I basically get a fancy magnetohydrodynamicmagneticsupercalafragalistic cooler which I could in future repurpose on a high-end CPU. My offer of $180 was accepted, that kept the total price under $200, once CA sales tax was included at payment time. Once it arrives I can do some 3D "air builds" to get a sense as to what kind of mining-rig frame to buy.

[Prime95]

Though I have no recent AMD experience, that looks like a good buy. If spacing is an issue an x1 to x16 extender will fit in you mobo's x16 slot.

[ewmayer]

Just ordered a barebones test-bench-style Alu. rack for the mobo, plus sundry other small items needed to get the non-GPU parts of the system up and running:

PNY CS900 240GB 2.5” SATA III Internal Solid State Drive

Electop 2 Pack 2 Pin SW PC Power Cable on/Off Push Button ATX Computer Switch Wire 45cm

Panda 300Mbps Wireless N USB Adapter -- Besides software-pkg management, I intend to use this for wireless/monitorless remote system administration, after setup is done

Electric Magic M-ATX MATX Motherboard Test Bench Open Air Frame Computer Case

SATA Cable, Inateck SATA Data Cable and SATA Power Splitter Cable, ST1003

Poniie PN1500 Portable Micro Electricity Usage Monitor -- Same model I got for my ATX case system to monitor wattage after Radeon VII install.

Total incl. tax: $150. Together with the $195 I spent on the above cpu/mobo/ram bundle, that leaves $1655 remaining under my $2000 build max-cost target, most of which will get eaten by the 3 Radeon VII cards.

[sdbardwick]

Ernst, I'd double check if the Electric Magic frame will work with your motherboard. The frame says it is for MATX (micro ATX), while your board is larger regular ATX.

Might consider a mining frame with PCI-E extenders.

[ewmayer]

Originally Posted by sdbardwick

Ernst, I'd double check if the Electric Magic frame will work with your motherboard. The frame says it is for MATX (micro ATX), while your board is larger regular ATX.

Might consider a mining frame with PCI-E extenders.

Thanks for the catch - just cancelled that item and replaced with this one, which adds $3.

Also forgot to include the power supply, cost incl. tax = $139.

So sub another $142 from my money-left-under-cap, leaving $1513 - that's gonna be a bit of a trick to get 3 Radeon VIIs. But we live in hope. :)

[M344587487]

Originally Posted by ewmayer

Just to be clear, you're saying that a cheap combo like this would work fine for hosting three (or more) 2-slot-wide GPUs, just with extenders on one or more of the PCI slots in order to space the GPUs apart for better airflow?

Edit: Just bought this bundle - that has just 3 pci3 slots, but they are well-spaced, and seller is including the stock cooler, so I basically get a fancy magnetohydrodynamicmagneticsupercalafragalistic cooler which I could in future repurpose on a high-end CPU. My offer of $180 was accepted, that kept the total price under $200, once CA sales tax was included at payment time. Once it arrives I can do some 3D "air builds" to get a sense as to what kind of mining-rig frame to buy.

Phew, for a second I though you went with an FX CPU and was cursing that I don't log in more frequently. Any Ryzen is good for the right price, FX is to be avoided in general but especially if efficiency matters.