Help needed - needing a Nvidia Tesla K80 in a Dell 7920 tower workstation

[drkirkby]

I'm trying to get an Nvidia Tesla K80 GPU I bought used from eBay, working in a Dell 7920 tower workstation. (There is also a rackmount version of this workstation).

The Nvidia Tesla K80 has no video connectors on it - it is purely designed for computing, and not driving a monitor(s).

If I put the K80 in the workstation, the power supply comes on for 20 seconds or so, then goes off. Then it comes on for another 20 seconds or so and then powers off. This process goes on in an infinite loop. The thing most people will think of insufficient power, but that should not be the case.

* The machine is designed to take up to 3 x 300 W GPUs.
* Power consumption with dual 26-core CPUs is about 430 W, but the PSU is 1400 W. So the power supply could provide about 1 kW more than its currently supplying.

If I look on the Dell website for this machine

https://www.dell.com/en-uk/work/shop...xctopt7920emea

I can see it is available with several graphics cards, from
* Single Radeon Pro WX 2100, 2GB (standard)
* Dual NVLink NVIDIA Quadro GV100, 32GB, 4 DP (loads of money)

but I don't see anything like the K80, which is a pure GPU - not graphics card.

I asked on the Nvidia forum
https://forums.developer.nvidia.com/...station/165871
and was provided with this link
https://forums.developer.nvidia.com/...ovo-p510/64917
with someone having issues with a K40 GPU, but his issues were less serious, as at least the computer would power up. That link has this comment from an Nvidia moderator:

"If you buy a Tesla card believing you can install it in any system you want, you are asking for trouble. It is simply not possible, in the general case, and there is no design intent to make it possible....There is no suggestion anywhere that Tesla cards can be placed in any system you want with an expectation of proper behavior."

I've enabled the "above 4 GB" option in the BIOS that's mentioned there, but the workstation will not even power up properly - there's nothing ever on the screen. I guess I am just out of luck, and either

a) The card is no good, which is a possibility.
b) It will not work in the Dell 7920.

I wondered if anyone had any ideas how I might get this to work

Dave

[Xyzzy]

Do you have the PCIe cables plugged into it?

[frmky]

Are you cooling it? Tesla cards often don't have a fan installed and require a high-speed fan to keep it cool.

[drkirkby]

Originally Posted by Xyzzy

Do you have the PCIe cables plugged into it?

As far as I'm aware, there is only one cable that needs connecting. There's one cable for each of 3 GPUs - see page 72 of the manual for the 7920 tower workstation.

https://dl.dell.com/topicspdf/precis...nual_en-us.pdf

The PSU does not have any cables coming out of it. The PSU just pulls out of the chassis just by undoing a clip on the rear of the machine. It is possible to change the PSU in under a minute.

There is a rack-mount version of this machine. They are quite similar, but not identical.

* Rack unit has dual redundant PSUs
* Rack unit has better remote management
* Tower unit has more space for drives (10).

The manual for the rack unit is better, so whilst it's not for an identical model, it is sometimes helpful

https://dl.dell.com/topicspdf/precis...nual_en-us.pdf
around page 70.

As far as I can see, the 3 cables near the GPUs are designed to plug into the GPUs.

Dave

[drkirkby]

Originally Posted by frmky

Are you cooling it? Tesla cards often don't have a fan installed and require a high-speed fan to keep it cool.

I did wonder about the cooling, but the 7920 is designed to take up to 3 x 300 W, or 2 x 375 W GPUs. I assume (perhaps incorrectly), that one or more of the 10 fans in the machine would speed up to provide sufficient cooling air. However, none of the fans speed up in the 20 seconds or so the power supply remains on. I don't know if the card has an air-sensor, or heats up so quickly it shuts down the power supply in 20 seconds before the fans get going.

If one runs the diagnostics on the machine from the BIOS, it gets fairly loud, but never that loud in the 20 s or so before the power supply cuts out.

Maybe it would worth my while trying additional cooling. I will have to check out how the supported cards are cooled.

Dave

[paulunderwood]

Originally Posted by drkirkby

https://www.dell.com/en-uk/work/shop...xctopt7920emea

I can see it is available with several graphics cards, from
* Single Radeon Pro WX 2100, 2GB (standard)
* Dual NVLink NVIDIA Quadro GV100, 32GB, 4 DP (loads of money)

but I don't see anything like the K80, which is a pure GPU - not graphics card.

The WX 2100 is 250w max and the GV100 is 35w. However, the k80 is 300w and this is possibly too much for your DELL system and risks damage to it.

[paulunderwood]

This was one solution: https://www.dell.com/community/Rack-...d/td-p/7629428

[Xyzzy]

There is no way that 300W can be drawn from the PCIe slot. You need at least an 8 pin PCIe connector. What connectors are on the card? The PDF you linked shows a single 6 pin cord.

Note the PCIe requirement here: https://www.techpowerup.com/gpu-specs/tesla-k80.c2616

Pictures of your card and PSU would help a bunch!

[paulunderwood]

https://www.youtube.com/watch?v=wB9QtsDeAMo

It looks like you have a power pin-out problem. This guy says you need a CPU plug not PCI-e. Maybe you can get a converter (at your own risk). If you have a 3rd CPU cable,,,

[Xyzzy]

A picture of the card's power connector would allow us to see how it is keyed.

[tServo]

Originally Posted by frmky

Are you cooling it? Tesla cards often don't have a fan installed and require a high-speed fan to keep it cool.

I believe you may have 2 problems: power and cooling. It's hard to say which is the culprit.

(1) cooling: I agree with frmky that these cards in a 7920 will need some sort of additional cooling. The 7920 box does NOT have sufficient airflow to cool these red hot beasts.
They were designed to be put in a "pizza box" server that has about 20 high power fans that scream so loud it sounds like a jet taking off. Years ago, I had a friend that put one of these in a 7910 ( I think ) and he cobbled a VERY high speed fan to do the job. The fan fit by the exhaust slots and pulled the air out. I can't remember how he attached it to the machine, he may have used duct tape ( The handyman's secret weapon ).
You can search ebay for "tesla k80 fan" for some guys who use printers to make fans that attach to the input side of the card.

(2) the cards need a very special cable that takes 2 standard pcie 8 pin power cables and output a special 8 pin cable. I would try using 2 different pcie power cables ( not just 1 cable split into 2 ). search ebay for "K80 power cable".

Good luck!