CPUs and GPUs (Oh My)

[Primeinator]

Hey, all. I am starting to look into different options for a custom built system. I am aware that for GPUs the Radeon VII is one of the best for PRP testing. With that in mind how do these compare:

Radeon RX 6900 XT 16 GB (and what is different with the MSI X Trio version?)
NVIDIA GeForce RTX 3090
Radeon RX 5500 XT 8 GB

For CPUs:
Will I be able to turn to out twice as many PRP tests with a 16 core CPU compared to an 8 core?
How much different will a 64 GB L3 cache make compared to a 32 GB?
..... the Ryzen Threadripper 3990X 64 core? (this seems super sexy but is it THAT much better? It should be for that price tag).


The highest end options for both CPUs and GPUs are MUCH more expensive. Is there a good "inflection point" for where price meets efficiency?

Is there a motherboard I should get that will in theory allow me to add a second GPU at some point (such as a second RX 6900?)

If this is the eventual goal what type of power supply/fans/cooling (liquid?) should I opt for?


Thanks!


Edit: I don't want a jet engine in my office. How loud would something like this be and what options are there to dampen it?

Edit 2: Is there truly any difference between a 64 GB RAM system or 32 GB RAM system for PRP testing?

[Uncwilly]

2 pages that might be of interest to you:
Estimated GPU trial-factoring performance
Estimated GPU Lucas-Lehmer performance, that may be reflective of PRP testing.

Memory banking appears to be more of a factor in PRP/LL testing than total amount.
Filling all of the available channels with the same speed memory will make for faster speeds than bigger modules in fewer channels.

[Viliam Furik]

Originally Posted by Primeinator

Hey, all. I am starting to look into different options for a custom built system. I am aware that for GPUs the Radeon VII is one of the best for PRP testing. With that in mind how do these compare:

Radeon RX 6900 XT 16 GB (and what is different with the MSI X Trio version?)
NVIDIA GeForce RTX 3090
Radeon RX 5500 XT 8 GB

Well, Uncwilly already posted a link to a page with performance benchmarks, so that's about it.

Originally Posted by Primeinator

For CPUs:
Will I be able to turn to out twice as many PRP tests with a 16 core CPU compared to an 8 core?

The answer depends on a lot of variables, so the simple version is "Kind of, yeah." In some cases, you will be able to produce more than two times, sometimes a bit less. For example Ryzens 3950X (16c) and 3800X (8c). The former has 64 MB of L3 cache, the latter only 32 MB. 32 MB is not enough to contain the entire current first-time test FFT data (that's about 50 MB). If it is contained, it can run faster, because the L3 cache is faster than RAM. Therefore the performance ratio in the test is higher than 2.

Originally Posted by Primeinator

How much different will a 64 GB L3 cache make compared to a 32 GB?

This was pretty much answered above, but I would like to point out, that cache is not in GB but in MB. If it were in the GB range, that would be awesome.

The difference also means, that if a CPU can do say 1 DC-LL test per day with 32 MB cache (current DC-LL takes about 25 MB of cache), then if it suddenly had 64 MBs, it could do two of them in about the same time, with each taking up only half of the cores. Continued in the next question...

Originally Posted by Primeinator

..... the Ryzen Threadripper 3990X 64 core? (this seems super sexy but is it THAT much better? It should be for that price tag).

That's why Threadrippers are great beasts. TR 3990X has 256 MB of L3 cache, which means it can do about 9-10 DC-LL tests. In other words, you get the most performance out of it, when you run that amount of workers at once. But due to it having 8 dies with 8 cores each, it's possible the most performance would come from 8 workers, each running on 8 cores.

It can also do 5 First-time tests, without losing on the total throughput, because that much tests fit into the L3 cache.

And it can contain a whole 100 million digit test, up to exponents of about 600M.

Originally Posted by Primeinator

The highest end options for both CPUs and GPUs are MUCH more expensive. Is there a good "inflection point" for where price meets efficiency?

In my opinion, Threadripper, if given enough attention in adjusting and benchmarking, can be the best of the best, especially the 3990X beast.

Originally Posted by Primeinator

Is there a motherboard I should get that will in theory allow me to add a second GPU at some point (such as a second RX 6900?)

Any (or at least most of them) full-size motherboard, which is for mid-range and high-end users has at least two PCIe 16x slots, thus can carry two GPUs. You can also use mining extensions to connect more card into one slot. This will not hurt the performance much, because the cards don't need much traffic in the PCIe link while computing.

Originally Posted by Primeinator

If this is the eventual goal what type of power supply/fans/cooling (liquid?) should I opt for?

It depends on how much money you are willing to spend. If you buy a 3990X (it's probably better to wait for Zen 3 Threadrippers, i.e. the Ryzen 5000), and some top-tier GPUs, you should definitely invest in high wattage Platinum (or Titanium, if possible) PSU. You will need many watts, and headroom is good for efficiency, as is the efficiency certification.

Fans don't have to be super fancy, you just need them to spin properly, and not too slowly. I think EK water cooling fans are a good choice.

And you should definitely invest in a water cooling loop for all that heat. You should water cool also the GPUs. It will give them lower temperatures and thus a bit more performance.

Originally Posted by Primeinator

Thanks!

You're welcome!

Originally Posted by Primeinator

Edit: I don't want a jet engine in my office. How loud would something like this be and what options are there to dampen it?

That depends mostly on the case and fan settings. If you choose a water cooling loop and don't do overclocking, it should be about the same as a normal computer.

Originally Posted by Primeinator

Edit 2: Is there truly any difference between a 64 GB RAM system or 32 GB RAM system for PRP testing?

For PRP testing, no. For P-1 factoring, yes. PRP needs more memory channels (unless is already contained in the L3 cache), not necessarily more memory.



If you choose to buy 3990X, (or even if you don't) I can help you with choosing the rest of the components, and finding the best settings for Prime95.

[Primeinator]

Originally Posted by Uncwilly

2 pages that might be of interest to you:
Estimated GPU trial-factoring performance
Estimated GPU Lucas-Lehmer performance, that may be reflective of PRP testing.

Memory banking appears to be more of a factor in PRP/LL testing than total amount.
Filling all of the available channels with the same speed memory will make for faster speeds than bigger modules in fewer channels.

Wow. That A100 SXM4 looks absolutely amazing. Seems like you cannot buy it standalone though. Nor could I afford it at the moment. The Radeon RX 6900 XT vs 6800 XT (compared to regular 6800) seem good options (since the Radeon VII has been discontinued) and it doesn't seem like there is really that much difference between the two given the significant price difference. And the RAM thing makes sense.

The answer depends on a lot of variables, so the simple version is "Kind of, yeah." In some cases, you will be able to produce more than two times, sometimes a bit less. For example Ryzens 3950X (16c) and 3800X (8c). The former has 64 MB of L3 cache, the latter only 32 MB. 32 MB is not enough to contain the entire current first-time test FFT data (that's about 50 MB). If it is contained, it can run faster, because the L3 cache is faster than RAM. Therefore the performance ratio in the test is higher than 2.

So larger L3 is always better but number of cores may not be. An 8 core compared to a 16 core, both with a 64 MB cache, must share it- correct? Is this where the RAM comes in and why it is important to have more channels of RAM versus just more RAM?

I think EK water cooling fans are a good choice.

And you should definitely invest in a water cooling loop for all that heat. You should water cool also the GPUs. It will give them lower temperatures and thus a bit more performance.

Is water cooled essentially no different than these "higher performance" coolants?
Once a liquid cooling system is installed is it easy to modify? For instance, if I only order one GPU now but want to buy a second or third later? Or should the initial cooling loops go over the area that will eventually be occupied by the future GPUs?

If you buy a 3990X (it's probably better to wait for Zen 3 Threadrippers, i.e. the Ryzen 5000

What will make these better?

If you choose to buy 3990X, (or even if you don't) I can help you with choosing the rest of the components, and finding the best settings for Prime95.

Thank you. I will likely need some expertise!

[Primeinator]

The Radeon RX 6900 XT vs 6800 XT (compared to regular 6800) seem good options (since the Radeon VII has been discontinued) and it doesn't seem like there is really that much difference between the two given the significant price difference

Actually, which is more important for turning out a PRP test faster? GFLOPS (and what is the difference between SP or DP) or GHz-days/day?

[Viliam Furik]

Originally Posted by Primeinator

So larger L3 is always better but number of cores may not be. An 8 core compared to a 16 core, both with a 64 MB cache, must share it- correct? Is this where the RAM comes in and why it is important to have more channels of RAM versus just more RAM?

Yes, it has to be shared. Generally, if there is the same number of cache, and double the cores, it's about 2 times faster. If there is the same number of cores, and double the cache, it gets more complicated, because it depends on the size of the test(s) you want to run. But you can get about double the throughput by running two workers if both tests fit into cache, as I mentioned.

The best way to find out is to directly run a benchmark.

Originally Posted by Primeinator

Is water cooled essentially no different than these "higher performance" coolants?
Once a liquid cooling system is installed is it easy to modify? For instance, if I only order one GPU now but want to buy a second or third later? Or should the initial cooling loops go over the area that will eventually be occupied by the future GPUs?

I'm not sure what you mean by higher performance coolants. Water cooling means you put special cooling blocks on the chips, either the CPU or the GPU (or both), and there is water going through the blocks. The active movement of the heat makes it a better way of cooling. It's like the AIO (all-in-one), which is a simplified version of the water cooling loop.

I attached a picture of such a loop. There is a radiator on the right side, a pump (with a built-in reservoir - the tank with the red liquid) next to it, and tubes connect it to the CPU block and GPU block.

I recommend soft tubing, which is bendable much more easily than hard tubing, which is basically a solid hard plastic and needs to be heated to be bendable. It also comes in handy when you want to extend the loop. For extension of the loop, you just disconnect tubes that need to be disconnected, and put in new components and close the loop. But before disconnecting anything, you need to drain the loop, otherwise, you would have the water spilt out everywhere... Pretty much a logical step.

To answer the question, it is easy in the sense, that it can be done without complications, but not easy in the sense that it takes some time to drain, add the components, and refill. For all of that, I would recommend finding some good enough expert that would do it for you. Unless you are willing to do it on your own, which places the burden of the risk of damaging components (or any kind of screw-up) on you.

So water cooling is good, but it's better to get an expert to do it, unless you are willing to take the risk yourself, and have read and seen enough tutorials on how to do it, such that you pretty much know what you are doing.

Originally Posted by Primeinator

What will make these better?

Well, new hardware has usually higher performance than the older one. You know, generational improvements. Ryzen 5000 CPUs have a new Zen 3 architecture, which comes with about a 17% IPC (instructions per clock) increase, which translates into higher performance. So Threadripper 5000 may have a better performance per dollar.

Originally Posted by Primeinator

Actually, which is more important for turning out a PRP test faster? GFLOPS (and what is the difference between SP or DP) or GHz-days/day?

Technically it is the same. GFLOPS can be directly converted into GHz-D/D, which is just a unit that's more convenient for GIMPS users. 500 GHz-D/D is 1 TFLOPS.

SP -> single precision is the performance measure for TF computation, which uses INT32 numbers. INT32 is usually lower than FP32, which is the SP by definition (SP = FP32). DP -> double-precision is the performance measure for PRP (and LL) tests and is much lower than SP.

[Primeinator]

Technically it is the same. GFLOPS can be directly converted into GHz-D/D, which is just a unit that's more convenient for GIMPS users. 500 GHz-D/D is 1 TFLOPS.

SP -> single precision is the performance measure for TF computation, which uses INT32 numbers. INT32 is usually lower than FP32, which is the SP by definition (SP = FP32). DP -> double-precision is the performance measure for PRP (and LL) tests and is much lower than SP.

Ah I was curious looking at the GPU benchmarks above. For instance the 6900 XT has higher GHz-D/D than the Radeon VII but lower GFLOPS (for double precision). The Titan V has substantially higher GFLOPS (DP) but is much lower on the GHz-D/D. Then the Radeon VII Pro looks to be the king of GFLOPS DP (over 3x the 6900 XT) but is still lower than Ghz-D/D for that GPU. Reviewing the list there are multiple other such instances where GHz-D/D may be significantly less than that of another device despite the other device having higher GFLOPS. What is the reason for these differences and which metric should I pay attention to most for PRP testing (GLFOPS-DP even if that device is significantly less GHz-D/D)?

What about GPUs that have their own built in liquid cooling system? Are these generally inferior?

[Viliam Furik]

Originally Posted by Primeinator

Ah I was curious looking at the GPU benchmarks above. For instance the 6900 XT has higher GHz-D/D than the Radeon VII but lower GFLOPS (for double precision). The Titan V has substantially higher GFLOPS (DP) but is much lower on the GHz-D/D. Then the Radeon VII Pro looks to be the king of GFLOPS DP (over 3x the 6900 XT) but is still lower than Ghz-D/D for that GPU. Reviewing the list there are multiple other such instances where GHz-D/D may be significantly less than that of another device despite the other device having higher GFLOPS. What is the reason for these differences and which metric should I pay attention to most for PRP testing (GLFOPS-DP even if that device is significantly less GHz-D/D)?

Due to most of the GPUs being bottlenecked by the speed of the memory (like the CPUs are bottlenecked by the memory, and thus run faster when the data fits in cache memory), there is usually a big difference between the potential TFLOPS or GHz-D/D, and the real performance.

GFLOPS is given by the producer and means the highest potential speed. GHz-D/D represents the real measured speed in GIMPS computations, which need to have a lot of memory bandwidth, thus overall, some performance is unused.

6900XT has a built-in L3 cache, similar to the Ryzen CPUs, thus can have really high speeds, even surpass the bottlenecked Radeon VII. Radeon VII has more potential performance than 6900XT, but 6900XT has 1,5 times faster memory, the cache (1,5 TB/s).

Originally Posted by Primeinator

What about GPUs that have their own built in liquid cooling system? Are these generally inferior?

They are good, but they are not meant for multi-GPU water cooling setups, but for single GPU gaming setups. Generally, it's better to buy a simple (and a bit cheaper) air-cooled GPU, remove the cooler, slap the GPU block on it, and water-cool it.

[ewmayer]

Originally Posted by Uncwilly

2 pages that might be of interest to you:
Estimated GPU trial-factoring performance
Estimated GPU Lucas-Lehmer performance, that may be reflective of PRP testing.

I can only speak for the Radeon VII numbers, for which that page's ~300GHz-D/D are just over half the ~500GHz-D/D I get from each of my R7's, running underclocked (sclk=3) and pulling ~200W/GPU.

Re. liquid cooling, I prefer the lower-tech approach of open cases and lots of airflow. My 3-R7 desktop open-frame system uses a simple $15 desktop fan at the left end of the system to blow air over it, which is fugly but highly effective in terms of cooling, I don't need to run the cards' fans near their limits, which keeps the noise level very reasonable - the desktop fan makes just a lower-pitch steady humming noise, nothing at all above "white noise" noticeability:

[Prime95]

Originally Posted by ewmayer

I can only speak for the Radeon VII numbers, for which that page's ~300GHz-D/D are just over half the ~500GHz-D/D I get from each of my R7's, running underclocked (sclk=3) and pulling ~200W/GPU.

Consult this list for a more trustworthy ranking of GPUs: https://drive.google.com/file/d/1Tim...l57X_RWO0/view

[M344587487]

tl;dr

• The 6 core Ryzen 3600 used to be the sweet spot. The 5600X may be the new sweet spot but it all depends on price
• If PRP is your only metric CPU's are irrelevant, GPU's are the sane option (or would be if prices were sane)
• The one forumite that's publically tried Big Navi sold it in frustration due to technical issues, it's only an option if don't mind being a canary. And if you have deep pockets
• Nvidia 3000 series is bad at PRP
• Most consumer motherboards other than mini-ITX have multiple PCIe slots, determine which CPU you want then browse all motherboards with that socket. 1x slots are usable if you get risers but it's a blag and another point of failure. Electrically it doesn't matter if they're 1x 8x or 16x but physcially for an easy life they should be 16x
• As long as you buy a motherboard with expansion in mind you're actually more limited by the PSU, you need a lot of 8 pins to run multiple GPU's
• As long as you tune for efficiency and don't build a farm the noise is manageable. If necessary you can get sound dampening inside the case and/or on flat surfaces nearby if you're going for the farm life. If this is an upstairs home office it might make sense to stand the case on something that dampens vibrations, it sounds like a silly consideration but some houses can transmit the slightest vibration through the floor which gets old fast for something on 24/7