George's dream build

[Prime95]

Originally Posted by Fred

Timings on my DDR4-2133 are 15-15-15-35. The DDR4-3200 are 16-18-18-38.

vs 10-12-12-31 for the DDR3-2400 on the Haswell system I used for my estimates.

[Fred]

Originally Posted by Prime95

1) Can you enable the "DDR4 non-Z OC" feature on the H110 board and does that make a difference in throughput and power consumption?

George, I think I need to start sending you tuition checks. Your posts teach me something new every day. I wasn't aware that memory overclocking of any kind was an option on nonZ asrock boards. I mis-spoke on my board, it's the B150 not the H110, but the "non-Z OC" feature is the same for both. I enabled it with literally 2 clicks in my bios, and the results are impressive.

Without non-Z OC enabled:
Timings for 4096K FFT length (4 cpus, 1 worker): 6.37 ms. Throughput: 156.97 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.66, 13.04 ms. Throughput: 155.68 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 25.68, 25.71, 24.64, 24.65 ms. Throughput: 159.00 iter/sec.

With non-Z OC enabled:
Timings for 4096K FFT length (4 cpus, 1 worker): 6.13 ms. Throughput: 163.12 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.82, 12.08 ms. Throughput: 160.72 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 23.95, 23.97, 23.68, 23.99 ms. Throughput: 167.39 iter/sec.

Power consumption increased by 4.2% (from 71w to 74w), but I'll take it for a 6.5% increase in performance, and no additional upfront costs!

So, if I can get away with the intel stock cpu cooler on future builds, that would be $331.95 for a system that has has 4096K FFT throughput of 167.39 iter/sec, and uses 74W. Seems pretty good to me.

[Prime95]

Thanks for the info. Indeed an impressive boost for little cost.

I do appreciate all you've done. I just bought three i5-6500s from newegg as they emailed a coupon code for $15 off expiring Feb 14. Your info will greatly influence my motherboard/memory choice.

[bgbeuning]

Originally Posted by Prime95

1) Did you try the included Intel heatsink/fan? If so, what were the temps? What are the temps now?

My i5-6500 (DDR4-2400) is using the stock Intel cooler with core temps around 60 C.

[Fred]

Originally Posted by Prime95

Thanks for the info. Indeed an impressive boost for little cost.

I do appreciate all you've done. I just bought three i5-6500s from newegg as they emailed a coupon code for $15 off expiring Feb 14. Your info will greatly influence my motherboard/memory choice.

What did you settle on for a hard drive solution? I'm using these http://www.amazon.com/SanDisk-Ultra-...2CS/ref=sr_1_1

Adds $10 to each build, but it's a simple solution that is working well for me.

[Prime95]

Originally Posted by Fred

What did you settle on for a hard drive solution?

No decision yet. I was going to use a cheap 128GB SSD on one motherboard and have the others network boot off that machine. Your solution may well be superior in cost, space and simplicity.

I have both a spare disk drive and USB stick, so I'll experiment.

[Prime95]

I have two boards in, an H110 and a Z170. My benchmark data matches Fred's:

H110 board, I5-6500, DDR4-2133, mechanical disk drive, PSU 82% efficient. Kill-a-watt reads 85.2.

Timings for 4096K FFT length (4 cpus, 1 worker):  6.31 ms.  Throughput: 158.50 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.37, 12.37 ms.  Throughput: 161.64 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 24.65, 24.65, 24.66, 24.68 ms.  Throughput: 162.21 iter/sec.

H110 board, I5-6500, DDR4-2133, mechanical disk drive, PSU 82% efficient, DDR4 non-Z OC set to SPORT. Kill-a-watt reads 86.0.

Timings for 4096K FFT length (4 cpus, 1 worker):  6.26 ms.  Throughput: 159.69 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.31, 12.31 ms.  Throughput: 162.43 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 24.35, 24.36, 24.35, 24.36 ms.  Throughput: 164.22 iter/sec.

H110 board, I5-6500, DDR4-2133, mechanical disk drive, PSU 82% efficient, DDR4 non-Z OC set to SPORT+. Kill-a-watt reads 87.1.

Timings for 4096K FFT length (4 cpus, 1 worker):  6.08 ms.  Throughput: 164.49 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.03, 12.07 ms.  Throughput: 165.98 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 23.73, 23.74, 23.73, 23.79 ms.  Throughput: 168.44 iter/sec.

Z170 board, I5-6500, DDR4-2133, USB disk, PSU 82% efficient. Kill-a-watt reads (a net increase of) 70. This board actually uses a Picopsu running with the H110 mobo above that is consuming 87.1 watts.

Timings for 4096K FFT length (4 cpus, 1 worker):  6.43 ms.  Throughput: 155.47 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 12.60, 12.59 ms.  Throughput: 158.79 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 25.18, 25.15, 25.15, 25.16 ms.  Throughput: 159.00 iter/sec.

Z170 board, I5-6500, DDR4-2133 OC'd to 3200, USB disk, PSU 82% efficient. Kill-a-watt reads (a net increase of) 81. This board actually uses a Picopsu running with the H110 mobo above that is consuming 87.1 watts.

Timings for 4096K FFT length (4 cpus, 1 worker):  5.53 ms.  Throughput: 180.81 iter/sec.
Timings for 4096K FFT length (4 cpus, 2 workers): 10.90, 10.90 ms.  Throughput: 183.50 iter/sec.
Timings for 4096K FFT length (4 cpus, 4 workers): 21.45, 21.45, 21.44, 21.32 ms.  Throughput: 186.76 iter/sec.

[Prime95]

And finally my best-bang-for-the-buck analysis using the data above. Assuming electricity costs are $1/watt/year.


5 ASRock Z170M-ITX/ac motherboards @130 = 650
5 2x4GB DDR4-3200 @60= 300
5 I5-6500 CPUs (3.2GHz, 65W) @200 = 1000
1 32GB USB stick = 10
4 PicoPSU picoPSU-90 @40 = 160
1 Case, power supply, network switch -- approximate value $100
Each of the 5 units consumes 81W or about 405W total.
Total cost of 3 year ownership = 2120 parts + 3 * 405 = 3335
Total cost of 4 year ownership = 2120 parts + 4 * 405 = 3740

Now lets define a metric to optimize -- throughput per dollar (TPD).
3 year TPD = 5 CPUs * 186.8 * 3 years / 3335 = 0.8402
4 year TPD = 5 CPUs * 186.8 * 4 years / 3740 = 0.9989

Let's compare that to a second system built with cheaper H110 motherboards that do not allow overclocking. We will save $60 for each motherboard and $20 for each RAM pair, for a total of $400. Watts is reduced by 11 per CPU.
Throughput for each CPU is 162.2. Now let's look at our TPD metric:
Total cost of 3 year ownership = 1720 parts + 3 * (5*70) = 2770
Total cost of 4 year ownership = 1720 parts + 4 * (5*70) = 3120
3 year TPD = 5 CPUs * 162.2 * 3 years / 2770 = 0.8773
4 year TPD = 5 CPUs * 162.2 * 4 years / 3120 = 1.0397

Now we turn on ASRock's non-Z OC for memory. Watts increase by 2 per CPU. Throughput increases to 168.4.
3 year TPD = 5 CPUs * 168.4 * 3 years / 2800 = 0.9021
4 year TPD = 5 CPUs * 168.4 * 4 years / 3160 = 1.0658

I've just bought a platinum PSU to increase PSU efficiency to 92%. This will change the calculations above slightly, but not the conclusion of using ASRock H110 mobos.

[S485122]

In view of the electricity costs, it would be interesting to look at the "T" variant of the processors their TDP is as low as 35 W compared to 65 W for the regular processors. Another way to achieve that would be to under-clock the regular processors. It should be interesting to measure the power usage of both variants.

Based on the prices of the CPU's in Belgium and the declared TDP the "T" would achieve much better throughput par Watt figures. you might even envisage the I5--6600T. Another advantage of under-clocking is that the memory bottleneck is less acute.

It is a bit unusual in the Prime95 world to envisage under-clocking but it is a logical consequence of the goal you choose (i.e. maximising throughput par Watt.)

Jacob

[Mark Rose]

Originally Posted by S485122

In view of the electricity costs, it would be interesting to look at the "T" variant of the processors their TDP is as low as 35 W compared to 65 W for the regular processors. Another way to achieve that would be to under-clock the regular processors. It should be interesting to measure the power usage of both variants.

Based on the prices of the CPU's in Belgium and the declared TDP the "T" would achieve much better throughput par Watt figures. you might even envisage the I5--6600T. Another advantage of under-clocking is that the memory bottleneck is less acute.

It is a bit unusual in the Prime95 world to envisage under-clocking but it is a logical consequence of the goal you choose (i.e. maximising throughput par Watt.)

Jacob

Interesting. It took a bit of digging to find the the 4 core turbo speed: 3.1 GHz. That makes it as fast as the i5-6400.

All that would be needed for a throughput-per-watt comparison is a benchmark of a 6500 or 6600 underclocked to 3.1 GHz.

[Fred]

Originally Posted by Prime95

And finally my best-bang-for-the-buck analysis using the data above.

George, I'm curious what your plans or thoughts are for housing the mobos. Currently I'm just going vertical on my desk with brass standoffs (about to add my third board, photo attached). I was thinking about getting it 4 or 5 boards high, then building (as simple as possible) 3 sided acrylic case of some sort, just so no one mistakenly zaps a board with static electricity or whatever. Do you plan any case?