George's dream build

[Prime95]

Originally Posted by Fred

George, I'm curious what your plans or thoughts are for housing the mobos.

You are ahead of me.

I hope to use an old computer case I have lying around. It will neatly house the PSU, and I can somehow attach the network switch near the drive bays. It already has 4 fans to keep things cool. This thing needs to be "portable" as it will change locations at least twice a year, so an enclosure of some kind is necessary.

Like you I'm planning on going vertical. I'll drill 4 holes in the case bottom, insert long screws, then put in spacers, mobo, spacers, mobo, etc. I like the look of your standoffs, I might go that route if I can find them.

I'm sure the plan will change as I build it.

[Prime95]

I just measured the interior of my case -- 13.5 inches high. Stacking 5 ITX boards will be a tight fit. I'll have about 2.6 inches per board. I don't know if that will allow for enough airflow. Stacking 4 ITX boards ought to be easy fit.

[bgbeuning]

I suggest the Cray-1 Theme.

https://www.parallella.org/2014/06/0...my-spare-time/

[Fred]

One more question for you George. When you settled on mini-itx vs micro-atx, I assume the main reason was power consumption. Have you tested (or do you know from experience) that the power difference outweighs initial price difference?

I see on PC partpicker that that they list about a 30 watt difference, which if true, makes the mini-itx a no-brainer. I was wondering though if that difference is just because a micro-atx has the potential to draw 30 more watts (with room for extra ram, expansion slots, etc), or with just a cpu and 2 sticks of ram, would micro-atx still always draw more than mini-itx? Makes me wonder where the extra 30 watts would be going....

[S485122]

I measured power usage at different multipliers on a i7-5820K. The conclusion is clear : under-clocking is more expensive : you need more energy to "produce" the same GHzd's. (It seems obvious to me now.)

But since the electricity used grows quicker than the speed, the nominal speed of the processor is almost as productive Watt per Watt as the speed with the highest multiplier.

Comp.	Freq.	Watt	GHzd/d	GHzd/W
5820K	idle	118		0
5820K	1500	177	29,9	0,169
5820K	2500	231	49,9	0,216
5820K	3375	276	67,3	0,244
5820K	4125	324	82,3	0,254

Where GHzd/d is the number of GHzdays produced daily and GHzd/W is the former number divided by the power used. (To get real GHzd/w units one would have to divide by 24*3600.)

On the other hand if the specifications of the T series of processors are to be believed, the i5-6600T should have much better productivity per watt than the i5-6500. What is needed is real life measures...

Jacob

[VBCurtis]

If you only changed multiplier and not voltage, this table makes sense. I don't know how speedstep adjusts voltages when multiplier is reduced; if your overclock settings had fixed voltage, this isn't representative of the actual decrease in power from running more slowly.

If voltage was set to auto-adjust, I suppose the table reflects the fixed overhead of non-CPU power use trumps the better work per CPU-watt from lower speed/lower voltage. Not an obvious conclusion, for sure- lower non-CPU overhead should tilt the outcome the other way, such as the mini-ITX boards mentioned elsewhere.

[Prime95]

Originally Posted by Fred

One more question for you George. When you settled on mini-itx vs micro-atx, I assume the main reason was power consumption.

I chose the mini-itx for its small size. I mentioned that this thing needs to be semi-portable, thus it was important for me to go with mini-itx as well as the picoPSUs. I find it difficult to believe that a micro-ATX board would use more power than a mini-itx, but I've not verified that in the real world.


BTW, my platinum PSU arrived today. Watts used for the 2 mobos dropped from 170 to 152. Very happy. When I get to 5 mobos running off this one PSU, the energy savings will pay for the new PSU in under 2 years.

Also, I ordered brass standoffs - thanks for the idea (I didn't know they made them that tall). It will take a while for those to get here from Asia.

[Prime95]

More progress. I got network booting to work! The ubuntu howtos had several errors, but googling found several other unofficial howtos. Combining the howtos resulted in a successful setup. All-in-all it only took a few hours. I have plenty of room on the 32GB USB stick to handle my planned for 3 CPUs. The only downside I see is I've created a single point of failure that can take down all the CPUs. On the plus side it ought to make backing up the 5 CPUs easier - only one USB stick to copy.

Removing the disk drive reduced power consumption to 144 watts. Nasty little disk drive was consuming 8 watts!

I'll be ordering three more mobos/cpus/ram/psus/cables in the coming weeks.

[Prime95]

More fiddling:

When you select non-Z OC of memory, the mobo automatically ups the RAM voltage to 1.35V. Since I intentionally paid $1 more for DDR4-2400 memory, I figured I could run that at the standard 1.2V. Thus, I forced the ram voltage in the BIOS to 1.2V. This saved 1 watt.

Next I tried undervolting the CPU. At -100mV the torture test would not start. At -70mV torture test ran OK for 3 minutes. Benchmark hung in one of the throughput tests. At -50mV torture test ran fine for 3 minutes before I quit it. Benchmark ran OK too. I'm going to try -40mV permanently. Saving for -70mV was 14 watts, savings for -50mV and -40mV was 12 watts.

Running a long torture test on both CPUs now. If this works, I'll have both the z170/DDR4-3200 and h110/OC'd DDR4-2133 boards running and consuming only ~120 watts. I'll torture results soon as well as accurate power readings.

Saving 12W per board makes me want to figure out a way to cram a 6th mobo in the case :)

[Prime95]

Both boards passed a 3 hour torture test both blend and small.

I reconfigured each for regular work but changed to multi-threaded DC until I'm satisfied they are producing good results. Unfortunately, one system hangs. I've changed that system to -35mV on the CPU and up to 1.25V on the RAM. If that fixes the hang, I'll go back and determine which voltage tweak was responsible.

The two systems combined are drawing 123 watts. Down from 144 where I started today. If stable, I'm very happy.

[Madpoo]

Originally Posted by Prime95

Both boards passed a 3 hour torture test both blend and small.

I reconfigured each for regular work but changed to multi-threaded DC until I'm satisfied they are producing good results. Unfortunately, one system hangs. I've changed that system to -35mV on the CPU and up to 1.25V on the RAM. If that fixes the hang, I'll go back and determine which voltage tweak was responsible.

The two systems combined are drawing 123 watts. Down from 144 where I started today. If stable, I'm very happy.

Cool. I can give you the query I've been using to get a list of suitable triple-checks needed (excluding any where you were one of the original testers).

I figure that ups your chances that you *should* match one of the previous results.

I think double-checks are a great way to start out with a system where you're making sure it works okay... saves a lot of hassle down the road if it ended up spitting out bad results.