Mini ITX in server case

bgbeuning · 2016-05-04, 02:42

I was impressed by George and Fred's mini ITX builds so started on my own.
My needs are a little different. Mine does not need to be portable but does need
to generate minimum heat.

I started with a 4U server case and removed the disk cages.

Rosewill Server Chassis Rackmount Metal

For best airflow over the boards, I decided to stand the ASrock H110M-ITX
so the I/O rear panel is up. This means the Ethernet cable is sticking up out
of the case. The alternative would have the RAM acting as a wind break
blocking the air flow. I spent a lot of time on which way to orient the boards.

The video connector is about 1/8 inch too high for the case lid to fit.
I plan to raise the lid a little.

Since my boards are restting on the side with the PicoPSU, I needed PSU with the
+12 V wiring facing toward the RAM instead of away.

The case has three 200 CFM fans pushing air thru the case.
This was a bad idea. The fans are so loud I can hear them on
a different floor. Going to try 100 CFM case fans next.

I wanted to pack in the boards and the Intel stock CPU cooler is very tall.
By using different coolers, I put the boards 45 mm apart. Even 40 MM
might be possible. To get them close, I had to try out different CPU
coolers. This is the main data I wanted to share with everyone.

I tried these CPU coolers

Supermicro SNK-P0046P 1U Passive Cpu

Dynatron K1 1u Cooler

Dynatron T450 80mm 2 Ball CPU Cooler

Silverstone Tek Super Slim Profile CPU Cooler

The Dynatron K1 and Supermicro are passive (no fan) coolers.
The Dynatron T450 and SilverStone both have fans.
(All temperatures are from the Linux sensors(1) tool.)
(All temps are inside the case with the big fans blowing over them.)

The Dynatron T450 cooled an i5-6500 to 60 C, but it is too tall and barely fits in 45 mm.

The Silverstone cooled an i5-6400T to 49 C and an i5-6500 to 70 C.
When the Silverstone on the i5-6500 was outside the case with the big fans,
the temp was in the high 70s so I don't recommend that combination for a normal PC.

The Supermicro cooled an i5-6400T to 44 C. Off course both passive coolers
can only be used with major airflow across them.

The Dynatron K1 (passive) cooled an i5-6400T to 44 C also.

Both of the passive coolers came with mounting plates that rest on
Printed Circuit Board components which worries me.

Stock Intel cooler for i5-6500 was 55 C, i5-6400T was 42 C, i5-6600T was 52 C.
(These three are outside the case with no extra fans on them.)

At 45 MM spacing, it looks like 8 boards will fit in the front of the case.
In the back of the case, the power supply is in the way and (someday)
only 5 or 6 will fit there.

I am using network (PXE) booting with each board having its own NFS mount
for the filesystem. It seems to generate a ton of network traffic so I am looking
into having most of the root file system in RAM (200 MB out of 8 GB is small).
But network booting is a topic for another day. (Not a fan of systemd!)

I hope the CPU temperature data is useful to anyone else thinking about
this kind of build.

Prime95 · 2016-05-04, 04:37

Good start.

When you get a chance, please let us know what the power draw is for the 6400T and 6600T and what your iter/s are for a 4M FFT with non-Z RAM OC set to Sport+.

And how does that chance as you undervolt them. Right now, of the I'm reliably undervolting the 6500 by 1@30mV, 4@40mv, 1@50mV, and one at 70mV.

Have fun!

BTW, what linux tool to you use to monitor the network traffic?

bgbeuning · 2016-05-04, 10:52

My first clue about network load was the switch activity lights were solid on.
(Switch is Netgear 16 port GB .)

For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min). Linux has IP Accounting
I was going to try next.

I will be on vacation a few days so will take the power measurements
when back home. The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values.
Any heat reduction is most welcome, my basement is getting hot!

henryzz · 2016-05-04, 14:13

Quote:

Originally Posted by bgbeuning

My first clue about network load was the switch activity lights were solid on.
(Switch is Netgear 16 port GB .)

For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min). Linux has IP Accounting
I was going to try next.

I will be on vacation a few days so will take the power measurements
when back home. The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values.
Any heat reduction is most welcome, my basement is getting hot!

Down is always safe. If you go too far you can reset the CMOS.

Prime95 · 2016-05-04, 16:54

Quote:

Originally Posted by bgbeuning

The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values

Even undervolting the maximum 100mV, the system will still boot.

What I do is something like this:

1) turn off all the C state stuff
2) turn on Sport+ mem overclocking
3) set mem voltage to 1.25 instead of 1.35 (never failed for me)
4) run a quick 5 minute torture test to confirm mem setting is good
5) choose an aggressive undervolting value.
6) Run a prime95 torture test for at least 2 hours. If it fails, increase voltage and repeat.
7) See if you can attain stability with memory at 1.20V (worked in 5 of 7 cases)
8) Run 2 to 4 double-checks.
9) Use web pages to change work type to first-time LL with 8% of time doing DC (to monitor machine stability on an on-going basis) and email notifications of problems.

Prime95 · 2016-05-04, 17:15

Quote:

Originally Posted by bgbeuning

For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min).

My NFS server has sent and received 19.1GB in 87 hours. Or 3.66MB/min. 5 NFS clients plus my laptop has an SSH shell monitoring mprime output. Each NFS client writes an 8.75MB save file every 30 minutes -- this accounts for 1.46MB/min.

Suggestions for reducing network traffic are welcome.

bgbeuning · 2016-05-11, 12:10

Quote:

Originally Posted by Prime95

When you get a chance, please let us know what the power draw is for the 6400T and 6600T

I had posted the power draw and benchmark results on the "George Dream build" thread.
i5-6400T 45 W, i5-6600T 57 W, i5-6500 67 W.

Quote:

Originally Posted by Prime95

what your iter/s are for a 4M FFT with non-Z RAM OC set to Sport+.

I could not find this setting in the BIOS.
A quick google says it requires compatible DRAM for the menu item to show up.
I guess mine is not compatible.

Quote:

Originally Posted by Prime95

And how does that chance as you undervolt them. Right now, of the I'm reliably undervolting the 6500 by 1@30mV, 4@40mv, 1@50mV, and one at 70mV.

I looked at the BIOS option for this.
It has relative and absolute modes.
When using relative, enter negative values for undervolting?

bgbeuning · 2016-05-11, 12:36

Quote:

Originally Posted by Prime95

Suggestions for reducing network traffic are welcome.

My understanding is lots of NFS traffic goes to checking if the files
have changed on the server since the last time the client accessed
the file (so called file attribute cache).
There are parameters to control how often NFS checks.
Google "actimeo".

I am considering using an in RAM root filesystem. My Ubuntu server
installs use about 26 MB for the root filesystem. Even making an 80 MB
RAM root image is 1% of RAM with 8GB.

You can mount just /home using NFS so the check point files get written
back to the NFS server.

If you want to save the /var/log files you can mount that too. Or configure
the client syslog server to forward logging to a box with a disk.

Here are some links for other people who have done RAM root images.

This guy focuses on building the smallest RAM image.

ram only pxe boot smallest diskless

These folks have 200 nodes and target easiest RAM image.

pegasus

Prime95 · 2016-05-11, 14:13

Quote:

Originally Posted by bgbeuning

I could not find this setting in the BIOS.

I looked at the BIOS option for this.
It has relative and absolute modes.
When using relative, enter negative values for undervolting?

You may need to flash the BIOS to the latest version to get non-Z OC capabilities.

To undervolt, use negative values for the offset.

2016-05-04, 02:42	#1
bgbeuning Dec 2014 377₈ Posts	Mini ITX in server case I was impressed by George and Fred's mini ITX builds so started on my own. My needs are a little different. Mine does not need to be portable but does need to generate minimum heat. I started with a 4U server case and removed the disk cages. Rosewill Server Chassis Rackmount Metal For best airflow over the boards, I decided to stand the ASrock H110M-ITX so the I/O rear panel is up. This means the Ethernet cable is sticking up out of the case. The alternative would have the RAM acting as a wind break blocking the air flow. I spent a lot of time on which way to orient the boards. The video connector is about 1/8 inch too high for the case lid to fit. I plan to raise the lid a little. Since my boards are restting on the side with the PicoPSU, I needed PSU with the +12 V wiring facing toward the RAM instead of away. The case has three 200 CFM fans pushing air thru the case. This was a bad idea. The fans are so loud I can hear them on a different floor. Going to try 100 CFM case fans next. I wanted to pack in the boards and the Intel stock CPU cooler is very tall. By using different coolers, I put the boards 45 mm apart. Even 40 MM might be possible. To get them close, I had to try out different CPU coolers. This is the main data I wanted to share with everyone. I tried these CPU coolers Supermicro SNK-P0046P 1U Passive Cpu Dynatron K1 1u Cooler Dynatron T450 80mm 2 Ball CPU Cooler Silverstone Tek Super Slim Profile CPU Cooler The Dynatron K1 and Supermicro are passive (no fan) coolers. The Dynatron T450 and SilverStone both have fans. (All temperatures are from the Linux sensors(1) tool.) (All temps are inside the case with the big fans blowing over them.) The Dynatron T450 cooled an i5-6500 to 60 C, but it is too tall and barely fits in 45 mm. The Silverstone cooled an i5-6400T to 49 C and an i5-6500 to 70 C. When the Silverstone on the i5-6500 was outside the case with the big fans, the temp was in the high 70s so I don't recommend that combination for a normal PC. The Supermicro cooled an i5-6400T to 44 C. Off course both passive coolers can only be used with major airflow across them. The Dynatron K1 (passive) cooled an i5-6400T to 44 C also. Both of the passive coolers came with mounting plates that rest on Printed Circuit Board components which worries me. Stock Intel cooler for i5-6500 was 55 C, i5-6400T was 42 C, i5-6600T was 52 C. (These three are outside the case with no extra fans on them.) At 45 MM spacing, it looks like 8 boards will fit in the front of the case. In the back of the case, the power supply is in the way and (someday) only 5 or 6 will fit there. I am using network (PXE) booting with each board having its own NFS mount for the filesystem. It seems to generate a ton of network traffic so I am looking into having most of the root file system in RAM (200 MB out of 8 GB is small). But network booting is a topic for another day. (Not a fan of systemd!) I hope the CPU temperature data is useful to anyone else thinking about this kind of build. Attached Thumbnails

2016-05-04, 04:37	#2
Prime95 P90 years forever! Aug 2002 Yeehaw, FL 2³·919 Posts	Good start. When you get a chance, please let us know what the power draw is for the 6400T and 6600T and what your iter/s are for a 4M FFT with non-Z RAM OC set to Sport+. And how does that chance as you undervolt them. Right now, of the I'm reliably undervolting the 6500 by 1@30mV, 4@40mv, 1@50mV, and one at 70mV. Have fun! BTW, what linux tool to you use to monitor the network traffic? Last fiddled with by Prime95 on 2016-05-04 at 04:44

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Mini STX	bgbeuning	Hardware	1	2016-04-20 13:35
What can I fit in mini-ITX and a 120W PSU	fivemack	Hardware	2	2016-01-01 17:50
Mini-Projects	MrHappy	Data	56	2012-09-06 02:48
Riesel base 3 - mini-drive I	gd_barnes	Conjectures 'R Us	199	2009-09-30 18:44
Sierp base 3 - mini-drive Ib	gd_barnes	Conjectures 'R Us	43	2009-03-06 08:41

2016-05-04, 10:52	#3
bgbeuning Dec 2014 FF₁₆ Posts	My first clue about network load was the switch activity lights were solid on. (Switch is Netgear 16 port GB .) For a little better measurement I looked at the ifconfig data for bytes sent and received, then divided by time since boot. I don't remember the exact number but it did not seem so bad (maybe 200k / min). Linux has IP Accounting I was going to try next. I will be on vacation a few days so will take the power measurements when back home. The BIOS voltage changes are a bit outside my comfort zone, but maybe I could try the more conservative values. Any heat reduction is most welcome, my basement is getting hot!