mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware

Reply
 
Thread Tools
Old 2016-05-04, 02:42   #1
bgbeuning
 
Dec 2014

111111112 Posts
Default Mini ITX in server case

I was impressed by George and Fred's mini ITX builds so started on my own.
My needs are a little different. Mine does not need to be portable but does need
to generate minimum heat.

I started with a 4U server case and removed the disk cages.

Rosewill Server Chassis Rackmount Metal

For best airflow over the boards, I decided to stand the ASrock H110M-ITX
so the I/O rear panel is up. This means the Ethernet cable is sticking up out
of the case. The alternative would have the RAM acting as a wind break
blocking the air flow. I spent a lot of time on which way to orient the boards.

The video connector is about 1/8 inch too high for the case lid to fit.
I plan to raise the lid a little.

Since my boards are restting on the side with the PicoPSU, I needed PSU with the
+12 V wiring facing toward the RAM instead of away.

The case has three 200 CFM fans pushing air thru the case.
This was a bad idea. The fans are so loud I can hear them on
a different floor. Going to try 100 CFM case fans next.

I wanted to pack in the boards and the Intel stock CPU cooler is very tall.
By using different coolers, I put the boards 45 mm apart. Even 40 MM
might be possible. To get them close, I had to try out different CPU
coolers. This is the main data I wanted to share with everyone.

I tried these CPU coolers

Supermicro SNK-P0046P 1U Passive Cpu

Dynatron K1 1u Cooler

Dynatron T450 80mm 2 Ball CPU Cooler

Silverstone Tek Super Slim Profile CPU Cooler

The Dynatron K1 and Supermicro are passive (no fan) coolers.
The Dynatron T450 and SilverStone both have fans.
(All temperatures are from the Linux sensors(1) tool.)
(All temps are inside the case with the big fans blowing over them.)

The Dynatron T450 cooled an i5-6500 to 60 C, but it is too tall and barely fits in 45 mm.

The Silverstone cooled an i5-6400T to 49 C and an i5-6500 to 70 C.
When the Silverstone on the i5-6500 was outside the case with the big fans,
the temp was in the high 70s so I don't recommend that combination for a normal PC.

The Supermicro cooled an i5-6400T to 44 C. Off course both passive coolers
can only be used with major airflow across them.

The Dynatron K1 (passive) cooled an i5-6400T to 44 C also.

Both of the passive coolers came with mounting plates that rest on
Printed Circuit Board components which worries me.

Stock Intel cooler for i5-6500 was 55 C, i5-6400T was 42 C, i5-6600T was 52 C.
(These three are outside the case with no extra fans on them.)

At 45 MM spacing, it looks like 8 boards will fit in the front of the case.
In the back of the case, the power supply is in the way and (someday)
only 5 or 6 will fit there.

I am using network (PXE) booting with each board having its own NFS mount
for the filesystem. It seems to generate a ton of network traffic so I am looking
into having most of the root file system in RAM (200 MB out of 8 GB is small).
But network booting is a topic for another day. (Not a fan of systemd!)

I hope the CPU temperature data is useful to anyone else thinking about
this kind of build.
Attached Thumbnails
Click image for larger version

Name:	Image00001.jpg
Views:	274
Size:	617.0 KB
ID:	14314  
bgbeuning is offline   Reply With Quote
Old 2016-05-04, 04:37   #2
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

22·1,873 Posts
Default

Good start.

When you get a chance, please let us know what the power draw is for the 6400T and 6600T and what your iter/s are for a 4M FFT with non-Z RAM OC set to Sport+.

And how does that chance as you undervolt them. Right now, of the I'm reliably undervolting the 6500 by 1@30mV, 4@40mv, 1@50mV, and one at 70mV.

Have fun!

BTW, what linux tool to you use to monitor the network traffic?

Last fiddled with by Prime95 on 2016-05-04 at 04:44
Prime95 is online now   Reply With Quote
Old 2016-05-04, 10:52   #3
bgbeuning
 
Dec 2014

111111112 Posts
Default

My first clue about network load was the switch activity lights were solid on.
(Switch is Netgear 16 port GB .)

For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min). Linux has IP Accounting
I was going to try next.

I will be on vacation a few days so will take the power measurements
when back home. The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values.
Any heat reduction is most welcome, my basement is getting hot!
bgbeuning is offline   Reply With Quote
Old 2016-05-04, 14:13   #4
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT/BST)

2×5×587 Posts
Default

Quote:
Originally Posted by bgbeuning View Post
My first clue about network load was the switch activity lights were solid on.
(Switch is Netgear 16 port GB .)

For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min). Linux has IP Accounting
I was going to try next.

I will be on vacation a few days so will take the power measurements
when back home. The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values.
Any heat reduction is most welcome, my basement is getting hot!
Down is always safe. If you go too far you can reset the CMOS.
henryzz is offline   Reply With Quote
Old 2016-05-04, 16:54   #5
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

1D4416 Posts
Default

Quote:
Originally Posted by bgbeuning View Post
The BIOS voltage changes are a bit outside my
comfort zone, but maybe I could try the more conservative values
Even undervolting the maximum 100mV, the system will still boot.

What I do is something like this:

1) turn off all the C state stuff
2) turn on Sport+ mem overclocking
3) set mem voltage to 1.25 instead of 1.35 (never failed for me)
4) run a quick 5 minute torture test to confirm mem setting is good
5) choose an aggressive undervolting value.
6) Run a prime95 torture test for at least 2 hours. If it fails, increase voltage and repeat.
7) See if you can attain stability with memory at 1.20V (worked in 5 of 7 cases)
8) Run 2 to 4 double-checks.
9) Use web pages to change work type to first-time LL with 8% of time doing DC (to monitor machine stability on an on-going basis) and email notifications of problems.
Prime95 is online now   Reply With Quote
Old 2016-05-04, 17:15   #6
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

165048 Posts
Default

Quote:
Originally Posted by bgbeuning View Post
For a little better measurement I looked at the ifconfig data for bytes sent
and received, then divided by time since boot. I don't remember the exact
number but it did not seem so bad (maybe 200k / min).
My NFS server has sent and received 19.1GB in 87 hours. Or 3.66MB/min. 5 NFS clients plus my laptop has an SSH shell monitoring mprime output. Each NFS client writes an 8.75MB save file every 30 minutes -- this accounts for 1.46MB/min.

Suggestions for reducing network traffic are welcome.
Prime95 is online now   Reply With Quote
Old 2016-05-11, 12:10   #7
bgbeuning
 
Dec 2014

3·5·17 Posts
Default

Quote:
Originally Posted by Prime95 View Post
When you get a chance, please let us know what the power draw is for the 6400T and 6600T
I had posted the power draw and benchmark results on the "George Dream build" thread.
i5-6400T 45 W, i5-6600T 57 W, i5-6500 67 W.

Quote:
Originally Posted by Prime95 View Post
what your iter/s are for a 4M FFT with non-Z RAM OC set to Sport+.
I could not find this setting in the BIOS.
A quick google says it requires compatible DRAM for the menu item to show up.
I guess mine is not compatible.

Quote:
Originally Posted by Prime95 View Post
And how does that chance as you undervolt them. Right now, of the I'm reliably undervolting the 6500 by 1@30mV, 4@40mv, 1@50mV, and one at 70mV.
I looked at the BIOS option for this.
It has relative and absolute modes.
When using relative, enter negative values for undervolting?
bgbeuning is offline   Reply With Quote
Old 2016-05-11, 12:36   #8
bgbeuning
 
Dec 2014

3·5·17 Posts
Default

Quote:
Originally Posted by Prime95 View Post
Suggestions for reducing network traffic are welcome.
My understanding is lots of NFS traffic goes to checking if the files
have changed on the server since the last time the client accessed
the file (so called file attribute cache).
There are parameters to control how often NFS checks.
Google "actimeo".

I am considering using an in RAM root filesystem. My Ubuntu server
installs use about 26 MB for the root filesystem. Even making an 80 MB
RAM root image is 1% of RAM with 8GB.

You can mount just /home using NFS so the check point files get written
back to the NFS server.

If you want to save the /var/log files you can mount that too. Or configure
the client syslog server to forward logging to a box with a disk.

Here are some links for other people who have done RAM root images.

This guy focuses on building the smallest RAM image.

ram only pxe boot smallest diskless

These folks have 200 nodes and target easiest RAM image.

pegasus
bgbeuning is offline   Reply With Quote
Old 2016-05-11, 14:13   #9
Prime95
P90 years forever!
 
Prime95's Avatar
 
Aug 2002
Yeehaw, FL

1D4416 Posts
Default

Quote:
Originally Posted by bgbeuning View Post

I could not find this setting in the BIOS.


I looked at the BIOS option for this.
It has relative and absolute modes.
When using relative, enter negative values for undervolting?
You may need to flash the BIOS to the latest version to get non-Z OC capabilities.

To undervolt, use negative values for the offset.
Prime95 is online now   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Mini STX bgbeuning Hardware 1 2016-04-20 13:35
What can I fit in mini-ITX and a 120W PSU fivemack Hardware 2 2016-01-01 17:50
Mini-Projects MrHappy Data 56 2012-09-06 02:48
Riesel base 3 - mini-drive I gd_barnes Conjectures 'R Us 199 2009-09-30 18:44
Sierp base 3 - mini-drive Ib gd_barnes Conjectures 'R Us 43 2009-03-06 08:41

All times are UTC. The time now is 06:00.

Sat May 15 06:00:46 UTC 2021 up 37 days, 41 mins, 0 users, load averages: 3.11, 2.45, 2.26

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.