View Single Post
Old 2020-10-27, 16:38   #47
kwalisch's Avatar
Sep 2015

2010 Posts
Default We have successfully verified PrimePi(1e28)

I am happy to announce that David and myself have successfully verified PrimePi(1e28) = 157589269275973410412739598. This time the computation took only 12.8 CPU core years (only physical CPU cores are counted). Hence the verification run took only half as much time as our first run and also only half as much time as our PrimePi(1e27) computation from 2015! The reason for this new speedup is that the verification run was computed entirely on new AMD EPYC servers (2nd Gen, codename "Rome") with much more memory.

Because I know that many people in this forum are hardware enthusiasts (just like myself), here are some more details about the server that I have used for PrimePi(1e28):

Computing new prime counting function records takes a long time. Up until 2019 I rented servers on AWS, however this became increasingly expensive as AWS kept on increasing prices over the past few years. Currently the spot price for an 48 core (96 threads) Intel server on AWS is about 0.94$ per hour, hence the cost per month is about 700$. For this reason I decided to buy a server myself at the beginning of 2020 for running primecount 24/7. I wanted to buy a dual socket AMD EPYC server because of its large number of CPU cores and memory channels. After evaluating the PROs and CONs of the different CPU models I picked the AMD EPYC 7642 with 48 cores and 96 threads. My idea was to initially buy only 1 CPU and then jointly compute PrimePi(1e28) with David and if that computation succeeds I would buy a 2nd AMD EYPC CPU along with more memory for computing PrimePi(1e29).

I also bought the pre-assembled "A+ Server AS-4023S-TRT" server tower from Supermicro (see attached screenshot), this way I only needed to install the CPU, memory and hard-disk. However once it was up and running there were two problems : the cooling was insufficient, the airflow inside this case is pretty bad as the case is narrow and contains a lot of pieces that block the air flow. The second issue was the noise level, it was incredibly loud. The server is located in my flat's basement and I measured a noise level of up to 70 decibel inside the room and up to 50 decibel outside the room. By Luxemburgish law the maximum allowed noise level is 40 decibel outside the room.

I tried a lot of stuff to reduce the noise level (e.g. noise dampening panels) but with little success. Eventually I realized that the tiny 80mm fans of my server that were spinning extremely fast (up to 9000 rpm) were the root cause of the high noise level. So in order to make progress and significantly reduce the noise level I bought the "Fractal Design Define 7 XL" gaming PC case (see screenshots) which allowed me to install many large fans that are spinning more slowly (~ 1600 rpm). With this new case the cooling and noise issues are now fixed, the noise level is now around 35 decibel outside of my basement. Another issue I had was that my memory (512 GiB ECC DDR4 3200 MHz) was running much hotter than I have ever seen on my desktop PCs. The memory reached up to 70 degree Celsius, this was even hotter than my CPU! For about 20$ I ended up buying a cheap memory cooler: the "Corsair Vengeance Airflow Memory" (see screenshots). It works great, it reduces my RAM temps by about 15C.

I had a lot of fun building my own tiny little data center and I have learnt a lot about servers, network devices, cooling, sensors, ... The server has now been running 24/7 for months without a single issue. The thing I probably like most about my server is that I have configured it like a cloud server: it has no graphics card, there is no monitor and keyboard attached to it. Instead I have installed Ubuntu server and sshd (OpenSSH server) on it. This allows me to conveniently connect to my server from my laptop using ssh even when I am on holiday.
Attached Thumbnails
Click image for larger version

Name:	Supermicro Server Tower.jpg
Views:	229
Size:	428.7 KB
ID:	23648   Click image for larger version

Name:	Fractal Design Define 7 XL.jpg
Views:	223
Size:	392.2 KB
ID:	23649   Click image for larger version

Name:	Corsair Memory Cooling.jpg
Views:	211
Size:	14.7 KB
ID:	23650   Click image for larger version

Name:	Cooling system.png
Views:	212
Size:	306.2 KB
ID:	23652   Click image for larger version

Name:	IPMI Server sensors.jpg
Views:	212
Size:	340.8 KB
ID:	23653  

Last fiddled with by kwalisch on 2020-10-27 at 16:51
kwalisch is offline   Reply With Quote