#1

[Xyzzy]

We don't know!



When you are doing big memory calculations for weeks/months isn't a bit of extra reliability worth the cost?

[axn]

Originally Posted by Xyzzy

PPS - ATX or mATX AKA 64GB or 128GB?

Future proof = 128GB.

[masser]

Originally Posted by Xyzzy

We don't know!



When you are doing big memory calculations for weeks/months isn't a bit of extra reliability worth the cost?

I don't know either, but here's a link on the topic.

[Xyzzy]

From that link's comment section:

I think this depends on the kind of computation you’re doing. Statistics I’ve heard on soft-errors of this kind are on the order of 1 bit error per terabit-year. If you’re building a cluster that’s going to crunch on a single problem for months at a time, where one bit error during the calculation will destroy months of work, and where multiple terabytes of RAM are in use on that calculation, ECC is absolutely worth it. That’s why getting ECC memory support into nVidia’s GPUs was such a big deal for projects like the Titan supercomputer, which has 693.5TiB of RAM running 24/7.

For web server applications like this where the worst thing that happens is one page loads incorrectly, I wouldn’t worry about it so much.

[Xyzzy]

The CPU arrived yesterday.

The packaging is ridiculous and wasteful.

Note that this is just the CPU. There isn't a fan in that package!

[Xyzzy]

We ordered some more parts today. We decided to go with the ATX motherboard.

Have:
1 CPU: AMD Threadripper 1920X 3.5 GHz 12-Core Processor ($199.99 @ Amazon)

Ordered:
1 CPU Cooler: Noctua NH-U14S TR4-SP3 82.52 CFM CPU Cooler ($79.90 @ Amazon)
1 Case: Corsair 275Q ATX Mid Tower Case ($99.99 @ Best Buy)
1 Power Supply: SeaSonic PRIME Ultra Platinum 550 W 80+ Platinum Certified Fully Modular ATX Power Supply ($112.98 @ Newegg)

Need to order:
1 Motherboard: ASRock X399 Taichi ATX sTR4 Motherboard ($319.99 @ Newegg)
8 Memory: Crucial 16 GB (1 x 16 GB) DDR4-2666 Memory ($799.92 @ Newegg)
1 Storage: Crucial P1 1 TB M.2-2280 NVME Solid State Drive ($99.99 @ Newegg)

Notes:
The CPU cooler should have 5mm clearance in the case we have chosen.
From a prior build we have a spare Seasonic 4+4 CPU power cable that will be needed. The ASRock motherboard takes a 4+4 and a 4! (See page 7 of PDF: #2 and #9)

Questions:
For NFS work, should the data be on a separate SSD?
Should we buy all 8 memory sticks ($$$) at one time for compatibility?
ECC?
-->$67 × 8 = $536 Crucial 16GB DDR4-3200 UDIMM
-->$111 × 8 = $888 Crucial 16GB DDR4-2666 VLP ECC UDIMM

[VBCurtis]

Originally Posted by Xyzzy

Questions:
For NFS work, should the data be on a separate SSD?
Should we buy all 8 memory sticks ($$$) at one time for compatibility?
ECC?
-->$67 × 8 = $536 Crucial 16GB DDR4-3200 UDIMM
-->$111 × 8 = $888 Crucial 16GB DDR4-2666 VLP ECC UDIMM

I don't see any reason for a separate SSD for NFS data. I also would choose to spend $300 more on a more-core CPU over $300 extra for ECC memory. I don't run into matrix-fail errors on my non-ECC machine that ever makes me think "gee, if only I had ECC, I wouldn't be repeating this 10-day calculation"... but even if I did repeat once such calculation every 50 jobs, I wouldn't come out ahead on ECC memory versus adding RAM to my previous machine so it could solve big matrices too.
In your case, the CPU is already chosen, so the $300 can just stay in your pocket. That said, if your plan is to help Greg with the 16e queue, your jobs will be 2-5 months in length; while I've solved exactly one matrix of that size so I don't have a basis to judge, I still wouldn't spring for ECC. If you're going to join us with the 15e queue, your jobs are a few weeks each and trivial to repeat in the super-rare case something fails.

[retina]

IMO reliability and dependability trumps a modest amount of extra cost. If my system says 63 hours to completion of the current job, then I have piece of mind that it will finish in 63 hours and give an accurate result.

Running something again because it failed due to some transient error is frustrating, annoying and stressful. So the cost of undependable gear is a combination of money for extra electricity, missed deadlines and my sanity. Compared to a small amount of extra cash upfront it is much cheaper to have ECC memory. IMO of course.

ETA: Plus with non-ECC memory, when your matrix fails to solve correctly is it because your have bad sieve data, or you set the parameters wrongly, or should you just try again because it might have been a memory failure. I'd much rather eliminate the last option immediately and know it is something else that I need to fix.

[chalsall]

Originally Posted by VBCurtis

I don't see any reason for a separate SSD for NFS data.

Just to share, I'm soon going to be taking delivery of a Sabrent 256GB Rocket NVMe PCIe M.2 2280 Internal SSD.

I'm going to run an experiment, and use most of it for a swap partition. I sometimes want to do Blender rendering jobs which won't fit in my main workstation's RAM, so I have to spin up a "cloud" instance.

I know that many people say that using an SSD for swap is ill-advised because of "wear", but I think the technology has advanced enough that it should work for long enough. Heck, these things are cheap enough that if I have to replace it every year or so, no big deal.

I'm also going to "underprovision" the partitions, to allow the device to have additional blocks to migrate to when needed. (Yes, I also know that this is no longer generally needed, as most modern SSDs already have a relatively large amount of spare blocks already available, but it wouldn't hurt.)

Really looking forward to seeing how it works out.

Has anyone else here tried doing something like this? If so, what was your experience? Any advice?

[PhilF]

I read a report somewhere about an outfit trying to make SSD's fail. They constantly read and write data to them, 24/7/365. A few failed early, a few failed around the time they were supposed to, but the vast majority are still going strong long after their expected lifetime.

There are many different underlying SSD technologies, so your mileage may vary. I don't baby them in any way and have never had a problem.

That said, there's an aspect to them that most aren't aware of. That is long term retention. They are not well suited to store a bunch of info on, then put away in a drawer un-powered for years.

[chalsall]

Originally Posted by PhilF

They are not well suited to store a bunch of info on, then put away in a drawer un-powered for years.

Thanks for your above; encouraging.

And I did know about the long-term retention issue. But then, frankly, I've had the same problem with SeaCrap drives. Seriously -- I have had several that I used for "cold storage", and even though they sealed in an antistatic bag (in a safe), they still did the "click-of-death" thing when I tried to bring them online after a couple of years.

Fortunately, I always have redundancy in my backups. And I haven't purchased a SeaCrap drive for several years now.