Seems to me you want two machines:
1) a desktop, a nice housebroken local machine to generate your video with a single 4 or more core sandy bridge CPU and a nice GPU, and 8 to 16Gig of RAM, and
2) a server class beast with multiple CPUs banished to the cool basement.

If you are generating your own power, a power budget certainly should be a concern...it will translate directly into either fuel or loss of capacity or required energy storage.

I'm curious why you would not want to run a small farm of boxes...they've gotten a lot more powerful, you'd end up with 5 or 10 of them on the $5K budget you mentioned. Having all those cores on the same bus makes for very fast intercommunication, nice on some of the more detailed factoring problems, but not needed for standard P95 work.

[QUOTE=fivemack;271772]What do you mean by a mirror?[/QUOTE]I would think that means a mirror for the data. For which I would still push a PCIe RAID card with SAS drives.

fivemack, as I said Intel do make some 8 core CPUs, the top of the line part is the X7560, 2.26 GHz and 24 MB L3 cache, based on the Nehalem arch but it is a native 8 core design. Four of these in a system would give a peak performance of 290 DP GFLOPS.

However, if AMD systems are to be considered then four of the 12 core Magny-Cours CPUs would give a higher theoretical performance. By way of example, four of the top end Opteron 6180 SE CPUs, 2.5 GHz and 12 MB L3 cache, (for a total of 48 cores) would give a peak performance of 480 DP GFLOPS.

I don't know how these would perform in the real world, but I'm sure some P95 benchmarks on them could be dug up. It may not be such a clear win for the AMD platform here since each CPU has half the L3 cache to use between 50% more cores, additionally the chips aren't a native 12 core design but rather two 6 core CPUs glued together.

One advantage of the two-box solution is that you can ask the one to mirror the other. For primenet type tasks, the performance requirements for a mirror are sufficiently minimal that a 10Megabit ethernet cable, or even dial-up, would work fine.

I'm not so sure about the business side there, but network attached storage is definitely something to consider. One huge advantage on the Linux side is that the installs aren't terribly painful, so if you keep a copy of your user data, a disk disaster isn't a big deal. If you have a lot of data, those USB hard drives are cheap, and very nice.

Lavalamp, what's the minimum Mobo I'd need to run one of those Magny-cours CPUs? I'm not quite ready to build box #2, but I'm thinking about it.

I have a computer with four Magny-Cours CPUs; it was quite expensive, I had to buy it as a system rather than as parts, and it's in a 1U case because workstation cases that support MEB-form-factor boards don't appear to be available.

If you want to run two Magny-Cours CPUs, there's a $430 board from Tyan available from newegg.com which fits in an EATX case (search for 'G34').

The Nehalem 8-core CPUs exist (and even Westmere 10-core CPUs), but since each CPU costs $4000 and the motherboard and system infrastructure round them costs the better part of $10000, they didn't seem relevant for this build.

[QUOTE=lavalamp;271796]I don't know how these would perform in the real world, but I'm sure some P95 benchmarks on them could be dug up.[/QUOTE]

They perform pretty poorly on individual Prime95 jobs; they are only 1.9GHz cores, it's just that there are an awful lot of them. You can't test a number in half a day by devoting 48 cores to the job.

The idea is a high powered beast, to be used in an office environment during daytime.

Power will be abundant, well in excess of what any machine demands / aircon capacity to cool it. Design is load at ~ 60 % of capacity ... plenty kVA headroom ...

The primary reason to avoid rows of low cost boxen is ... low cost boxen. I am in the bush in west africa, between the middle of nowhere and the end of the world ... one does not go to the corner store to buy a spare failed part ... everything must be imported, having the very best makes this project much easier ...

There are issues of 4 core performance when all 4 cores are saturated. An 8 core might be worse off, is there any evidence to support otherwise? Thus, a 4 x 4 core is an effective 12 core machine - would a 2 x 8 core be an effective 12 core, or perhaps more? I imagine there would be a better choice of 2 x boarts?

This discussion of boxen versus beasts reminds me of the debate about missiles versus tanks in war.

Two (or 10) boxen have an advantage over a single Beasten: You can easily afford a full set of spares for the boxen, and have it on hand. To be able to say the same for the first beast costs much, much more.
Then there's also the idea that boxen will degrade gracefully in the face of failure. One beast either works or doesn't. Remember RAID? The boxen implement RAIC -- Redundant array of inexpensive computers.

A quick look at server Mobos on newegg leads to a Supermicro brand Mobo with 32/128 Gig of DDR3 slots with a socket for a single Magny-cours 8 core CPU at about $250. Adding the CPU makes it about $500. To justify that over 2 Phenom II x6 boxes, at $500 each, I need to be doing something that needs and uses all that memory and connection bandwidth...that's not P95. It might be if I was doing the linear algebra for GNFS to factor M1061 or larger.

**********
Pagefault, I think you are confusing quality and reliability with the amount of computing a box can do. If your replacement parts are super-expensive, requiring 10 pounds of baksheesh and a difficult visit with 10 local village headmen and petty tyrants, then you want to get the most reliable hardware you can, along with spare parts, then run it conservatively. I promise you that any new computer you can get, including a single boxen or reasonable laptop, will have enough power for the business end.

This is the rational end speaking. GIMPS isn't 100% rational, however, so fivemack has his beast, and there are some other beasts owned by names on this forum you will recognise. I encourage you to look at the rational end and then decide if you are willing to deal with the consequences of being irrational.

[QUOTE=PageFault;271841]The idea is a high powered beast, to be used in an office environment during daytime.

Power will be abundant, well in excess of what any machine demands / aircon capacity to cool it. Design is load at ~ 60 % of capacity ... plenty kVA headroom ...

The primary reason to avoid rows of low cost boxen is ... low cost boxen. I am in the bush in west africa, between the middle of nowhere and the end of the world ... one does not go to the corner store to buy a spare failed part ... everything must be imported, having the very best makes this project much easier ...[/QUOTE]

I think at that point I would buy three lowest-end macbook airs with Applecare; take one with you to use, take another with you to keep in a sealed box with silica gel and all the normal precautions, leave a third one and a load of shipping labels with a trusted friend in Europe. If yours breaks, send it back to Europe to be fixed by applecare, get the new one out of its box, and get your European friend to send #3 to you to be kept in the resealed box with new silica gel.

The idea is to use reliable machines (so ones of a kind sold by the hundred-thousand, with manufacturers' warranty, and without moving parts or software-compatibility issues) which are cheap and small enough to be replicated and to be replaced-and-refurbished rather than repaired.

If you need a lot more storage, get firewire or thunderbolt disc arrays with the same threefold replication.

If you also want prime95, buy hosted compute from hetzner.de (50 euros + 50 euros per month, for an i7/920 box in a data centre somewhere in Germany) and run prime95 on that. I would say that, in an environment where repair is difficult and costly, the last thing you want is to get overpowered self-assembled machines and then impose extra load on them by running prime95 24/7.

But I have not tried working in the kind of environment you mention and I may well be missing something absolutely fundamental.

Fivemack, one question about your plan:
How is the data on this macbook to be backed up? Assume that the internet "cloud" is not an option, due to reliability and privacy concerns....

Well, quality, reliable parts tend to run without downtime, with a long service life, and happen to be at the high end of what is available. That correlates very strongly with the amount of computing it can do.

What has confused the hell out of all those here is that I want the reliability, guaranteed in the server market, with the overkill for prime95. It is no coincidence that the server market also provides the overkill I want ...

Nothing runs 100 % forever. Cheap boxen don't run very long - been there, done that before, in much friendlier environments. And I don't have time to build replacement boxen every week, even if I have a crate of parts on hand. That time is for whisky ...

[QUOTE=Christenson;271855]

Pagefault, I think you are confusing quality and reliability with the amount of computing a box can do. If your replacement parts are super-expensive, requiring 10 pounds of baksheesh and a difficult visit with 10 local village headmen and petty tyrants, then you want to get the most reliable hardware you can, along with spare parts, then run it conservatively. I promise you that any new computer you can get, including a single boxen or reasonable laptop, will have enough power for the business end.

This is the rational end speaking. GIMPS isn't 100% rational, however, so fivemack has his beast, and there are some other beasts owned by names on this forum you will recognise. I encourage you to look at the rational end and then decide if you are willing to deal with the consequences of being irrational.[/QUOTE]

@Fivemack: For that macbook air, the one in the silica gel, how do you keep the batteries in the best of shape?

In my own, limited personal experience of the last decade, laptops die most frequently...I have 3 or 4 dead laptops around with the inevitable battery, screen, or motherboard issues. Desktop systems (these are name brand ones from Dell, HP) lose displays (one flat panel one of my students repaired, one CRT) or hard drives (2-3 personal ones several at work). The servers at work have had several issues with memory, hard drives, and tape drives that I am aware of; I don't maintain these, so that's a very much "in passing".
Hard drives and power supplies seem to be the issues with the crop of recycled boxes that the students had donated to them.

Windows bloat itself killed one of my panasonic Toughbooks; this is a bit of hardware you should seriously consider...it will run even if your air conditioning doesn't and you are outside at midday in the desert. It will also set you back more than your average laptop, like a grand or three.

Heat is your enemy, and running P95 generates heat. I think Lavalamp had it right on my 6-core beast when he told me not to get cheap on the power supply, and get a box with lots of fans. Servers concentrate the heat, and therefore, although their initial quality may be higher, the heat (especially if you run P95 constantly) degrades them.