Running Prime on High Performance Computer Clustering

fxmulder · 2003-11-20, 14:45

Hi all,

I'm going to play a bit with HPCC clustering computing.

I have a bunch (~100) of server (bi-, quad-, ia64, and so on ..) , and I will work on making them a fully working HPCC system.

I start from scratch ... so it may take long time, before it say "hello world !"

I was just wondering if anyone have developed code that could run on a HPCC system ... I can offer the hardware & the configuration ... but would like to make this power available for Mersenne search, as I'm an old fan of this search ...

I'm not interested in any Glory ... so I will assign the computing resource to the person / team that will give me the nice piece of software :)

Cheers,

fxmulder

Prime Monster · 2003-11-20, 15:38

As far as I know the GIMPS client will not benefit from running on a cluster, apart from using the available CPU power in general. Its CPU and memory bandwidth requirements are such that running any part of the client across a high speed network will incure a terrible loss in performance. That means you will not benefit from having a special cluster based prime client. (Not sure if I explained that well

, other can probably do better)

On the other hand running individual clients assigned to each node in a cluster is very easy to do.

I am running an LTSP based cluster for GIMPS, with 24 nodes. Mostly P4s, but also PIIIs and Athlon XPs. This cluster runs only the prime client, so I have no other functionality on the cluster, but in theory an openmosix (or similar) cluster solution should make the cluster somewhat more usable for other applications as well.

Basically what I did was to have one machine set up as a common server, with all the disk/file storage, and then let the cluster nodes load the client from there. This happens automagically at boot time, due to use of LTSP and some recoding of the OS boot and startup code. (LTSP is the Linux Terminal Server Project).

I wish you the best of luck in getting a proper HPCC cluster up and running with prime though

PM

fxmulder · 2003-11-20, 16:03

Well,

you are rigth ... that's why I was asking for new code ;)

Indeed my idea (that may be not very smart) was to have multiple system working on the same number, the code would have split the iterations between the multiple nodes, scope to reduce the time to analyze the single number ... I know prime born with a different scope... analyzing many numbers at same time ... it's a question of perspective ... I need this just as a way to test my HPCC, just because I have nothing else to run on it .. so may be it's not that smart ... but what I was searching for :)

Prime Monster · 2003-11-20, 16:37

I believe someone is working on an SMP version of mprime or was it glucas . (Do a search for it on the boards here). That version would probably be the closest you would get to a version of prime that can be tailored for a cluster environment.

PM

Complex33 · 2003-11-20, 16:39

The problem with splitting up the computation of the LL test is that it is inherently serial, where by every iteration is dependent on the previous iterations result. What you propose may be better geared towards factoring where by each node could work on the same number but at a different level. I could see where this would make the factoring of numbers blazing fast, although factoring gives only negative prime results.

crash893 · 2003-11-20, 22:42

as stated above

the you dont need new code you need a new problem

do to the nature of the math you need to know what the answer was to the first problem to do the next

so you cant just jump in the middle

but what your wanting to do is in essancse what we are already doing by giving 1 number to diffrent comptuers and having them report back

its just on a diffrent scale than you want

2003-11-20, 14:45	#1
fxmulder Nov 2003 Montpellier - France 10₂ Posts	Running Prime on High Performance Computer Clustering Hi all, I'm going to play a bit with HPCC clustering computing. I have a bunch (~100) of server (bi-, quad-, ia64, and so on ..) , and I will work on making them a fully working HPCC system. I start from scratch ... so it may take long time, before it say "hello world !" I was just wondering if anyone have developed code that could run on a HPCC system ... I can offer the hardware & the configuration ... but would like to make this power available for Mersenne search, as I'm an old fan of this search ... I'm not interested in any Glory ... so I will assign the computing resource to the person / team that will give me the nice piece of software :) Cheers, fxmulder

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
A high-performance degree-6 root sieve	jasonp	Msieve	20	2010-06-11 22:40
how to tell if prime is running on a remote computer	crash893	Software	5	2005-09-12 13:26
Running factoring on one computer	Unregistered	PrimeNet	11	2005-07-09 04:38
computer seems to LAg when Prime95 is running	Firedog18	Hardware	2	2003-07-25 22:03
5% performance gain running NFSNET + PRIME on a P 4 HT	TauCeti	Hardware	1	2003-07-02 22:22

2003-11-20, 15:38	#2
Prime Monster Aug 2002 100000100₂ Posts	As far as I know the GIMPS client will not benefit from running on a cluster, apart from using the available CPU power in general. Its CPU and memory bandwidth requirements are such that running any part of the client across a high speed network will incure a terrible loss in performance. That means you will not benefit from having a special cluster based prime client. (Not sure if I explained that well , other can probably do better) On the other hand running individual clients assigned to each node in a cluster is very easy to do. I am running an LTSP based cluster for GIMPS, with 24 nodes. Mostly P4s, but also PIIIs and Athlon XPs. This cluster runs only the prime client, so I have no other functionality on the cluster, but in theory an openmosix (or similar) cluster solution should make the cluster somewhat more usable for other applications as well. Basically what I did was to have one machine set up as a common server, with all the disk/file storage, and then let the cluster nodes load the client from there. This happens automagically at boot time, due to use of LTSP and some recoding of the OS boot and startup code. (LTSP is the Linux Terminal Server Project). I wish you the best of luck in getting a proper HPCC cluster up and running with prime though PM

2003-11-20, 16:03	#3
fxmulder Nov 2003 Montpellier - France 2 Posts	Well, you are rigth ... that's why I was asking for new code ;) Indeed my idea (that may be not very smart) was to have multiple system working on the same number, the code would have split the iterations between the multiple nodes, scope to reduce the time to analyze the single number ... I know prime born with a different scope... analyzing many numbers at same time ... it's a question of perspective ... I need this just as a way to test my HPCC, just because I have nothing else to run on it .. so may be it's not that smart ... but what I was searching for :)

2003-11-20, 16:37	#4
Prime Monster Aug 2002 2²·5·13 Posts	I believe someone is working on an SMP version of mprime or was it glucas . (Do a search for it on the boards here). That version would probably be the closest you would get to a version of prime that can be tailored for a cluster environment. PM

2003-11-20, 16:39	#5
Complex33 Aug 2002 Texas 5×31 Posts	The problem with splitting up the computation of the LL test is that it is inherently serial, where by every iteration is dependent on the previous iterations result. What you propose may be better geared towards factoring where by each node could work on the same number but at a different level. I could see where this would make the factoring of numbers blazing fast, although factoring gives only negative prime results.

2003-11-20, 22:42	#6
crash893 Sep 2002 450₈ Posts	as stated above the you dont need new code you need a new problem do to the nature of the math you need to know what the answer was to the first problem to do the next so you cant just jump in the middle but what your wanting to do is in essancse what we are already doing by giving 1 number to diffrent comptuers and having them report back its just on a diffrent scale than you want