mersenneforum.org  

Go Back   mersenneforum.org > Prime Search Projects > No Prime Left Behind

Closed Thread
 
Thread Tools
Old 2008-03-04, 06:11   #12
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3·2,083 Posts
Default

Quote:
Originally Posted by gd_barnes View Post
I was thinking for this rally, would it make sense to have Beyond running on one server and the rest of us on another server? If Beyond's machines crash the server he is on, at least he can lesson the load a little bit on that one server while still having substantial tests going on and also having the least impact to all of us.


Gary
Hmm...interesting idea. Maybe for this rally, Beyond can go on one of Carlos's servers (since they need to be cleaned anyway), and the rest of us can do IronBits' port 5000 server as was originally planned? I know Carlos's servers can't handle as much of a load as IronBits' can, but they should be able to handle Beyond just fine as long as everybody else isn't on the same server. (Remember, when we only had Carlos's servers, they were able to handle the load of Beyond as well as a bunch of other people, as long as we weren't doing a rally--so Beyond alone shouldn't be a problem for them.)

Last fiddled with by mdettweiler on 2008-03-04 at 06:13
mdettweiler is offline  
Old 2008-03-04, 09:59   #13
kar_bon
 
kar_bon's Avatar
 
Mar 2006
Germany

22·691 Posts
Default

i think, and IronBits agreed, it's not the numerous amount of Beyonds clients (70 or more cores), it is his proxy:
he reserves a wide range (many k/n-pairs) at once and when they all ready to submit, the server gets too many connections without closing (the resultfiles show that over some minutes there are 8-10 results submitted per second!)
see also the graph: peaks at Feb24, Feb27 and Feb29 at the scoring page with more than 5000 pairs per hour!
so it think this can crash every LLRnet server!
kar_bon is offline  
Old 2008-03-04, 11:33   #14
Beyond
 
Dec 2002

22·3·37 Posts
Default

Quote:
Originally Posted by kar_bon View Post
i think, and IronBits agreed, it's not the numerous amount of Beyonds clients (70 or more cores), it is his proxy:
he reserves a wide range (many k/n-pairs) at once and when they all ready to submit, the server gets too many connections without closing (the resultfiles show that over some minutes there are 8-10 results submitted per second!)
see also the graph: peaks at Feb24, Feb27 and Feb29 at the scoring page with more than 5000 pairs per hour!
so it think this can crash every LLRnet server!
I'm not convinced it is the proxy.

The outcome is the same if I cache 10 workunits on the proxy and 10 on each client or 1000 on the proxy and 100 on each client. I've tryed dozens of different combinations. Also tried setting uploads from 10 to 50 to 100 at a time.

It is only a matter of time depending on the caching scheme employed, that I start seeing a lot of "no knpairs recieved from server" messages on the proxy. Usually within 10 to 15 minutes of the no knpairs recieved messages, I get "cannot connect with server" messages. This is the point were the server goes down or hits the max connection limits and locks everyone out.

If I'm not here to shutdown and restart the proxy, then the results start to pile up quickly. When I finally get to do a restart after a prolonged period you will see the hugh spike in returned results, usually in the thousands.

So from my end it appears the server cannot handle all the request for more work.

ATM, I'm working on getting a large file (manual reservation) to put on the proxy to test for problems on this end.

Last fiddled with by Beyond on 2008-03-04 at 11:35
Beyond is offline  
Old 2008-03-04, 16:33   #15
henryzz
Just call me Henry
 
henryzz's Avatar
 
"David"
Sep 2007
Cambridge (GMT)

5,647 Posts
Default

surely beyond doing manual reservation through his proxy would be the best solution
henryzz is offline  
Old 2008-03-04, 17:41   #16
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

186916 Posts
Default

Quote:
Originally Posted by Beyond View Post
...
ATM, I'm working on getting a large file (manual reservation) to put on the proxy to test for problems on this end.
Please note: if you're using a manual reservation file in your proxy, you'll want to set your llr-serverconfig.txt so that the proxy behaves as a proper server, otherwise it will try to return the results from your manual reservation file to either IronBits' server when they're completed.
mdettweiler is offline  
Old 2008-03-06, 05:00   #17
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

100111011001012 Posts
Default

I want to get this decided ahead of time to avoid problems. Questions for the upcoming rally:

1. Does it make sense to have Beyond on one server and the rest of us on another?

2. Since Beyond is working on a proxy server, is there still an issue with us needing more than one server for the rally? If not, is submitting through a proxy server for a rally allowed? I'm assuming his k/n pairs would be time stamped when sent en-mass to one of our servers. Is that correct?

3. If we need 2 servers, should Beyond do IronBits' server 5000 on k=300-400 while the rest of us do Carlos' server 300 on k=400-1001? With Ironbits' server having more memory, I think that works better than the reverse scenario.

All you computer techie folks, please weigh in on your thoughts here.


Thanks,
Gary

Last fiddled with by gd_barnes on 2008-03-06 at 05:03
gd_barnes is offline  
Old 2008-03-06, 06:04   #18
IronBits
I ♥ BOINC!
 
IronBits's Avatar
 
Oct 2002
Glendale, AZ. (USA)

3×7×53 Posts
Default

What ever you want to do is fine with me.
If you need another port opened up on the server for a different range, that's easy to do, just let me know.

Beyond is working out a server to server communications problem...
If he put 1/3 on Carlos, 1/3 on port 500 and 1/3 on port 5000, using 3 personal proxy servers, he might do ok. ;)

Last fiddled with by IronBits on 2008-03-06 at 06:05
IronBits is offline  
Old 2008-03-06, 06:18   #19
mdettweiler
A Sunny Moo
 
mdettweiler's Avatar
 
Aug 2007
USA (GMT-5)

3×2,083 Posts
Default

(my reply is interspersed throughout Gary's original message)
Quote:
Originally Posted by gd_barnes View Post
I want to get this decided ahead of time to avoid problems. Questions for the upcoming rally:

1. Does it make sense to have Beyond on one server and the rest of us on another?
I think so, even if it's only to make sure that we don't all go down if he crashes the server, or vice versa.

Quote:
2. Since Beyond is working on a proxy server, is there still an issue with us needing more than one server for the rally? If not, is submitting through a proxy server for a rally allowed? I'm assuming his k/n pairs would be time stamped when sent en-mass to one of our servers. Is that correct?
Yes, that should be fine; at the rate Beyond crunches, even with his proxy set to cache 500-1000 completed k/n pairs before sending them in, they still won't be sitting around for long.

Quote:
3. If we need 2 servers, should Beyond do IronBits' server 5000 on k=300-400 while the rest of us do Carlos' server 300 on k=400-1001? With Ironbits' server having more memory, I think that works better than the reverse scenario.
That's an interesting idea. It would allow us to clean up Carlos's server much more quickly, though with the obvious disadvantage of drawing resources away from the crucial 300<k<400 range. However, since it would be on Carlos's port 300 server, we still would be finding lots of top-5000 primes.

Quote:
All you computer techie folks, please weigh in on your thoughts here.


Thanks,
Gary
mdettweiler is offline  
Old 2008-03-06, 11:07   #20
Beyond
 
Dec 2002

22×3×37 Posts
Default

Do not setup a seperate server. I have plenty of work here to keep my machines occupied over the weekend.

I am happy to report that the proxy has performed well. Average rate of returned results is in the nieghborhood of 32/min. The lower range files will be complete on Friday morning.

For the higher range file, I'm setting up a server (at work) to test the proxy's communication with a server. It should up and running Friday evening.

So basically, I'm going to skip this rally and continue testing the proxy/server and the ranges I already have on hand.
Beyond is offline  
Old 2008-03-06, 16:48   #21
gd_barnes
 
gd_barnes's Avatar
 
May 2007
Kansas; USA

5×2,017 Posts
Default

Quote:
Originally Posted by Beyond View Post
Do not setup a seperate server. I have plenty of work here to keep my machines occupied over the weekend.

I am happy to report that the proxy has performed well. Average rate of returned results is in the nieghborhood of 32/min. The lower range files will be complete on Friday morning.

For the higher range file, I'm setting up a server (at work) to test the proxy's communication with a server. It should up and running Friday evening.

So basically, I'm going to skip this rally and continue testing the proxy/server and the ranges I already have on hand.

Ah, OK. One note there...I wasn't implying that we set up a new separate server...only that we use 2 already-existing servers; Carlos' port 300 and Ironbits' port 5000. Hey, I'm thinking the rest of us combined may actually have a good chance vs. Free-DC this weekend now. Everyone get all of their crunchers ready!

Note to all...technically it doesn't matter if Beyond crunches with the rally or just continues doing manual LLRing using big files sent to him. He's still processing about the same amount of work either way. I just received another request from him to send him another big file before next Monday.

One important thing that I want to bring up here:
I do not want to 'disenfranchise' anyone with less resources here. It should be fun for everyone, regardless of whether people want manual or automated LLRing; 1 file or 20. Before sending Beyond another big file, I'll initiate a discussion about what ranges people would like to be testing before deciding what to send him.

If it can be done, I might suggest that Carlos pull a big chunk of the file out of his server port 100 since he would like to dry that one, and send it to Beyond. (I haven't checked what is left; may not be feasible now.) That would assist a lot in cleaning up 'Gary's mess' so that we can all be searching for top-5000 primes!


Gary
gd_barnes is offline  
Old 2008-03-06, 16:55   #22
em99010pepe
 
em99010pepe's Avatar
 
Sep 2004

B0E16 Posts
Default

As Beyond I'll be out of the rally due to a bunch of work I still have to crunch on all my cores.
em99010pepe is offline  
Closed Thread

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
LLRnet rally Apr. 30th to May 2nd gd_barnes No Prime Left Behind 51 2010-05-12 22:20
LLRnet server rally 400<k<1001 August 8-10 mdettweiler No Prime Left Behind 66 2008-08-11 03:00
LLRnet server rally 400<k<1001 June 20-22 mdettweiler No Prime Left Behind 67 2008-06-23 15:32
LLRnet server rally port 300 May 23rd-25th gd_barnes No Prime Left Behind 172 2008-06-04 19:21
LLRnet server rally port 300 May 3rd-4th gd_barnes No Prime Left Behind 45 2008-05-05 19:56

All times are UTC. The time now is 02:37.

Tue Mar 31 02:37:05 UTC 2020 up 6 days, 10 mins, 0 users, load averages: 1.72, 1.67, 1.67

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.