LLRnet server rally March 8th-9th

[mdettweiler]

Originally Posted by gd_barnes

I was thinking for this rally, would it make sense to have Beyond running on one server and the rest of us on another server? If Beyond's machines crash the server he is on, at least he can lesson the load a little bit on that one server while still having substantial tests going on and also having the least impact to all of us.


Gary

Hmm...interesting idea. Maybe for this rally, Beyond can go on one of Carlos's servers (since they need to be cleaned anyway), and the rest of us can do IronBits' port 5000 server as was originally planned? I know Carlos's servers can't handle as much of a load as IronBits' can, but they should be able to handle Beyond just fine as long as everybody else isn't on the same server. (Remember, when we only had Carlos's servers, they were able to handle the load of Beyond as well as a bunch of other people, as long as we weren't doing a rally--so Beyond alone shouldn't be a problem for them.)

[kar_bon]

i think, and IronBits agreed, it's not the numerous amount of Beyonds clients (70 or more cores), it is his proxy:
he reserves a wide range (many k/n-pairs) at once and when they all ready to submit, the server gets too many connections without closing (the resultfiles show that over some minutes there are 8-10 results submitted per second!)
see also the graph: peaks at Feb24, Feb27 and Feb29 at the scoring page with more than 5000 pairs per hour!
so it think this can crash every LLRnet server!

[Beyond]

Originally Posted by kar_bon

i think, and IronBits agreed, it's not the numerous amount of Beyonds clients (70 or more cores), it is his proxy:
he reserves a wide range (many k/n-pairs) at once and when they all ready to submit, the server gets too many connections without closing (the resultfiles show that over some minutes there are 8-10 results submitted per second!)
see also the graph: peaks at Feb24, Feb27 and Feb29 at the scoring page with more than 5000 pairs per hour!
so it think this can crash every LLRnet server!

I'm not convinced it is the proxy.

The outcome is the same if I cache 10 workunits on the proxy and 10 on each client or 1000 on the proxy and 100 on each client. I've tryed dozens of different combinations. Also tried setting uploads from 10 to 50 to 100 at a time.

It is only a matter of time depending on the caching scheme employed, that I start seeing a lot of "no knpairs recieved from server" messages on the proxy. Usually within 10 to 15 minutes of the no knpairs recieved messages, I get "cannot connect with server" messages. This is the point were the server goes down or hits the max connection limits and locks everyone out.

If I'm not here to shutdown and restart the proxy, then the results start to pile up quickly. When I finally get to do a restart after a prolonged period you will see the hugh spike in returned results, usually in the thousands.

So from my end it appears the server cannot handle all the request for more work.

ATM, I'm working on getting a large file (manual reservation) to put on the proxy to test for problems on this end.

[henryzz]

surely beyond doing manual reservation through his proxy would be the best solution

[mdettweiler]

Originally Posted by Beyond

...
ATM, I'm working on getting a large file (manual reservation) to put on the proxy to test for problems on this end.

Please note: if you're using a manual reservation file in your proxy, you'll want to set your llr-serverconfig.txt so that the proxy behaves as a proper server, otherwise it will try to return the results from your manual reservation file to either IronBits' server when they're completed.

[gd_barnes]

I want to get this decided ahead of time to avoid problems. Questions for the upcoming rally:

1. Does it make sense to have Beyond on one server and the rest of us on another?

2. Since Beyond is working on a proxy server, is there still an issue with us needing more than one server for the rally? If not, is submitting through a proxy server for a rally allowed? I'm assuming his k/n pairs would be time stamped when sent en-mass to one of our servers. Is that correct?

3. If we need 2 servers, should Beyond do IronBits' server 5000 on k=300-400 while the rest of us do Carlos' server 300 on k=400-1001? With Ironbits' server having more memory, I think that works better than the reverse scenario.

All you computer techie folks, please weigh in on your thoughts here.


Thanks,
Gary

[IronBits]

What ever you want to do is fine with me.
If you need another port opened up on the server for a different range, that's easy to do, just let me know.

Beyond is working out a server to server communications problem...
If he put 1/3 on Carlos, 1/3 on port 500 and 1/3 on port 5000, using 3 personal proxy servers, he might do ok. ;)

[mdettweiler]

(my reply is interspersed throughout Gary's original message)

Originally Posted by gd_barnes

I want to get this decided ahead of time to avoid problems. Questions for the upcoming rally:

1. Does it make sense to have Beyond on one server and the rest of us on another?

I think so, even if it's only to make sure that we don't all go down if he crashes the server, or vice versa.

2. Since Beyond is working on a proxy server, is there still an issue with us needing more than one server for the rally? If not, is submitting through a proxy server for a rally allowed? I'm assuming his k/n pairs would be time stamped when sent en-mass to one of our servers. Is that correct?

Yes, that should be fine; at the rate Beyond crunches, even with his proxy set to cache 500-1000 completed k/n pairs before sending them in, they still won't be sitting around for long.

3. If we need 2 servers, should Beyond do IronBits' server 5000 on k=300-400 while the rest of us do Carlos' server 300 on k=400-1001? With Ironbits' server having more memory, I think that works better than the reverse scenario.

That's an interesting idea. It would allow us to clean up Carlos's server much more quickly, though with the obvious disadvantage of drawing resources away from the crucial 300<k<400 range. However, since it would be on Carlos's port 300 server, we still would be finding lots of top-5000 primes.

All you computer techie folks, please weigh in on your thoughts here.


Thanks,
Gary

[Beyond]

Do not setup a seperate server. I have plenty of work here to keep my machines occupied over the weekend.

I am happy to report that the proxy has performed well. Average rate of returned results is in the nieghborhood of 32/min. The lower range files will be complete on Friday morning.

For the higher range file, I'm setting up a server (at work) to test the proxy's communication with a server. It should up and running Friday evening.

So basically, I'm going to skip this rally and continue testing the proxy/server and the ranges I already have on hand.

[gd_barnes]

Originally Posted by Beyond

Do not setup a seperate server. I have plenty of work here to keep my machines occupied over the weekend.

I am happy to report that the proxy has performed well. Average rate of returned results is in the nieghborhood of 32/min. The lower range files will be complete on Friday morning.

For the higher range file, I'm setting up a server (at work) to test the proxy's communication with a server. It should up and running Friday evening.

So basically, I'm going to skip this rally and continue testing the proxy/server and the ranges I already have on hand.

Ah, OK. One note there...I wasn't implying that we set up a new separate server...only that we use 2 already-existing servers; Carlos' port 300 and Ironbits' port 5000. Hey, I'm thinking the rest of us combined may actually have a good chance vs. Free-DC this weekend now. Everyone get all of their crunchers ready!

Note to all...technically it doesn't matter if Beyond crunches with the rally or just continues doing manual LLRing using big files sent to him. He's still processing about the same amount of work either way. I just received another request from him to send him another big file before next Monday.

One important thing that I want to bring up here:
I do not want to 'disenfranchise' anyone with less resources here. It should be fun for everyone, regardless of whether people want manual or automated LLRing; 1 file or 20. Before sending Beyond another big file, I'll initiate a discussion about what ranges people would like to be testing before deciding what to send him.

If it can be done, I might suggest that Carlos pull a big chunk of the file out of his server port 100 since he would like to dry that one, and send it to Beyond. (I haven't checked what is left; may not be feasible now.) That would assist a lot in cleaning up 'Gary's mess' so that we can all be searching for top-5000 primes!


Gary

[em99010pepe]

As Beyond I'll be out of the rally due to a bunch of work I still have to crunch on all my cores.