LLRnet server rally port 300 May 23rd-25th

[kar_bon]

i just arrived at home.
the AES-server get stuck two times this early morning on my two machines (a Quad and a P4) and both came back onself.
about the stats and rally results:
if i got all results then it's no problem to make a stats of all k/n-pairs of all included server.
the only problem is the time zones: i think i will not display further the llrnet-pairs per hour. if so i had to recalculate/replace the server-submission-times and that's too much work to handle. perhaps for the rally it's ok then if i got all timezones for any server.
preparing the stats-pages but i got today (and tomorrow too) not much time to do so.

[kar_bon]

Originally Posted by Anonymous

Hmm...the results in that file are in a different format than LLRnet results usually are. Not a big deal, I can easily convert it to the other format (for compatibility with my scripts) with a little search/replace in a text editor.

yep, replace " user=sm5ymt " with "\nuser=sm5ymt\n" (\n -> newline) and i have to edit my script a little: first line date/time, second is user and third the result.
no problem at all!

[Lennart]

Originally Posted by kar_bon

i just arrived at home.
the AES-server get stuck two times this early morning on my two machines (a Quad and a P4) and both came back onself.
about the stats and rally results:
if i got all results then it's no problem to make a stats of all k/n-pairs of all included server.
the only problem is the time zones: i think i will not display further the llrnet-pairs per hour. if so i had to recalculate/replace the server-submission-times and that's too much work to handle. perhaps for the rally it's ok then if i got all timezones for any server.
preparing the stats-pages but i got today (and tomorrow too) not much time to do so.

The time in my results are UTC
/Lennart

[gd_barnes]

Originally Posted by kar_bon

i just arrived at home.
the AES-server get stuck two times this early morning on my two machines (a Quad and a P4) and both came back onself.
about the stats and rally results:
if i got all results then it's no problem to make a stats of all k/n-pairs of all included server.
the only problem is the time zones: i think i will not display further the llrnet-pairs per hour. if so i had to recalculate/replace the server-submission-times and that's too much work to handle. perhaps for the rally it's ok then if i got all timezones for any server.
preparing the stats-pages but i got today (and tomorrow too) not much time to do so.

I can understand what you would face trying to do rally stats by hour. That would be too much work. Don't concern yourself with it. Just total stats by person for the rally would be fine.

I'm running great with all 22 cores on port 400 now.

Is anyone having any problems with port 300 now?

Hopefully my pulling mine off of port 300 and Lennart running his personal server will ease the load enough that we will have no more issues there.

The good news is that port 5000 (drive 3) got some significant work done this morning!

Here is my thinking for future rally's and non-rally situations: Let's have any heavy-hitters, i.e. more than 25 cores, running port 400. All others run port 300. That is, let's have 2 servers on drive 1 at all times. I know it'll be more administrative work for us but I don't want this to happen again. Sometime we may lose searchers permanently if our servers don't work, even for a few hours.

Bruce, when you get back, in your situation, I'd just suggest rebooting your machines. Don't fiddle with changing servers, which I know first-hand, takes quite a while for many machines. Port 300 should be fine without Lennart and me on it (and possibly others), which knocks off over 80 cores.

To all who may be a little frustrated at the moment: I know it may seem easier to run manual reservation files but in the long run, LLRnet is much more convienient and faster if you have connected machines with a stable connection. We just have to get to the capacity that we need. Having 2 permanent servers, Ironbits of which should be very heavy duty, should resolve that problem now.


Gary

[mdettweiler]

Originally Posted by gd_barnes

To all who may be a little frustrated at the moment: I know it may seem easier to run manual reservation files but in the long run, LLRnet is much more convienient and faster if you have connected machines with a stable connection. We just have to get to the capacity that we need. Having 2 permanent servers, Ironbits of which should be very heavy duty, should resolve that problem now.

One small correction: manual LLR is actually a little faster than LLRnet. That is because manual LLR uses LLR 3.7.1, which is about 6-7% faster than LLR 3.5.0, which is what LLRnet is based on. The difference may not seem like much, but overall, it sure piles up.

So, as far as I'm concerned, if anyone wants to do manual LLR instead of LLRnet (at least during non-rally times), be my guest.

[mdettweiler]

@Lennart: do you mind if others crunch on your server? I've got one core on it right now but can move it to IronBits' port 4 if you mind.

If you're OK with this, would you like us to change your reservation to show it as reserved by "LLRnet (L6)" (i.e. using our standard notation for reservations by public LLRnet server)?

[Brucifer]

300 server rejected results for:
k=927 n=444956
k=983 n=444962
???

Just running the queues down now from the stuff that was loaded up last night. Still getting some hanging at times in trying to send in results on a few systems. Haven't refilled any queues since the second lockup cycle and do not intend to since I don't have the time to babysit.

[mdettweiler]

Originally Posted by Brucifer

300 server rejected results for:
k=927 n=444956
k=983 n=444962
???

Just running the queues down now from the stuff that was loaded up last night. Still getting some hanging at times in trying to send in results on a few systems. Haven't refilled any queues since the second lockup cycle and do not intend to since I don't have the time to babysit.

Now that we've taken the load off port 300, it shouldn't lock up again. However, if you're still a little wary of port 300, then I'd recommend that you move to port 400 (server llrnet.ironbits.net)--that should require no babysitting whatsoever.

As for the rejected results: that happens sometimes if the k/n pair expires, gets reassigned, then the original assignee returns it before the reassignee returns it, in which case the reassignee's result will be refused. Nothing to worry about unless you're getting tons and tons of rejected results.

[Lennart]

Originally Posted by Anonymous

@Lennart: do you mind if others crunch on your server? I've got one core on it right now but can move it to IronBits' port 4 if you mind.

If you're OK with this, would you like us to change your reservation to show it as reserved by "LLRnet (L6)" (i.e. using our standard notation for reservations by public LLRnet server)?

No i have nothing against it just hook up.

/Lennart

[mdettweiler]

Originally Posted by Lennart

No i have nothing against it just hook up.

/Lennart

Okay, thanks!

[gd_barnes]

Originally Posted by Anonymous

One small correction: manual LLR is actually a little faster than LLRnet. That is because manual LLR uses LLR 3.7.1, which is about 6-7% faster than LLR 3.5.0, which is what LLRnet is based on. The difference may not seem like much, but overall, it sure piles up.

So, as far as I'm concerned, if anyone wants to do manual LLR instead of LLRnet (at least during non-rally times), be my guest.

You missed my point Anon. My point is that since there should be little manual intervention required for LLRnet, THAT is why it's faster.

The down times in between finishing and starting new manual files always lost more CPU time for me than the slower LLR times for LLRnet.

What piles up is idle CPU time. If we have the servers set up to handle the load, there should be zero idle CPU time for all of the people's cores who are connected.

I remember before when I went on a trip, I would have to make sure I loaded enough manual work on ALL of ~30 cores. It was a lot of work, several hours worth. Now I just leave on the trip and don't worry about it.


Gary