PRPnet rally Feb. 18th-25th

[mdettweiler]

Originally Posted by odicin

Not only at this time, I try to remember around some months ago (before avx) a similar problem with the older 3.8.4 llr and some AMD CPU's at PG. In fact the problem was the used gnum version so Primegrid updated all there stock apps to 3.8.6 llr which uses a newer gnum version.

Rebirthers AVX build of llr uses the early beta of gnum 27.3, the latest llr from jean penne still use 26.5. I suppose this is the problem there.

You can track down the problem with rebirthers non-avx x64 llr build. This one also use gnum 27.3 instead of 26.5.

Regards Odi

Hmm...so the non-AVX x64 builds are based on gwnum 27.3 as well? How does that work? Is there a compile flag or something that can enable/disable AVX for a given (v27.x) gwnum version?

[Lennart]

I have big problem to reach the server ! Only 50-60% is conecting but there are no problem to reach other server on PG.

Are there some problem or some manage ongoing ?


Lennart

[odicin]

Correct. You can swap to another gnum lib, because it's not a part of llr itself. But I don't know how exactly it works, better you ask this rebirther or rouge.

Regards Odi

[rogue]

Originally Posted by odicin

Correct. You can swap to another gnum lib, because it's not a part of llr itself. But I don't know how exactly it works, better you ask this rebirther or rouge.

Regards Odi

gwnum is linked into the executable.

[mdettweiler]

Originally Posted by Lennart

I have big problem to reach the server ! Only 50-60% is conecting but there are no problem to reach other server on PG.

Are there some problem or some manage ongoing ?


Lennart

Yeah, I've been monitoring this. mysqld is (as you might imagine) under fairly heavy load, this being the highest amount of activity we've ever supported on a PRPnet server. Gary had three PRPnet clients running on "spare" cores of the server machine--well, under normal load they would be spare. I shut down one of them to give MySQL more headroom, and will keep an eye on things to shut down more if needed.

Edit: Okay, it looks like it's not quite enough--I'm still seeing fallbacks to other servers on my own clients. I went ahead and shut down all the PRPnet clients running on the server--to be safe we should probably leave it completely free to do its server job during the rally.

[mdettweiler]

Originally Posted by rogue

gwnum is linked into the executable.

I guess what I was wondering, though, was whether it's possible to build both AVX and non-AVX versions of LLR using gwnum v27.3? I had understood that the non-AVX builds were based on v26.x, but odicin mentioned Rebirther's non-AVX x64 build being based on v27 so I wanted to make sure I had that correctly.

[Lennart]

Originally Posted by mdettweiler

Yeah, I've been monitoring this. mysqld is (as you might imagine) under fairly heavy load, this being the highest amount of activity we've ever supported on a PRPnet server. Gary had three PRPnet clients running on "spare" cores of the server machine--well, under normal load they would be spare. I shut down one of them to give MySQL more headroom, and will keep an eye on things to shut down more if needed.

Edit: Okay, it looks like it's not quite enough--I'm still seeing fallbacks to other servers on my own clients. I went ahead and shut down all the PRPnet clients running on the server--to be safe we should probably leave it completely free to do its server job during the rally.

Have you optimized the tables ? Close the server and all deamons and restart mysql. Then start prpnet servers again.

Lennart

[rebirther]

Originally Posted by mdettweiler

I guess what I was wondering, though, was whether it's possible to build both AVX and non-AVX versions of LLR using gwnum v27.3? I had understood that the non-AVX builds were based on v26.x, but odicin mentioned Rebirther's non-AVX x64 build being based on v27 so I wanted to make sure I had that correctly.

The non avx version is built with gwnum 26.6. AVX with latest 27.3.

[mdettweiler]

Originally Posted by Lennart

Have you optimized the tables ? Close the server and all deamons and restart mysql. Then start prpnet servers again.

Lennart

Hmm, okay...I'll try to do that in 20 minutes or so when the next stats refresh minutes. (BTW: what do you mean by optimize the tables? I don't know too much about MySQL so I don't know how to do that; does simply restarting mysqld do it automatically?)

Notice to all: all noprimeleftbehind.net PRPnet servers will be going down briefly at about 12:20 PM server time. The plan is to have them back online within 5-10 minutes at the max, and this should take care of the issues we've been seeing lately.

[mdettweiler]

Okay, I restarted mysqld and all PRPnet servers are back online again. Hopefully that will do the trick.

Edit: looking at the stats from the last few hours, it seems that shutting down all the PRPnet clients running on the server did help, at least some--I notice a marked increase in the number of pairs/hour returned. Between that and restarting mysqld, we should be good to go now.

Edit #2: Ah yes, I'm seeing much less CPU usage from mysqld now--on the order of 15-50% CPU (on the Linux scale where 100%=1 core), versus 150-200% before the mysqld restart.

[gd_barnes]

Is there some sort of "memory leak" so to speak of the CPU usage that causes this to "build up"?