PRPnet servers for NPLB - Page 15

rogue · 2010-12-25, 00:46

I want to make it clear that the memory wasn't leaked (because valgrind couldn't detect it). The problem was with how the server manages threads. Because the server was not detaching a thread after creating it, it would eat more memory. I suspect that under the covers that pthread does a fork(), which replicates the memory footprint of the main thread when starting the new thread. The difference is that when pthread detaches after creating and starting the new thread, that the extra memory is freed.

gd_barnes · 2010-12-25, 04:38

Nice work all in isolating the cause of the mysterious issues both with PRPnet and the sluggishness of the main server machine!

rogue · 2010-12-25, 13:08

Quote:

Originally Posted by gd_barnes

Nice work all in isolating the cause of the mysterious issues both with PRPnet and the sluggishness of the main server machine!

I need to re-release it today. I bunged a change that was experimental. It causes PRPNet to not release server connections.

gd_barnes · 2010-12-27, 06:51

Quote:

Originally Posted by mdettweiler

@Gary: this memory leak, I discovered, is actually the cause of the mysterious sluggishness on jeepford. As much as I would like to blame the absolutely horrible Ubuntu 9.04 as per our original theory, it seems that the many GB of leaked memory from the multiple running prpservers caused the system to be tied up while it shuffled data from active memory into virtual memory (to make room for normal GUI activities) every time someone used the computer after it had sat for a while. With the memory leaks now under control, most of the sluggishness should be gone from here on out. (Not to mention that it will spare the hard drive quite a bit of beating...)

I'm not sure where you came up with this. I think you were just wishful thinking. Jeepford is no quicker than it was before. When getting on it for the first time in several hours or a day or more, it hesitates just as badly as it did before.

Flatlander · 2011-01-31, 16:59

Quote:

port: 9000
k-range: 600-800 (6th Drive)
n-range: 780K-785K, 795K-800K, 830K-833K, 860K-863K, 870K-873K, 880K-890K, 930K-940K, 960K-970K

Does that still apply or is it purely 960k-970k now?

gd_barnes · 2011-01-31, 17:15

Quote:

Originally Posted by Flatlander

Does that still apply or is it purely 960k-970k now?

I leave it that way until we process the results for the other ranges. 960K-970K is all that is actually remaining to be tested from the server right now. (actually ~967K-970K)

mdettweiler · 2011-02-05, 05:02

Hi all,

Just a little heads-up that I will need to dry out port 9000 after its current range completes (n=960K-970K, the last of the 6th Drive--should be finished within a few days). I will then be upgrading it to PRPnet version 4.1.4, which adds a number of handy new features; most of them don't particularly apply to NPLB, but there are some minor tweaks that still make the upgrade worthwhile. The biggest addition is support for teams in the server's built-in stats; as with the server's built-in user stats, these will remain entirely informal since we already have a fancier stats system at http://www.noprimeleftbehind.net/ which keeps track of all the servers together. Nonetheless, the server's built-in stats can be useful to compare contributions on that particular server at a glance, and now they'll keep track of teams too.

The server will hopefully not be down more than a half-day or so for the upgrade. If you don't already have a backup server or two configured in prpclient.ini, I'd recommend adding some--that way your client will still be able to do some sort of work even while port 9000 is down. One such server is at noprimeleftbehind.net port 1300, which is operated under the auspices of our sister project, Conjectures 'R Us. (Note however that due to entirely coincidental timing, port 1300 may end up drying for an upgrade of its own right around the same time as port 9000; I'll do what I can to ensure that the downtimes don't overlap but can't make any promises. I would recommend adding at least one other server in case they're both down; any of the various PRPnet servers run by other projects will do, such as noprimeleftbehind.net port 12000 from the TPS project, which I've already upgraded to 4.1.4, and the myriad servers run by PrimeGrid, which may already be in your prpclient.ini if you downloaded your client from them.)

Thanks for bearing with this (hopefully) brief downtime as we upgrade the server!

Max

gd_barnes · 2011-02-07, 18:53

The server has dried out and will be upgraded before being loaded with 7th drive work next. Max will provide more details later on.

mdettweiler · 2011-02-09, 04:50

Just as an update...I still haven't yet gotten to the server upgrade, but I should be able to do it sometime tomorrow. (I was out of the house for much of today, and much of what time I did have at home was wrapped up in preparing for the outing.) My apologies for the somewhat extended downtime.

In the meantime, I did manage to get CRUS's port 1300 upgraded yesterday, and it's loaded up with plenty of work to keep it busy for a long while, so if you're looking for something to run your clients on while port 9000 is out, you might want to give it a try!

mdettweiler · 2011-02-11, 04:25

Hi all,

The port 9000 upgrade went smoothly and we are now on PRPnet version 4.1.4!

(4.2.0 was released not long ago and we'll upgrade to it once it's been around for a little while and we've tested it on non-production servers.)

I already gave the rundown of what's new in this release back in post #161 of this thread, so I won't bother repeating myself.

If any of you want to take advantage of the new support for teams in the server's built-in stats (which are unofficial in NPLB's context, but are used as the primary stats-tracking system by other projects), you can upgrade to the latest 4.2.0 client. You'll need to get a fresh prpclient.ini from the new version's client package since some of the options have changed (with the addition of team support in 4.1.x, and some changes to the startoption= choices in 4.2.x), but other than that the rest of your existing client setups should work as-is. And, of course, if you'd like you can stay with the 3.3.6 client, though I'd recommend upgrading since there have been a number of little bugfixes along the way that reduce the likelihood of one of your results being lost due to a network glitch or such.

Max

mdettweiler · 2011-02-11, 04:58

A couple more things I forgot to note in the last post:

-When configuring your team name on a >=4.1.x client, if your team name contains spaces, you'll need to remove the spaces when entering it into prpclient.ini. For instance, my team "Raiders of the Lost Primes" is entered as "RaidersOfTheLostPrimes". This way, we can ensure consistency in team names and avoid duplicate entries for the same team (each with a slightly different rendition of the name).

If you do end up making a mistake and submit some results under a wonky team name, though, let me know and I'll try to fix it.

-If you're upgrading to a 4.1.x or 4.2.x client and don't want to participate under a team, just leave "teamid=" blank in prpclient.ini. Your stats will show up under "None" in the server's team stats. (Pre-4.1.x clients aren't counted in the team stats at all.)

-If you want to change teams, just edit your prpclient.ini accordingly. Simple as that--your previous points will stay with the old team since PRPnet stores the team field on a per-result basis. (Note that you'll also need to post in our teams thread so that we can move you to the new team in the real stats at http://www.noprimeleftbehind.net/. For those stats, the team is stored on a per-user basis, so moving to a different team will require me to first split off your earlier contributions to a separate username listed under the old team. This does, unfortunately, have the side effect of resetting your individual contributions to zero from that point on, but it is necessary to avoid massive fluctuations in team stats when somebody changes teams.)

-There is a minor bug in the PRPnet 4.1.4 server that causes the "Client" and "Team" columns to be switched in the user primes tables (at the top of http://noprimeleftbehind.net:9000/all.html). The data is in the right place on the back-end; the bug is merely cosmetic, so it will be fixed once we eventually upgrade to v4.2.x.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
PRPnet Servers for CRUS	MyDogBuster	Conjectures 'R Us	83	2021-01-28 10:08
LLRnet servers for NPLB	kar_bon	No Prime Left Behind	1343	2014-08-20 09:38
Public PRPNet Servers	rogue	Open Projects	26	2013-01-16 01:33
PRPNet servers down?	opyrt	Prime Sierpinski Project	13	2009-11-04 21:33
Servers for NPLB	gd_barnes	No Prime Left Behind	0	2009-08-10 19:21

2010-12-25, 00:46	#155
rogue "Mark" Apr 2003 Between here and the 11·577 Posts	I want to make it clear that the memory wasn't leaked (because valgrind couldn't detect it). The problem was with how the server manages threads. Because the server was not detaching a thread after creating it, it would eat more memory. I suspect that under the covers that pthread does a fork(), which replicates the memory footprint of the main thread when starting the new thread. The difference is that when pthread detaches after creating and starting the new thread, that the extra memory is freed.

2010-12-25, 04:38	#156
gd_barnes May 2007 Kansas; USA 3³·5·7·11 Posts	Nice work all in isolating the cause of the mysterious issues both with PRPnet and the sluggishness of the main server machine!

2011-02-05, 05:02	#161
mdettweiler A Sunny Moo Aug 2007 USA (GMT-5) 3×2,083 Posts	Hi all, Just a little heads-up that I will need to dry out port 9000 after its current range completes (n=960K-970K, the last of the 6th Drive--should be finished within a few days). I will then be upgrading it to PRPnet version 4.1.4, which adds a number of handy new features; most of them don't particularly apply to NPLB, but there are some minor tweaks that still make the upgrade worthwhile. The biggest addition is support for teams in the server's built-in stats; as with the server's built-in user stats, these will remain entirely informal since we already have a fancier stats system at http://www.noprimeleftbehind.net/ which keeps track of all the servers together. Nonetheless, the server's built-in stats can be useful to compare contributions on that particular server at a glance, and now they'll keep track of teams too. The server will hopefully not be down more than a half-day or so for the upgrade. If you don't already have a backup server or two configured in prpclient.ini, I'd recommend adding some--that way your client will still be able to do some sort of work even while port 9000 is down. One such server is at noprimeleftbehind.net port 1300, which is operated under the auspices of our sister project, Conjectures 'R Us. (Note however that due to entirely coincidental timing, port 1300 may end up drying for an upgrade of its own right around the same time as port 9000; I'll do what I can to ensure that the downtimes don't overlap but can't make any promises. I would recommend adding at least one other server in case they're both down; any of the various PRPnet servers run by other projects will do, such as noprimeleftbehind.net port 12000 from the TPS project, which I've already upgraded to 4.1.4, and the myriad servers run by PrimeGrid, which may already be in your prpclient.ini if you downloaded your client from them.) Thanks for bearing with this (hopefully) brief downtime as we upgrade the server! Max

2011-02-07, 18:53	#162
gd_barnes May 2007 Kansas; USA 3³·5·7·11 Posts	The server has dried out and will be upgraded before being loaded with 7th drive work next. Max will provide more details later on.

2011-02-09, 04:50	#163
mdettweiler A Sunny Moo Aug 2007 USA (GMT-5) 3×2,083 Posts	Just as an update...I still haven't yet gotten to the server upgrade, but I should be able to do it sometime tomorrow. (I was out of the house for much of today, and much of what time I did have at home was wrapped up in preparing for the outing.) My apologies for the somewhat extended downtime. In the meantime, I did manage to get CRUS's port 1300 upgraded yesterday, and it's loaded up with plenty of work to keep it busy for a long while, so if you're looking for something to run your clients on while port 9000 is out, you might want to give it a try!

2011-02-11, 04:25	#164
mdettweiler A Sunny Moo Aug 2007 USA (GMT-5) 6249₁₀ Posts	Hi all, The port 9000 upgrade went smoothly and we are now on PRPnet version 4.1.4! (4.2.0 was released not long ago and we'll upgrade to it once it's been around for a little while and we've tested it on non-production servers.) I already gave the rundown of what's new in this release back in post #161 of this thread, so I won't bother repeating myself. If any of you want to take advantage of the new support for teams in the server's built-in stats (which are unofficial in NPLB's context, but are used as the primary stats-tracking system by other projects), you can upgrade to the latest 4.2.0 client. You'll need to get a fresh prpclient.ini from the new version's client package since some of the options have changed (with the addition of team support in 4.1.x, and some changes to the startoption= choices in 4.2.x), but other than that the rest of your existing client setups should work as-is. And, of course, if you'd like you can stay with the 3.3.6 client, though I'd recommend upgrading since there have been a number of little bugfixes along the way that reduce the likelihood of one of your results being lost due to a network glitch or such. Max

2011-02-11, 04:58	#165
mdettweiler A Sunny Moo Aug 2007 USA (GMT-5) 6249₁₀ Posts	A couple more things I forgot to note in the last post: -When configuring your team name on a >=4.1.x client, if your team name contains spaces, you'll need to remove the spaces when entering it into prpclient.ini. For instance, my team "Raiders of the Lost Primes" is entered as "RaidersOfTheLostPrimes". This way, we can ensure consistency in team names and avoid duplicate entries for the same team (each with a slightly different rendition of the name). If you do end up making a mistake and submit some results under a wonky team name, though, let me know and I'll try to fix it. -If you're upgrading to a 4.1.x or 4.2.x client and don't want to participate under a team, just leave "teamid=" blank in prpclient.ini. Your stats will show up under "None" in the server's team stats. (Pre-4.1.x clients aren't counted in the team stats at all.) -If you want to change teams, just edit your prpclient.ini accordingly. Simple as that--your previous points will stay with the old team since PRPnet stores the team field on a per-result basis. (Note that you'll also need to post in our teams thread so that we can move you to the new team in the real stats at http://www.noprimeleftbehind.net/. For those stats, the team is stored on a per-user basis, so moving to a different team will require me to first split off your earlier contributions to a separate username listed under the old team. This does, unfortunately, have the side effect of resetting your individual contributions to zero from that point on, but it is necessary to avoid massive fluctuations in team stats when somebody changes teams.) -There is a minor bug in the PRPnet 4.1.4 server that causes the "Client" and "Team" columns to be switched in the user primes tables (at the top of http://noprimeleftbehind.net:9000/all.html). The data is in the right place on the back-end; the bug is merely cosmetic, so it will be fixed once we eventually upgrade to v4.2.x.