PRPNet 3.0.0 Released

[rogue]

Originally Posted by Mini-Geek

I just noticed that the same problem exists with the server's sorting of k's to hand out. (e.g. when set to sort by k, with k's including 77 and 1003, it starts with 1003)

That has been fixed on the leading edge.

[rogue]

I have put an alpha for 3.2.2 here:

http://home.roadrunner.com/~mrodenki...pnet_3.2.2.zip

I have not gone through any of the heavy duty testing for NPLB, but there are enough other improvements that people have been waiting for. Here is a summary of the changes:

All executables:

• Added client socket ID to all log messages so that it is easier to track connection traffic.

Client only:

• Fixed bug where base 2 tests were using PFGW instead of LLR/phrot.

Server only:

• Use server_stats.html as the default webpage.
• Fix prime/PRP counts on server stats webpage.
• Fixed SELECT statement for FixedKBC servers when updating group stats.
• Sync group stats about three times per hour rather than whenever a completed test is sent, unless a prime/PRP is returned.
• Modified main work sender loop to retrieve fewer records, but to be re-iterated multiple times until enough work is sent or until there are no more records returned by the cursor.
• Added a new index to CandidateTest to be added through MySQL with the following command:
  
  alter table CandidateTest add index ix_residue (CandidateName, Residue);

The PRPNet server on Ubuntu problems have not been resolved yet, despite many attempts (with Lennart's help). I hope to have some time to work on it this coming weekend. Those of you who have been working with me on it understand my frustration over this problem.

[Mini-Geek]

I'm having a problem with PRPnet. The server is returning "INFO: No available candidates are left on this server." when it shouldn't. It recently started doing this persistently (I had not seen the problem, at least not on my local server and clients, at all before). It was shortly after I imported a large number of candidates (62763) and had successfully (and uneventfully, as far as I could tell) completed 5640 of those. I had also recently backed up and deleted some older work from the database, but I saw no ill effects from that. I didn't change anything, and wasn't even at my computer, when it stopped working.
I tried reverting the server .exe to 3.2.1 (from 3.2.2) with no change.
The logs don't seem to have anything particularly helpful, (unless you can see a telling lack of something..?) even set to the most detailed setting.
Server:

[2010-03-23 01:29:51 GMT] PRPNet Server application v3.2.2 started.
[2010-03-23 01:29:51 GMT] Please contact Mark Rodenkirch at rogue@wi.rr.com for support
[2010-03-23 01:29:51 GMT] Waiting for connections on port 7101
[2010-03-23 01:29:55 GMT] 1756: ODBC Connection String: <Driver={MySQL ODBC 5.1 Driver};Server=localhost;Port=3306;Database=prpnet;User=root;Password=[removed];Option=3;>
[2010-03-23 01:29:55 GMT] 1756: successfully connected to the database
[2010-03-23 01:29:55 GMT] 1756: Executing statement: <select CandidateName, CompletedTests,        DecimalLength, LastUpdateTime,        k, b, c, n   from Candidate  where HasPendingTest = 0    and IsPRP = 0    and IsPrime = 0    and DoubleChecked = 0    and HasSierpinskiRieselPrime = 0     and CompletedTests = 0    and CandidateName > ' '    and DecimalLength > 0 order by DecimalLength limit 100; >
[2010-03-23 01:29:57 GMT] 1756: disconnected from database
[2010-03-23 01:29:57 GMT] 1768: ODBC Connection String: <Driver={MySQL ODBC 5.1 Driver};Server=localhost;Port=3306;Database=prpnet;User=root;Password=[removed];Option=3;>
[2010-03-23 01:29:57 GMT] 1768: successfully connected to the database
[2010-03-23 01:29:58 GMT] 1768: Executing statement: <select CandidateName, CompletedTests,        DecimalLength, LastUpdateTime,        k, b, c, n   from Candidate  where HasPendingTest = 0    and IsPRP = 0    and IsPrime = 0    and DoubleChecked = 0    and HasSierpinskiRieselPrime = 0     and CompletedTests = 0    and CandidateName > ' '    and DecimalLength > 0 order by DecimalLength limit 100; >
[2010-03-23 01:30:00 GMT] 1768: disconnected from database

And client:

[2010-03-23 01:29:57 GMT] 396: sending [FROM tim.sorbera@gmail.com i5-1 Mini-Geek 3.2.2]
[2010-03-23 01:29:58 GMT] 396: received [Connected to server]
[2010-03-23 01:29:58 GMT] local: Getting work from server localhost at port 7101
[2010-03-23 01:29:58 GMT] 396: sending [GETWORK 3.2.2 10]
[2010-03-23 01:29:58 GMT] 396: sending [llr]
[2010-03-23 01:29:58 GMT] 396: sending [phrot]
[2010-03-23 01:29:58 GMT] 396: sending [pfgw]
[2010-03-23 01:29:58 GMT] 396: sending [genefer]
[2010-03-23 01:29:58 GMT] 396: sending [End of Message]
[2010-03-23 01:29:58 GMT] 396: received [ServerInfo: 1 3.2.2]
[2010-03-23 01:30:00 GMT] 396: select indicates socket has data, but recv returned nothing
[2010-03-23 01:30:00 GMT] 396: received [INFO: No available candidates are left on this server.]
[2010-03-23 01:30:00 GMT] local: INFO: No available candidates are left on this server.
[2010-03-23 01:30:00 GMT] 396: received [End of Message]
[2010-03-23 01:30:00 GMT] 396: received [Start Greeting]
[2010-03-23 01:30:00 GMT] 396: received [Hi, me!]
[2010-03-23 01:30:00 GMT] 396: received [End Greeting]
[2010-03-23 01:30:00 GMT] 396: closing socket
[2010-03-23 01:30:00 GMT] Could not connect to any servers and no work is pending.  Pausing 1 minute

I'm at a loss. Any suggestions on how to figure out what's going on?
My gut feeling is that it's gotta be related to the number of candidates I added (I've never done that many before), but that doesn't explain why it blew up like this or how I can get it to work.

Edit: And also I noticed that this bug is still present in 3.2.2:

Originally Posted by Mini-Geek

I just noticed that the same problem exists with the server's sorting of k's to hand out. (e.g. when set to sort by k, with k's including 77 and 1003, it starts with 1003)

Was 3.2.2 not included in the "leading edge" where you fixed this?

[rogue]

Originally Posted by Mini-Geek

I'm having a problem with PRPnet. The server is returning "INFO: No available candidates are left on this server." when it shouldn't. It recently started doing this persistently (I had not seen the problem, at least not on my local server and clients, at all before). It was shortly after I imported a large number of candidates (62763) and had successfully (and uneventfully, as far as I could tell) completed 5640 of those. I had also recently backed up and deleted some older work from the database, but I saw no ill effects from that. I didn't change anything, and wasn't even at my computer, when it stopped working.
I tried reverting the server .exe to 3.2.1 (from 3.2.2) with no change.
The logs don't seem to have anything particularly helpful, (unless you can see a telling lack of something..?) even set to the most detailed setting.
I'm at a loss. Any suggestions on how to figure out what's going on?
My gut feeling is that it's gotta be related to the number of candidates I added (I've never done that many before), but that doesn't explain why it blew up like this or how I can get it to work.

Edit: And also I noticed that this bug is still present in 3.2.2:Was 3.2.2 not included in the "leading edge" where you fixed this?

I have a couple of things on this. First, it seems that you have set sortoption in the prpserver.ini to 'L', not 'K'. Even so, I did miss one change on the ordering that I will fix in 3.2.3. Second, this select:

select CandidateName, CompletedTests, DecimalLength,
LastUpdateTime, k, b, c, n
from Candidate
where HasPendingTest = 0 and IsPRP = 0 and IsPrime = 0
and DoubleChecked = 0 and HasSierpinskiRieselPrime = 0
and CompletedTests = 0 and CandidateName > ' '
and DecimalLength > 0
order by DecimalLength limit 100;

is not returning any rows. Have you investigated why?

[Mini-Geek]

Originally Posted by rogue

I have a couple of things on this. First, it seems that you have set sortoption in the prpserver.ini to 'L', not 'K'. Even so, I did miss one change on the ordering that I will fix in 3.2.3.

Currently, yes. But earlier, I had it sorting by K, and at this time it had begun handing out only k=1013, in accordance with the broken sorting.

Originally Posted by rogue

Second, this select:

select CandidateName, CompletedTests, DecimalLength,
LastUpdateTime, k, b, c, n
from Candidate
where HasPendingTest = 0 and IsPRP = 0 and IsPrime = 0
and DoubleChecked = 0 and HasSierpinskiRieselPrime = 0
and CompletedTests = 0 and CandidateName > ' '
and DecimalLength > 0
order by DecimalLength limit 100;

is not returning any rows. Have you investigated why?

I didn't know this was so. (looking back, I suppose that's what the server logs implied) Now I have investigated it, and found that the candidates that aren't being handed out have their DecimalLength in the DB at 0.
I suppose the decimal lengths from that point on weren't generated correctly for some reason at the time they were added to the DB. Is there a way to have the server calculate and store them now, or should I try to re-add them, or what?
Any idea why the decimal lengths weren't set? In case it's of interest, I used PRPadmin to add them from an ABC format file while the server was active (receiving and sending work). Was something there a problem that I should avoid to keep it from happening, or is this a bug that can be fixed, or was it just some random glitch that I should ignore?

[rogue]

Originally Posted by Mini-Geek

I didn't know this was so. (looking back, I suppose that's what the server logs implied) Now I have investigated it, and found that the candidates that aren't being handed out have their DecimalLength in the DB at 0.
I suppose the decimal lengths from that point on weren't generated correctly for some reason at the time they were added to the DB. Is there a way to have the server calculate and store them now, or should I try to re-add them, or what?
Any idea why the decimal lengths weren't set? In case it's of interest, I used PRPadmin to add them from an ABC format file while the server was active (receiving and sending work). Was something there a problem that I should avoid to keep it from happening, or is this a bug that can be fixed, or was it just some random glitch that I should ignore?

Did anything happen to the server while it was loading candidates? After loading from prpadmin, it will go through and set decimal length on all candidates without one. I have never seen any problems with this process and I have loaded 500,000 candidates at one time with 3.2.0 and the code hasn't changed. What you can do is have prpadmin add one candidate (from an ABC file), then it will fix the lengths on all of the others. If it doesn't, then I would like you to set debuglevel=4 and e-mail the log.

[mdettweiler]

Hey Mark, I noticed that version 3.2.2's completed_tests.log format differs slightly from 3.2.1:

[2010-03-20 02:04:55 GMT] Socket 5324: 64*177^50027-1 received by Email: x  User: Mini-Geek  Client: i5-2 Program: pfgw.exe  Residue: 19A1A5BB14E84E12  
[2010-03-22 19:17:04 GMT] 1876: 64*177^99971-1 received by Email: x  User: Mini-Geek  Client: i5-3 Program: pfgw.exe  Residue: BD8CFF4D66EF36BC

Essentially, instead of saying "Socket x:" it's just saying "x:". Needless to say, it's kind of throwing off my processing scripts. I'd just finished tweaking my PRPnet-to-LLRnet conversion script for the "Socket x:" code in 3.2.1 when now it's changed further--and unfortunately this one's a bit harder to detect, so it's not so easy to just add another conditional to handle this version's output.

Could you possibly change it back to say "Socket x:" in 3.2.3? Thanks.

[Mini-Geek]

Originally Posted by rogue

Did anything happen to the server while it was loading candidates?

I don't recall anything like that...unless the crash I had (which happened for no apparent reason, as I recall) interrupted setting the lengths.

Originally Posted by rogue

What you can do is have prpadmin add one candidate (from an ABC file), then it will fix the lengths on all of the others. If it doesn't, then I would like you to set debuglevel=4 and e-mail the log.

I started this, and it started setting all the lengths. Then I had some clients connect, and it was still working. Then I opened the MySQL Command Line Client and connected to the database, and it was still working. Then while I was typing a command in, prpserver.exe crashed...
Nothing was written to the log as it crashed. Here was the last line it wrote:

[2010-03-23 12:06:06 GMT] 1756: Executing statement: <update Candidate    set DecimalLength = 35756  where CandidateName = '5305*168^16066+1'; >

I started the server again, added one candidate through PRPadmin again, and it added the decimal lengths for the rest of the candidates without another problem.

[rogue]

Originally Posted by mdettweiler

Could you possibly change it back to say "Socket x:" in 3.2.3? Thanks.

Done. I could remove the socket component of the message entirely. In 3.2.3 I will separate what is echoed to the terminal (and logged to prpserver.log) from what is put into completed_tests.log. Up through 3.2.2 these were the same message. I modified 3.2.2 to consistently output the socket in the same position so that I could easily trace client connections.

Originally Posted by Mini-Geek

I don't recall anything like that...unless the crash I had (which happened for no apparent reason, as I recall) interrupted setting the lengths.

I started this, and it started setting all the lengths. Then I had some clients connect, and it was still working. Then I opened the MySQL Command Line Client and connected to the database, and it was still working. Then while I was typing a command in, prpserver.exe crashed...
Nothing was written to the log as it crashed.

Can you send me your ABC and prpserver.ini files? Can you also tell me what OS this is on? I will try to recreate the problem.

[Mini-Geek]

Originally Posted by rogue

Can you send me your ABC and prpserver.ini files? Can you also tell me what OS this is on? I will try to recreate the problem.

Windows XP SP3 (32-bit).
The requested files are attached.

(I don't know if this will be of any help, but...) It originally failed at n=11443.

The last completed numbers (prior to the problem making itself apparent):
[2010-03-23 00:28:40 GMT] 8688: 740*168^11443+1 sent to Email: tim.sorbera@gmail.com  User: Mini-Geek  Client: i5-1
[2010-03-23 00:28:40 GMT] 8688: 901*168^11443+1 sent to Email: tim.sorbera@gmail.com  User: Mini-Geek  Client: i5-1
[2010-03-23 00:28:40 GMT] 8688: 6824*168^11443+1 sent to Email: tim.sorbera@gmail.com  User: Mini-Geek  Client: i5-1
...
And the first few DecimalLength updates:
[2010-03-23 12:01:37 GMT] 1756: Executing statement: <update Candidate    set DecimalLength = 25467  where CandidateName = '1398*168^11443+1'; >
[2010-03-23 12:01:37 GMT] 1756: Executing statement: <update Candidate    set DecimalLength = 25467  where CandidateName = '256*168^11443+1'; >
[2010-03-23 12:01:37 GMT] 1756: Executing statement: <update Candidate    set DecimalLength = 25468  where CandidateName = '2963*168^11443+1'; >
[2010-03-23 12:01:37 GMT] 1756: Executing statement: <update Candidate    set DecimalLength = 25468  where CandidateName = '3385*168^11443+1'; >

[mdettweiler]

Originally Posted by rogue

Done. I could remove the socket component of the message entirely. In 3.2.3 I will separate what is echoed to the terminal (and logged to prpserver.log) from what is put into completed_tests.log. Up through 3.2.2 these were the same message. I modified 3.2.2 to consistently output the socket in the same position so that I could easily trace client connections.

Thanks. BTW, if you remove the socket component of the message in 3.2.3, could you please make sure it goes exactly back to the 3.2.0-and-earlier format? That way I don't have to change my script again:

[2010-03-23 03:30:58 GMT] 2036*9^694880+1 received by Email: max@noprimeleftbehind.net  User: mdettweiler  Client: Quad Program: pfgw.exe  Residue: 6ED61A2F0D4852FF