PRPNet 3.0.0 Released

[rogue]

This is an beta There are significant changes, including numerous bug fixes. I highly recommend that users upgrade their personal servers. I would not recommend upgrading a public server until backward compability has been tested. Here are a list of changes from 3.0.0:

• all: Fixed issue with large buffered messages to avoid memory leak.
• admin: Send 1000 entries at a time to the server and wait for a response before continuing. This has been done to prevent the admin tool from flooding the server's socket, thus preventing new clients from connecting.
• server: Send a single INFO or ERROR message for each workunit instead of an acknowledgement and then immediately commit/rollback the transaction in the database. This will significantly reduce the possibility of one happening without the other.
• server: The "End of WorkUnit" message must now specify the CandidateName and TestID for the workunit that it references.
• server: Add SecondsForTest to CandidateTest table.
• server: Send positive confirmation to prpadmin after every 1000 entries has been processed so that the admin tool can continue. The server can now hand out candidates by n. Check out sortoption= in the prpserver.ini file.
• server: Send greeting as part of GETWORK message. GETGREETING is no longer supported.
• server: Changed CandidateGroupStats table column names and meanings, thus affecting server_stats.html. If upgrading from 3.0.0, this table must be dropped and recreated and repopulated. It can be repopulated through the admin tool.
• server: Changed servertype FixedKBNC to FixedBKC.
• server: Changed servertype OtherKBNC to FixedBNC.
• server: Close SQL cursor so successive updates are executed. This caused multiple issues where database updates would not be synchronized.
• server: In theory this server is backward compatible with previous versions of the clients, but has not been tested.
• client: Add CandidateName and TestID to the "End of WorkUnit" message to that the server matches the begin and end of the message for that workunit. Without it, the server might record the test results against the wrong candidate.
• client: Instead of sending all workunits then waiting for a response, send them one at at time and wait for the server to send an INFO or ERROR message before sending the next one.
• client: The client now sends the number of seconds for the PRP test back to the server.
• client: In theory this client is backward compatible with previous versions of the server, but has not been tested.
• client: New logic to handle greeting from the server.
• client: When using PFGW, do not create ABC file, just write candidate to input as it will reduce the amount of output generated by PFGW.

Note the table changes (CandidateTest and CandidateGroupStats). These must be done before new clients can connect. The admin tool and browsers can connect, but not clients. CandidateTest has a new column that can be added via the command:

alter table CandidateTest add SecondsForTest double;

You just need to drop and create CandidateGroupStats, then use option 6 from prpadmin to resync the table.

I strongly recommend running the following SQL statements (if upgrading) to correct mistakes in the database due to bugs that have been fixed. If you do not run this SQL, then the server will not send tests for candidates that 3.0.x created a problem with.

update Candidate set HasPendingTest = 0;
update Candidate set HasPendingTest = 1 where exists (select 'x' from CandidateTest where
CandidateTest.CandidateName = Candidate.CandidateName and residue is null);

As far as I know, this addresses all known bugs that have been posted in this thread.

[rogue]

Based upon some preliminary tests, 2.4 clients work with the 3.1 server.

[PCZ]

Mark

I installed the new version clean no upgrade.
All appeared to be going well until i tried to remove some candidates using a factors file.

prpadmin option 5
Sent 1000, server has processed 0

Never progressed past this point.
Prpadmin was using a whole core to itself, left it 5 minutes and it was still hitting the CPU hard but showing no progress.

I stopped prpadmin and that upset prpserver.

[2010-01-07 07:37:58 GMT] PRPNet_Admin_user@ at 127.0.0.1 is connecting as admin
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
[2010-01-07 07:42:59 GMT] Error sending [processed] to socket 388
Lots more of the same

Log got very large very fast so i had to stop and restart prpserver.

Tried a couple of times and the same thing happened.

[Mini-Geek]

I'm getting the same thing as PCZ. More specifically: When I remove a candidate through prpadmin, it repeats "Sent 1, all have been processed" very quickly, prpserver adds "[2010-01-07 14:08:19 GMT] PRPNet_Admin_user@ at 127.0.0.1 is connecting as admin", then when I close prpadmin with Ctrl+C, prpserver starts repeating "Error sending [processed] to socket 260" very quickly, and must be closed by ending the process (Ctrl+C doesn't work).
The candidate is removed from the database correctly. This happens no matter if the candidate was in the database when you issued the command, and no matter if the candidate was the last in the database.

[rogue]

Originally Posted by PCZ

Mark

I installed the new version clean no upgrade.
All appeared to be going well until i tried to remove some candidates using a factors file.

prpadmin option 5
Sent 1000, server has processed 0

Never progressed past this point.
Prpadmin was using a whole core to itself, left it 5 minutes and it was still hitting the CPU hard but showing no progress.

I stopped prpadmin and that upset prpserver.

Lots more of the same

Log got very large very fast so i had to stop and restart prpserver.

Tried a couple of times and the same thing happened.

I'll investigate. It's probably something really stupid on my part.

[rogue]

The only change from 3.1.0 is to address handling factors from the admin tool. You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.1.1.zip

[rogue]

I tracked down an issue with duplicate keys. I had made an assumption that MySQL supported row level locks by default. It doesn't. I had to change the tables to use the InnoDB engine. Once that I did that the throughput increased by a factor of at least 2. I also addressed a related issue where the server could send the same test to two clients concurrently. I really pounded the server code hard for this release. I had 6 clients connecting at a time, each getting 200 tests and those tests taking about .1 to .2 seconds each. The server had tons of throughput problems until I switched over to InnoDB and those problems went away.

To change the engine (if upgrading) you need to do the following in MySQL:

alter table <table_name> engine=innodb;

for each table in the PRPNet database schema. New tables will be created with that engine by default.

You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.1.2.zip.

Unless any new issues show up, I feel confident that this release could be used to replace LLRNet.

[PCZ]

I have also been having lots of troubles with duplicates, failure to get lock etc
If you work it too hard you get DB has gone away or sql error 1040.

Will give the latest version a test drive.

[rogue]

Originally Posted by PCZ

I have also been having lots of troubles with duplicates, failure to get lock etc
If you work it too hard you get DB has gone away or sql error 1040.

Will give the latest version a test drive.

Let me know if that solves your problem.

BTW, in SQLStatement.h you might need to change ii_AffectedRows to SQLINTEGER instead of int32_t. Windows won't complain, but Unix does.

[rogue]

In this release I have fixed a couple of issue with using LLR clients. One prevented it from getting the time of the test, the other causes the client to fail on 64-bit Linux. I also reduced the buffer size from 1000 records to 100 in prpadmin. Linux is really annoying with the way it handles large data buffers and sockets.

You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.1.3.zip.

If you have a problem with compiling SQLStatement.cpp, then you still need to change

SQLINTEGER ii_RowsAffected;

to this

long int ii_RowsAffected;

in SQLStatement.h. I don't know yet how to #define around this. The Linux header file sql.h is wrong if you have to change this source.

[rogue]

In this release I addressed an issue with using select() on a socket. Apparently on some OS's it pegs the CPU. I added GFN support when phrot is used. It requires version 0.72 of phrot. I also added the -e switch when doing primality tests so that PFGW can get to 33% factorization to prove primality.

You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.1.4.zip.