PRPNet 3.0.0 Released

[rogue]

I have not tested any of the changes for this release. I expect them to work, but I don't have time to test right now. The worst case scenario is that it doesn't work and you rollback to 3.1.4.

In this release I have changed logging to eliminate the debug.log file. Items previously written there are now written to prpserver.log and prpclient.log when debuglevel is set.

I have added a test_results.log on the client side. It contains a timestamp, candidate number, helper program, residue, and time (rounded to the nearest number of seconds).

On the server side I added a mutex. This mutex will allow a single thread at a time get work from the server. This should eliminate the duplicate key issue. AFAICT, the code should already have done that, but due to either a misunderstanding on my part WRT MySQL or a potential bug in MySQL, I use the mutex to avoid the root cause of the issue.

You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.1.5.zip.

[gd_barnes]

Originally Posted by rogue

I have added a test_results.log on the client side. It contains a timestamp, candidate number, helper program, residue, and time (rounded to the nearest number of seconds).

EXCELLENT!! Thanks. I look forward to testing the new version.

[rogue]

Originally Posted by gd_barnes

EXCELLENT!! Thanks. I look forward to testing the new version.

Glad to be of service.

[rogue]

As the title states, I am releasing 3.2.0. I was going to call it 3.1.6, but with the number of changes, I decided to bump it up to 3.2.0 instead. This release has changes to address all known issues. I have done extensive testing on this release, so I consider this a beta.

Here is a list of changes for the client:

• Added "echotest" option to prpclient.ini. Setting this to 0 will tell the client not to echo "INFO" messages to the console for accepted tests.
• Fixed a bug when using the client to test GFNs. When a roundoff error would occur, the client would neglect to delete temporary files generated. The next version of genefer would run, but read the same roundoff error instead of the actual test result.
• Modified client to wait up to 60 seconds to get a response from the server.
• Fixed an issue on Linux builds with an uninitialized mutex causing the software to crash.

Here is a list of changes for the server:

• Added "echotest" option to prpserver.ini. Setting this to 0 will tell the server not to echo "Test sent" and "Test received" messages to the console.
• Changed HTML format of Sierpinski/Riesel server_stats.html to create a separate table for each base along with a header and summary row for the base. The summary row will now be shown in red if all k have a prime or PRP.
• Added UserPrimes table, which is populated upon server start up if the table is empty. If the server fails to load the table, then it will not run until the problem is fixed.
• Added user_primes.html to show all primes found, grouped by user.
• Added unhideprimehours= to prpserver.ini. This is used to control how long a prime/PRP is hidden from user_primes.html before it can be viewed by the public.
• Added server version to HTML title on webpages.
• Replaced "Other k/n/b/c" with "Fixed b/n/c" in prpserver.ini. This changes the grouping and sorting of server stats.
• Limit size of result set on server when getting work because loading large numbers or candidates (100000 or more) into memory when opening the cursor can lead to the client timing out.
• Changes locking mechanism when getting work since MySQL does not seem to always respect "for update" on InnoDB database.
• Added socket to logs to aid in trouble-shooting.
• Changed LengthCalculator to select blocks of records at a time so that it is less likely to timeout the admin tool.
• Fixed an issue on Linux builds with an uninitialized mutex causing the software to crash.

Both are completely compatible with 3.1.x versions of the client and server. This has not been tested with LLR 3.8.0. As long as Jean has not altered the format of the output line for base 2 tests, they should work seamlessly together.

You can d/l it from http://home.roadrunner.com/~mrodenki...pnet_3.2.0.zip.

[gd_barnes]

Originally Posted by rogue

As the title states, I am releasing 3.2.0. I was going to call it 3.1.6, but with the number of changes, I decided to bump it up to 3.2.0 instead. This release has changes to address all known issues. I have done extensive testing on this release, so I consider this a beta.

So, by extensively tested, I can assume it's been tested on Linux clients also, correct?

[rogue]

Originally Posted by gd_barnes

So, by extensively tested, I can assume it's been tested on Linux clients also, correct?

No, I have not tested on Linux.

[gd_barnes]

Originally Posted by rogue

No, I have not tested on Linux.

A small amount of testing in Linux revealed several glaring issues including a garbled web page and possible use of an incorrect DB. I think Max is getting with you by Email on the issues.

If possible, please utilize my Linux server machine for testing before doing a public release. Thanks.

Can we please add no more new features until we get the existing features 100% correct across all platforms? Otherwise I feel like we're testing a moving target, which is very difficult to hit.

Would it make sense to get a copy of Ubuntu 9.04 or later and make one of your Windows machines a dual-boot Windows/Linux machine? It doesn't cost a dime and there are plenty of people including Max and Ian that could probably help you get it set up for testing a server. I knew nothing about Linux 2 years ago and still know a limited amount now. But I know enough to know that it is far more effective and efficient than Windows when running CPU-intensive programs and there's little fear of viruses/adware/etc. Windows has too much crap running in the background constantly and will shut itself down if you don't turn off automatic updates. It also "forces" new features and security on you with each new version that you frequently don't need or are hard to understand. Vista was the worst I've ever seen in that regard. Windows 7 is pretty decent though but Windows in general is just not oriented towards programming types. It tries to be too helpful and eats up CPU time doing so.


Gary

[rogue]

Originally Posted by gd_barnes

A small amount of testing in Linux revealed several glaring issues including a garbled web page and possible use of an incorrect DB. I think Max is getting with you by Email on the issues.

If possible, please utilize my Linux server machine for testing before doing a public release. Thanks.

Can we please add no more new features until we get the existing features 100% correct across all platforms? Otherwise I feel like we're testing a moving target, which is very difficult to hit.

I wasn't aware of any garbled webpages. Max mentioned something about a table being used incorrectly, but I think that is actually a check that the server makes when it starts up to determine if it needs to populate one of the new tables. I will remove that code in 3.2.1 and provide an SQL statement to populate it if upgrading from 3.1.x.

There will be no new features for a while. I will be focusing my effort on squashing bugs.

As mentioned before I cannot install anything on my wife's PC without getting into a lot of trouble. I will take advantage of the account Max set up for so that I can test any bug fixes before they I put out any patch releases. As for Windows, it is not possible for me to test on Vista or Windows 7. I only have XP at my disposal.

Gary, I appreciate your patience.

[henryzz]

Originally Posted by rogue

As mentioned before I cannot install anything on my wife's PC without getting into a lot of trouble. I will take advantage of the account Max set up for so that I can test any bug fixes before they I put out any patch releases. As for Windows, it is not possible for me to test on Vista or Windows 7. I only have XP at my disposal.

Do you not have any pc of your own?

[rogue]

Originally Posted by henryzz

Do you not have any pc of your own?

I have two Macs and a laptop from work. I cannot install it on my laptop from work.

[gd_barnes]

Originally Posted by rogue

I have two Macs and a laptop from work. I cannot install it on my laptop from work.

Now I understand things a little better. I had no idea you were working from such limited resources. I figured you had a couple of Window's quads and a couple of Mac's.

Before I get into this, did Max give you the details on the garbled web page? If not, I think I have enough details from some testing that Ian did.

Max needs to get you remote access to at least 2 of my Linux quads. BTW, I'm running 12 quads total so if for some reason, I need to stop the 7-8 cores running on 2 of the quads, that's no problem. Just let me/Max know. He has permission to "fiddle" with (start and/or stop) what they are running as needed for such testing.

Unfortunately my only Windows machines are a slower 32-bit dual-core laptop and a brand new (screaming ) I7 running Windows 7. The problem with the I7 is that I don't really want it for public use. It's my one computer I'm going to have all of my personal files on. But...Max might be able to talk me into having it for public use if he can convince me that everything on it can be made completely inaccessable except for prime search/factoring stuff. I'm sure he can but I need to "see" that it can first. I was born in MO; gotta "show me". :-)

I mention this because we need to get you at least 4 and preferrably 8 cores to run Windows testing on also. My I7 is really not quite as much as I'd like to use for alpha testing but it would probably be better than what we're currently doing.

BTW, how have you been testing changes in Windows?

OK, Mark, I'd like to define some things and make a suggestion, if I could, please:

First: Alpha testing definition: Testing of all PRPnet changes across all platforms running at least 4 cores and preferrably 8 cores at once to get a "little bit" of stress on the server. I'll define "all" platforms as Windows, Linux, and Mac. Windows XP, IMHO, should be fine for testing on. For example, for the Windows test, I think it would be sufficient, for you to utilize 2 cores on your Windows XP machine plus the (effectively) 5 cores of my I7. I7's use hyper-threading and 8 "virtual" CPUs to make them process slightly more than the 4 cores of a quad core.

Second: Beta testing definition: Testing all PRPnet changes across the Linux and Windows platforms running at least 40 cores and preferrably 60-100 cores at once. Note that I'm leaving out Mac in this case. I don't personally know of any heavy hitters running a large # of Mac machines.

What I would like to suggest please:

1. No more public releases until a full alpha test has been conducted. This means that all features have been tested and verified across all platforms. A limited # of us, perhaps, Ian, Max, and myself could possibly assist in the alpha testing phase but I'd really like that to be fully done by you if possible once you get the appropriate access to my machines as needed. Note: From your perspective, you may need to enlist the help of people from other projects to test other "forms" like twins or GFNs or other such tests.

2. Once full alpha testing is complete, then do a new public release in this thread and elsewhere as needed and announce that it is in "beta testing" phase. This is when we do the all out stress testing, preferrably running 50+ cores with a mixture of Windows and Linux machines. (I know heavy hitters running both platforms so this shouldn't be a problem.)

What #2 will do is greatly reduce the # of public releases and will make it much more managable for the "heavy hitters". I myself know that it takes a lot of time to continually update software across a lot of cores. Others like Lennart and Vaughan likely have many more cores than I do. The reason it will be more manageable is that it is only in beta testing where they will come in. They will then update all of their cores with the new PRPnet and let 'er rip.

My point is that I feel that we've wasted a lot of the heavy hitters time with what has frequently turned out to be non-stress test related problems. I don't want to bother the big boys with stress testing until we have all of the trivial problems out of the way.

Does this sound like something we can live with?


Gary