PRPnet - Page 7

kar_bon · 2010-01-25, 00:26

only for another information: i've changed my LLRnet-file 'llrnet.lua' and 'client.lua' for displaying the results in 'lresults.txt' like this:

Code:

[2010-01-24 03:31:11] 2445*2^526111-1 is not prime.  Res64: 718DAA79CA386FD4  Time : 610.165 sec.
[2010-01-24 03:41:22] 2493*2^526111-1 is not prime.  Res64: 330234CFD2F0B77B  Time : 609.989 sec.
[2010-01-24 03:51:32] 2563*2^526111-1 is not prime.  Res64: 837A3BF4E9B5FB2F  Time : 610.112 sec.
[2010-01-24 04:01:42] 2781*2^526111-1 is not prime.  Res64: B481441102FA5DAE  Time : 609.728 sec.
[2010-01-24 04:11:52] 2929*2^526111-1 is not prime.  Res64: 3A54A9D73847E985  Time : 609.897 sec.
[2010-01-24 04:22:02] 2991*2^526111-1 is not prime.  Res64: 3BB36A1F3EE4E26A  Time : 609.961 sec.
[2010-01-24 04:32:12] 2043*2^526112-1 is not prime.  Res64: F126A11EE5562927  Time : 609.768 sec.
[2010-01-24 04:42:22] 2117*2^526112-1 is not prime.  Res64: D6F069913D1BD6E3  Time : 609.736 sec.

the line with '... sucessfully sent to the server' omitted and time/dates in the file added.

all primes found will also recorded in a separate file 'primes.txt' in the dir-level above the LLRnet-dir:

Code:

[2009-10-29 07:50:19] 741 724834
[2009-12-02 08:25:37] 699 736845
[2009-12-23 10:16:49] 1895 665256

so only a wink of an eye to see, if there's a new prime found!

perhaps a switch to turn off all server/client messages in the PRP-output will do it, too.
i think, there's no need for all those messages put in a file when all is working fine. if not switch on and record those texts.

for PRPnet with 4 cores (-> 4 dirs with different ini's) i use such script to determine the number of tests or primes found:

Code:

@echo off
if exist PRPnet_G3001\llr.prime type PRPnet_G3001\llr.prime
find /C "Residue" PRPnet_G3001\prpclient.log
if exist PRPnet_G3002\llr.prime type PRPnet_G3002\llr.prime
find /C "Residue" PRPnet_G3002\prpclient.log
if exist PRPnet_G3003\llr.prime type PRPnet_G3003\llr.prime
find /C "Residue" PRPnet_G3003\prpclient.log
if exist PRPnet_G3004\llr.prime type PRPnet_G3004\llr.prime
find /C "Residue" PRPnet_G3004\prpclient.log
pause

which gives me this info:

Code:

---------- PRPNET_G3001\PRPCLIENT.LOG: 343
---------- PRPNET_G3002\PRPCLIENT.LOG: 341
---------- PRPNET_G3003\PRPCLIENT.LOG: 332
---------- PRPNET_G3004\PRPCLIENT.LOG: 331

(no prime yet so all llr.prime do not exist)

Karsten

gd_barnes · 2010-01-26, 00:23

I agree with Karsten here about needing separate results and logging. But...I think Mark is giving us our wish with version 3.1.5. We've needed a separate results and logging file for a very long time now. I made the mistake in thinking that Max had brought this up to you at some point.

You said you need what pieces of information you want logged as for the results on the client side. I gave that to you in detail cut-and-pasted from an LLRnet server and client. Please reread my prior post. As for the server messages that have been in prpclient.log that need to be in a separate file from the results, I'll leave it up to Max on that. I think what we're getting now is fine.

As for how I stated the requirement. I stated it in a manner that I thought you knew what a results file was like is written out by LLR, PFGW, Proth, Proht, LLRnet, etc. That's all we've needed this entire time. I had not realized that you had not run an LLRnet server before. I'm sorry if it came out in a confusing manner. Now that I know that, I'll know to give you specific examples cut-and-pasted from other software of what is needed instead of just stating what is needed.

Not to beat a dead horse here Mark, but I feel like it would be very good if you would download an LLRnet server, set it up, load it with pairs, and then put at least 2 quads worth of clients on it for several hours. I realize that it is somewhat klunky and inflexible and that we are only base 2 and that LLRnet is only intended for base 2. But the files that LLRnet writes out are a close approximation of what many different prime search projects need. That is results, pairs remaining, a joblist (explained below), and a rejected file. That's the main crux of the output from LLRnet on the server side: The joblist.txt, knpairs.txt, rejected.txt, and results.txt file. Each has a very specific purpose and is easy to wade through...no extraneous information.

What it doesn't have is PRPnet's flexibility in its programs used and logging of problems.

I promise you: LLRnet will enlighten you as to what many prime search projects need. One of its best features is the joblist.txt file. For pairs handed out and not processed, it's a file that shows exactly when they were handed out and who has them "reserved". What makes it easy is that that is ALL that is shows...no other information to filter through. It can help isolate problem users.

One more thought: Would it make sense to always test a release on at least 2-3 quads for at least an hour or so before rolling it out to the general public for testing? Set the batching to 1, load it with some very small k/n pairs, and let it rip for a good starting stress test. I'm fairly confident that only a Windows setup running ~10 cores would find some of these problems related to load on the server ahead of time. (Make sure the k/n pairs have an n-value around n=50000 for the best test.) Even without a Linux testing setup, I really feel that this will nip a lot of small problems in the bud and there will be less releases. (I don't know; maybe you're already doing that level of testing.)

What I'm trying to come up with is an ahead-of-time test plan that we can all live with. The idea here is to reduce the # of releases and total testing time by everyone.

This is only my two cents on what I feel would help everyone in testing. Please know that we are with you with the MySQL version of PRPnet. We really want to get it correct this time around.

Max, I'd like just you and me to test version 3.1.5. Mark has informed us at CRUS that he didn't test it so we need to do some serious testing on it. I don't want to coordinate any kind of testing problems with several different people here. The good news is that it is supposed to have separate results (with a testing time) and server logging messages. That's a huge deal to many of us "record keepers" and "work coordinators" here at NPLB and CRUS.

Gary

mdettweiler · 2010-01-26, 00:33

Quote:

Originally Posted by gd_barnes

Max, I'd like just you and me to test version 3.1.5. Mark has informed us at CRUS that he didn't test it so we need to do some serious testing on it. I don't want to coordinate any kind of testing problems with several different people here. The good news is that it is supposed to have separate results (with a testing time) and server logging messages. That's a huge deal to many of us "record keepers" and "work coordinators" here at NPLB and CRUS.

Okay. Mark just sent me an email answering a question I had about building the 3.1.5 server for Linux, so I should be able to get it built and ready to roll shortly.

rogue · 2010-01-26, 02:05

Quote:

Originally Posted by gd_barnes

Not to beat a dead horse here Mark, but I feel like it would be very good if you would download an LLRnet server, set it up, load it with pairs, and then put at least 2 quads worth of clients on it for several hours. I realize that it is somewhat klunky and inflexible and that we are only base 2 and that LLRnet is only intended for base 2. But the files that LLRnet writes out are a close approximation of what many different prime search projects need. That is results, pairs remaining, a joblist (explained below), and a rejected file. That's the main crux of the output from LLRnet on the server side: The joblist.txt, knpairs.txt, rejected.txt, and results.txt file. Each has a very specific purpose and is easy to wade through...no extraneous information.

IMO, just because LLRNet does something doesn't mean that PRPNet has to do it the same way, if at all. I created the other thread so that you and others could give me detailed requirements so that PRPNet could replace LLRNet. None of what you are listing here is in that thread. Note that PrimeGrid is using PRPNet on many diverse projects and nobody over there has asked for these things.

I agree that I need to do more testing, but I don't have any quads to do testing with. I also do not have Linux boxes to test with and Linux has been the source of most of the more difficult problems that PRPNet has run into. I have done a lot of testing on Mac OS X and Windows, and they have not exhibited any (and I mean any) of the socket or MySQL problems that Linux has exhibited. Many of the mods for PRPNet are specifically written to address the nuances of Linux.

gd_barnes · 2010-01-26, 04:45

Quote:

Originally Posted by rogue

IMO, just because LLRNet does something doesn't mean that PRPNet has to do it the same way, if at all. I created the other thread so that you and others could give me detailed requirements so that PRPNet could replace LLRNet. None of what you are listing here is in that thread. Note that PrimeGrid is using PRPNet on many diverse projects and nobody over there has asked for these things.

I agree that I need to do more testing, but I don't have any quads to do testing with. I also do not have Linux boxes to test with and Linux has been the source of most of the more difficult problems that PRPNet has run into. I have done a lot of testing on Mac OS X and Windows, and they have not exhibited any (and I mean any) of the socket or MySQL problems that Linux has exhibited. Many of the mods for PRPNet are specifically written to address the nuances of Linux.

I'm through with this conversation.

This post says what we wanted related to the testing time; that is what side it needed to be on (client side), yet somehow we weren't specifc enough. Did we need to say exactly what file it needed to be in? If so, sorry. I'll do that in the future.

You don't need "linux boxes". You can run Linux Ubuntu on a Windows machine.

The problems are load related, not Linux related.

Just because LLRnet has it doesn't mean that PRPnet should NOT have it.

etc. etc.

I could say more but it's not worth it.

rogue · 2010-01-26, 13:55

Quote:

Originally Posted by gd_barnes

You don't need "linux boxes". You can run Linux Ubuntu on a Windows machine.

The problems are load related, not Linux related.

I would get into quite a bit of trouble "borrowing" my wife's PC to install Linux. I get into enough trouble running other stuff on it. My other windows PC is from work and I definitely cannot touch that one. In other words, I don't have a Windows box that I am free to install Linux on.

Depending upon the problem, a number of them have been Linux only. All of the issues I've had with buffering of data in sockets can only be reproduced on Linux. The duplicate keys problem existed in an early version of 3.x, but was fixed when changing the engine to InnoDB and making a code change on the server. I pounded my own server harder than you guys are pounding yours (small workunits and 5 clients) and I did not see the duplicate key issue after those changes. From the logs I saw you quickly got duplicate key issues with only 4 clients and medium sized workunits.

I don't have enough information yet regarding the "No candidates available on server" message. This is the most troublesome issue (to me) that I would like to get resolved.

vaughan · 2010-01-27, 08:19

Quote:

Originally Posted by rogue

I would get into quite a bit of trouble "borrowing" my wife's PC to install Linux. I get into enough trouble running other stuff on it. ..

Yeah I get that scolding too.

It runs something like ... "What's this cr@p running on MY computer. Get it off there now."

My solution is to run projects that allow it eg. the old D2OL project and currently DPAD/Muon from a USB memory stick. When I hear her car in the driveway I quickly hit Ctrl-C, Y and safely remove the flash drive. A quick shutdown of Windows completes the task.

henryzz · 2010-01-27, 16:37

Quote:

Originally Posted by rogue

I would get into quite a bit of trouble "borrowing" my wife's PC to install Linux. I get into enough trouble running other stuff on it. My other windows PC is from work and I definitely cannot touch that one. In other words, I don't have a Windows box that I am free to install Linux on.

What about a virtual machine? Virtualbox should do exactly what you need.

rogue · 2010-01-27, 16:49

Quote:

Originally Posted by henryzz

What about a virtual machine? Virtualbox should do exactly what you need.

The issue is that I use her PC and put other software on it. I don't see how a virtual environment solves the problem.

henryzz · 2010-01-27, 17:23

Quote:

Originally Posted by rogue

The issue is that I use her PC and put other software on it. I don't see how a virtual environment solves the problem.

If you use a virtual environment then linux will effectively be like a program instead of having to create a partition, boot into it etc. and yet it will still have all the functionality of normal linux.

mdettweiler · 2010-01-28, 00:22

Actually, what might be even better is for me to set up an account on the server itself for Mark. That way he could debug and test the software directly on the machine we're intending to use it on. Thoughts?

2010-01-26, 00:23	#68
gd_barnes May 2007 Kansas; USA 7²×211 Posts	I agree with Karsten here about needing separate results and logging. But...I think Mark is giving us our wish with version 3.1.5. We've needed a separate results and logging file for a very long time now. I made the mistake in thinking that Max had brought this up to you at some point. You said you need what pieces of information you want logged as for the results on the client side. I gave that to you in detail cut-and-pasted from an LLRnet server and client. Please reread my prior post. As for the server messages that have been in prpclient.log that need to be in a separate file from the results, I'll leave it up to Max on that. I think what we're getting now is fine. As for how I stated the requirement. I stated it in a manner that I thought you knew what a results file was like is written out by LLR, PFGW, Proth, Proht, LLRnet, etc. That's all we've needed this entire time. I had not realized that you had not run an LLRnet server before. I'm sorry if it came out in a confusing manner. Now that I know that, I'll know to give you specific examples cut-and-pasted from other software of what is needed instead of just stating what is needed. Not to beat a dead horse here Mark, but I feel like it would be very good if you would download an LLRnet server, set it up, load it with pairs, and then put at least 2 quads worth of clients on it for several hours. I realize that it is somewhat klunky and inflexible and that we are only base 2 and that LLRnet is only intended for base 2. But the files that LLRnet writes out are a close approximation of what many different prime search projects need. That is results, pairs remaining, a joblist (explained below), and a rejected file. That's the main crux of the output from LLRnet on the server side: The joblist.txt, knpairs.txt, rejected.txt, and results.txt file. Each has a very specific purpose and is easy to wade through...no extraneous information. What it doesn't have is PRPnet's flexibility in its programs used and logging of problems. I promise you: LLRnet will enlighten you as to what many prime search projects need. One of its best features is the joblist.txt file. For pairs handed out and not processed, it's a file that shows exactly when they were handed out and who has them "reserved". What makes it easy is that that is ALL that is shows...no other information to filter through. It can help isolate problem users. One more thought: Would it make sense to always test a release on at least 2-3 quads for at least an hour or so before rolling it out to the general public for testing? Set the batching to 1, load it with some very small k/n pairs, and let it rip for a good starting stress test. I'm fairly confident that only a Windows setup running ~10 cores would find some of these problems related to load on the server ahead of time. (Make sure the k/n pairs have an n-value around n=50000 for the best test.) Even without a Linux testing setup, I really feel that this will nip a lot of small problems in the bud and there will be less releases. (I don't know; maybe you're already doing that level of testing.) What I'm trying to come up with is an ahead-of-time test plan that we can all live with. The idea here is to reduce the # of releases and total testing time by everyone. This is only my two cents on what I feel would help everyone in testing. Please know that we are with you with the MySQL version of PRPnet. We really want to get it correct this time around. Max, I'd like just you and me to test version 3.1.5. Mark has informed us at CRUS that he didn't test it so we need to do some serious testing on it. I don't want to coordinate any kind of testing problems with several different people here. The good news is that it is supposed to have separate results (with a testing time) and server logging messages. That's a huge deal to many of us "record keepers" and "work coordinators" here at NPLB and CRUS. Gary Last fiddled with by gd_barnes on 2010-01-26 at 00:37

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
PSP goes prpnet	ltd	Prime Sierpinski Project	86	2012-06-06 02:30
Setting up PRPnet	Mattyp101	Conjectures 'R Us	2	2011-02-07 13:53
PRPNet 4.0.1 Released	Joe O	Sierpinski/Riesel Base 5	1	2010-10-22 20:11
PRPNet 3.0.0 Released	rogue	Conjectures 'R Us	220	2010-10-12 20:48
PRPNet released!	rogue	Conjectures 'R Us	250	2009-12-27 21:29

2010-01-28, 00:22	#77
mdettweiler A Sunny Moo Aug 2007 USA (GMT-5) 1100001101001₂ Posts	Actually, what might be even better is for me to set up an account on the server itself for Mark. That way he could debug and test the software directly on the machine we're intending to use it on. Thoughts?