Testing.... - Page 18

mdettweiler · 2010-03-04, 05:58

Quote:

Originally Posted by gd_barnes

Max, you said that the do.pl script should work in Windows. Can you try cancelling pairs with it on your Windows machine and see if it properly writes a "CANCEL" record to joblist.txt for all pairs including the final pair? I'm wondering if this is a Windows-related issue or if it is an Awk script related issue. I really don't want the same pair showing up on the screen as having been canceled twice.

Okay, tested this and it works on Windows.

kar_bon · 2010-03-04, 06:40

Quote:

Originally Posted by gd_barnes

An enhancement:
I've added code to the pairs cancellation that I think is very beneficial to the user. I've added displays to the screen that show how many completed results are sent and how many pairs are canceled when the user does do.pl -c. Here is the way that the previous example looks now that I've made the change:

the counting is not easy with normal DOS-batch commands ('find' can count lines but the results is a line of text not only a number!).
i try to find another solution.

to the cancel-test:
i've paused the script during the cancellation and added a blank line maually. after the first pair was cancelled, the blank line was gone (-> llrnet writes the workfile.txt new!) i added it again. but not the effect you discribed for perl.

sugggestion:
let this be as it is in windows for now and i search more for the problem in the llrnet.lua. eliminating the issue there is the only right thing for 'real' programmers!

gd_barnes · 2010-03-04, 06:49

Quote:

Originally Posted by mdettweiler

Okay, tested this and it works on Windows.

Very good. Thanks for doing that Max. Karsten, is there a way to simulate a null line in the Awk script? If not, can you somehow suppress the 2nd display of the cancellation of the final pair on the screen?

In the mean time, I've now fully tested the Linux client on Riesel base 2, Sierp 2, Riesel 3, and Sierp 3. All is working as it should.

The only remaining quirks that I am aware of that cannot be easily coded for at this time are:
1. Bases that are powers of 2 and k's that are multiples of the base that won't work. (I haven't tested it but after thinking a bit about it, I suspect that LLR does not convert powers of other bases such as bases 9, 25, 36, etc.
2. A Frobenius PRP will show up as "Prime!" in the server results. (Perhaps Karsten can make a change to a server file for this issue.)

Attached is the final do.pl script. The only change from the last one was to add the displays of the # of results completed and # of pairs cancelled when doing the do.pl -c cancellation.

I'm now out of time to carefully review and sync up all files in the Linux and Windows clients and run some parallel tests before leaving on my trip. I'll have some limited time to do that while I'm gone.

Here is what I think would get this done as quickly as possible:

1. Karsten look into that final cancelled pair issue.
2. Karsten, I would suggest testing all n=1 to 1000 for Riesel base 2, Sierp 2, Riesel 3, and Sierp 3 for a large k. (Perhaps you've already done this.)
3. Karsten, is it possible to quickly fix the issue with "prime!" showing up in the server results for Frobenius PRPs? If not, don't worry about it.
4. I will Email both of you guys my entire client.
5. Max, please find the PM from Karsten where he says that some corrections are needed in README.txt on the Linux side. I did a partial rewrite of some of your wording to 3rd person and ended up with some incorrect statements in the 3rd para. as well as the word "code" used 3 different times in the 1st sentence of the 1st para.
6. Max, please download Karsten's latest client.
7. Max, for files with the same name, please run that Perl script that you have that compares files to one another looking for differences.
8. Discuss between you how to reconcile any differences between the files.
9. After the client files are synced up, Karsten, please inform Max if there are any changes to the server files that are needed. If so, please coordinate on syncing those up.
10. After all documentation correction/file sync up issues are resolved, Email me and I'll coordinate some parallel testing between the 2 clients. Likely I'll remotely load up 2 servers with the same pairs and I will run a Linux client against one of them and one of you guys can run a Windows client against the other.

I'll be pressed for time on the business trip but I still usually end up doing something for the projects for up to 2 hours each day while I'm gone.

The above is just my two cents so please don't take is as "orders". :-) I'm just trying to get a starting point for the final stuff that needs to be done before we release this publicly. If you guys don't have time or feel differently about who should do what, then by all means, change it up a little.

One final bit of information: For parallel testing on the gb servers, we'll use port 9950 for Windows clients and port 9985 for Linux clients. I'm actually going to load them up right now with some Riesel base 2 tests for n=1 to 1000; although I will not actually start the servers. When we're ready to start them, either Max or I can get them rollilng.

Both of you, I wanted to thank you immensely. Karsten especially for conceiving of the idea to begin with. Max for a quick job of writing a Perl script that closely replicated Karsten's script. Even with some of the issues that I found, I was amazed at how close both scripts were in their original state before testing to what will likely be the final version of them before we release this.

Gary

mdettweiler · 2010-03-04, 08:16

@Gary: sounds like a plan. I've uploaded the latest files to the web.

Could you also send me your latest README.txt file for do.pl? I must not have the latest as mine still has the "code" in three places in one sentence.

I looked in my PM box but couldn't find a PM from Karsten detailing changes that needed to be made to the readme; when I get your latest one I'll read it over in depth and fix any problems I find.

kar_bon · 2010-03-04, 08:36

Quote:

Originally Posted by gd_barnes

Karsten, is there a way to simulate a null line in the Awk script? If not, can you somehow suppress the 2nd display of the cancellation of the final pair on the screen?

a 'null'-line as you did it in the do.pl script ('\n') a newline-char is the same in WIN!
as i mentioned, llrnet.exe (or the runable in Unix) will (so the llrnet.lua says) write the workfile.txt new after one pairs was cancelled, so this 'newline' at the end should be deleted! please test this again: pause the script in the for-loop when "llrnet -c" is called!
look at workfile.txt, if the last empty line is there! (the "print WKF "\n";" is before the loop)

Quote:

2. A Frobenius PRP will show up as "Prime!" in the server results. (Perhaps Karsten can make a change to a server file for this issue.)
(...)

1. Karsten look into that final cancelled pair issue.
2. Karsten, I would suggest testing all n=1 to 1000 for Riesel base 2, Sierp 2, Riesel 3, and Sierp 3 for a large k. (Perhaps you've already done this.)
3. Karsten, is it possible to quickly fix the issue with "prime!" showing up in the server results for Frobenius PRPs? If not, don't worry about it.

to 1:
i'll try to fix this problem. i think it has to do with the pair-list handled in the whole server-client-thing: the queue holds all kn-pairs from workfile.txt except the first one: this is a global declaration of t/k/n-values in the source. so if only one pair is in workfile.txt the queue is empty and t/k/n filled with the one pair!

to 2:
i will test when all is ready for the WIN-version.

to 3:
this is no problem, but what do you want the results should look like?
so we could say PRP is result="-3" and the server puts "... is PRP" instead of prime/not prime. but this has consequences in processing those result-files: you have to support PRP in there, too (like i use my processing script for a long time -> changes have to be made!).

Karsten

kar_bon · 2010-03-04, 14:22

Quote:

Originally Posted by mdettweiler

@Gary: sounds like a plan. I've uploaded the latest files to the web.

Could you also send me your latest README.txt file for do.pl? I must not have the latest as mine still has the "code" in three places in one sentence.

I looked in my PM box but couldn't find a PM from Karsten detailing changes that needed to be made to the readme; when I get your latest one I'll read it over in depth and fix any problems I find.

here:

Code:

sentence:
"Karsten also made small changes to the LLR 3.8.0 source code to remove some (...)"
is quite false! i've not made changes in the source code! i've only patched (means changed
some binary values) in the cLLR.exe directly!

sentence:
"modified Windows LLR executable, the changes to Linux LLR are as follows:
-------------
Patched "cllr.exe":
4560CC: 00 -> V1 = ... ; Computing U0...done.
456344: 00 -> Starting Lucas Lehmer Riesel prime test of...
-------------"

this is only true for the WIN cLLR.exe! the unix program is totally different. i have not
checked this and if that would work in Unix, too. in compiled Win-programs, say executable,
a string for output will ends (mostly i know) with a hex '00'. so i made the beginning
(first char) of this text to '00' only and the print function has nothing to print -> end of text!

perhaps a note/example for the llr-clientconfig.txt should be given (as it is in the testing).

the remaining text is for the Unix-version only so a separate ReadMe for Win i think?
it's quite better because i use other setting-variables.

mdettweiler · 2010-03-04, 17:04

Quote:

Originally Posted by kar_bon

a 'null'-line as you did it in the do.pl script ('\n') a newline-char is the same in WIN!
as i mentioned, llrnet.exe (or the runable in Unix) will (so the llrnet.lua says) write the workfile.txt new after one pairs was cancelled, so this 'newline' at the end should be deleted! please test this again: pause the script in the for-loop when "llrnet -c" is called!
look at workfile.txt, if the last empty line is there! (the "print WKF "\n";" is before the loop)

Hmm...well, the strange thing is, printing "\n" to the end of the file works perfectly with do.pl both on Windows and Linux. It definitely makes a difference from how it behaved without the "\n", since this way it actually cancels everything correctly.

Quote:

to 3:
this is no problem, but what do you want the results should look like?
so we could say PRP is result="-3" and the server puts "... is PRP" instead of prime/not prime. but this has consequences in processing those result-files: you have to support PRP in there, too (like i use my processing script for a long time -> changes have to be made!).

What I might suggest is something like this: have it use result="-2" but put in the residual field "Frobenius PRP". Then have the server piece together the line as follows:
x is [result]! Time: x.x sec.
For a prime, you'd say "prime" in the residual field, or for Frobenius PRP you'd put that there. So it would produce:
x is prime! Time: x.x sec.
x is Frobenius PRP! Time: x.x sec.
Of course this still has the disadvantage of requiring results-handing scripts (not to mention the stats database) to be changed, which is not a particularly ideal prospect.

Perhaps what might be best, at least for now, is just to leave it as is. The admins would then just know whether they're dealing with pairs that might come up PRP instead of prime; usually for anything small enough to do that, a re-verification run to prove them all for sure with PFGW would be quite trivial.

Quote:

Originally Posted by kar_bon

here:

Code:

sentence:
"Karsten also made small changes to the LLR 3.8.0 source code to remove some (...)"
is quite false! i've not made changes in the source code! i've only patched (means changed
some binary values) in the cLLR.exe directly!
 
sentence:
"modified Windows LLR executable, the changes to Linux LLR are as follows:
-------------
Patched "cllr.exe":
4560CC: 00 -> V1 = ... ; Computing U0...done.
456344: 00 -> Starting Lucas Lehmer Riesel prime test of...
-------------"
 
this is only true for the WIN cLLR.exe! the unix program is totally different. i have not checked this and if that
would work in Unix, too. in compiled Win-programs, say executable, a string for output will ends (mostly i know)
with a hex '00'. so i made the beginning (first char) of this text to '00' only and the print function has nothing to
print -> end of text!
 
perhaps a note/example for the llr-clientconfig.txt should be given (as it is in the testing).
 
the remaining text is for the Unix-version only so a separate ReadMe for Win i think?
it's quite better because i use other setting-variables.

Oh! Okay. I didn't realize you just patched the .exe directly. I'll fix that then.

mdettweiler · 2010-03-04, 17:12

Gary, I got your email with the latest client. Here's the uploaded packages:
http://nplb-gb1.no-ip.org/sieves/llr...0.70-win32.zip
http://nplb-gb1.no-ip.org/sieves/llr...70-linux32.zip
(Note that the links are different from before because I realized just now that I was previously uploading 0.70 clients to links that said 0.61 in the name.

)

kar_bon · 2010-03-04, 21:27

here're some new findings:

primary note: as i can see from the source-code, llrnet-server will prune his joblist and knpairs files only when there's a request/result from any client.
so if the server dried and the joblist contains jobs in process and the results were sent, the server has to push for pruning, say calling 'llrserver -s' for simplifying.

what i have done:
i haven't made any change in the llrnet.lua (same as from Gary's mail sent today) or llrserver.lua.
i started the local server with 20 knpairs. the client WUChacheSize was set to 5.
calling do.bat and the first 5 pairs were tested and sent. from the next 5 pairs, cllr tested 2 and the script was cancelled with CTRL-C.
i converted the 2 results (manually) to tosend.txt and called 'llrnet' -> the 2 results were sent to the server (all now 7 results were in the server-result-file!). delete those 2 pairs from client-workfile.txt.
next i called 4 times 'llrnet -c', 4 because one more than pairs in the workfile from the client! important!
now the 3 pairs cancelled from the client are listed in the server-joblist with
"status"="abandonned" and "result"="CANCEL"!

could someone please check this behaviour before i edit the luas!

perhaps a final 'llrserver -s' will do the same when 'llrnet -c' called 3times only.

kar_bon · 2010-03-04, 21:46

new test:

almost same procedure as before:
- 5 pairs done and sent to server
- without removing those 2 pairs from the client-workfile, converting/submitting the 2 results to server and calling 'do -c' and a separate 'llrnet -c' again
-> server got 7 results, 3 cancelled pairs still in server-knpair-file, joblist contains 3 cancelled pairs with abandonned/CANCEL!

please check this, too.

so it seems, there's only an extra -c call from llrnetclient to do the cancelling correct, perhaps a server -s could be necessary.

short:
- submit all pairs done by cllr to the server
- calling llrnet -c 1times-more-than-pairs-in-workfile
- (perhaps call llrserver -s)

that seems all!

kar_bon · 2010-03-04, 23:30

ok, here is the WIN-version of the local server/client with the above changes in the do.bat and a knpair-file in the server for testing.

how to:
- start the server by calling 'llrserver' (in folder LLRnet_server)
- start 'do.bat' (in folder LLRnet_client)
- let cllr do 5 pairs first and another 2
- stop the client with CTRL-C
- call 'do -c'
- stop server
- call 'simplify.bat' (in folder LLRnet_server)

check all files (server-result, job/knpair-files, client results) for completeness!

2010-03-04, 21:27	#196
kar_bon Mar 2006 Germany 2³×379 Posts	cancelling: neverending st... no not really here're some new findings: primary note: as i can see from the source-code, llrnet-server will prune his joblist and knpairs files only when there's a request/result from any client. so if the server dried and the joblist contains jobs in process and the results were sent, the server has to push for pruning, say calling 'llrserver -s' for simplifying. what i have done: i haven't made any change in the llrnet.lua (same as from Gary's mail sent today) or llrserver.lua. i started the local server with 20 knpairs. the client WUChacheSize was set to 5. calling do.bat and the first 5 pairs were tested and sent. from the next 5 pairs, cllr tested 2 and the script was cancelled with CTRL-C. i converted the 2 results (manually) to tosend.txt and called 'llrnet' -> the 2 results were sent to the server (all now 7 results were in the server-result-file!). delete those 2 pairs from client-workfile.txt. next i called 4 times 'llrnet -c', 4 because one more than pairs in the workfile from the client! important! now the 3 pairs cancelled from the client are listed in the server-joblist with "status"="abandonned" and "result"="CANCEL"! could someone please check this behaviour before i edit the luas! perhaps a final 'llrserver -s' will do the same when 'llrnet -c' called 3times only. Last fiddled with by kar_bon on 2010-03-04 at 21:48

2010-03-04, 21:46	#197
kar_bon Mar 2006 Germany 3032₁₀ Posts	new test: almost same procedure as before: - 5 pairs done and sent to server - without removing those 2 pairs from the client-workfile, converting/submitting the 2 results to server and calling 'do -c' and a separate 'llrnet -c' again -> server got 7 results, 3 cancelled pairs still in server-knpair-file, joblist contains 3 cancelled pairs with abandonned/CANCEL! please check this, too. so it seems, there's only an extra -c call from llrnetclient to do the cancelling correct, perhaps a server -s could be necessary. short: - submit all pairs done by cllr to the server - calling llrnet -c 1times-more-than-pairs-in-workfile - (perhaps call llrserver -s) that seems all! Last fiddled with by kar_bon on 2010-03-04 at 21:56

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Anti-poverty drug testing vs "high" tax deduction testing	kladner	Soap Box	3	2016-10-14 18:43
What am I testing?	GARYP166	Information & Answers	9	2009-02-18 22:41
k=243 testing ??	gd_barnes	Riesel Prime Search	20	2007-11-08 21:13
Testing	grobie	Marin's Mersenne-aries	1	2006-05-15 12:26
Speed of P-1 testing vs. Trial Factoring testing	eepiccolo	Math	6	2006-03-28 20:53

2010-03-04, 08:16	#191
mdettweiler A Sunny Moo Aug 2007 USA 2³·3³·29 Posts	@Gary: sounds like a plan. I've uploaded the latest files to the web. Could you also send me your latest README.txt file for do.pl? I must not have the latest as mine still has the "code" in three places in one sentence. I looked in my PM box but couldn't find a PM from Karsten detailing changes that needed to be made to the readme; when I get your latest one I'll read it over in depth and fix any problems I find.

2010-03-04, 17:12	#195
mdettweiler A Sunny Moo Aug 2007 USA 6264₁₀ Posts	Gary, I got your email with the latest client. Here's the uploaded packages: http://nplb-gb1.no-ip.org/sieves/llr...0.70-win32.zip http://nplb-gb1.no-ip.org/sieves/llr...70-linux32.zip (Note that the links are different from before because I realized just now that I was previously uploading 0.70 clients to links that said 0.61 in the name. )

2010-03-04, 23:30	#198
kar_bon Mar 2006 Germany 2³·379 Posts	ok, here is the WIN-version of the local server/client with the above changes in the do.bat and a knpair-file in the server for testing. how to: - start the server by calling 'llrserver' (in folder LLRnet_server) - start 'do.bat' (in folder LLRnet_client) - let cllr do 5 pairs first and another 2 - stop the client with CTRL-C - call 'do -c' - stop server - call 'simplify.bat' (in folder LLRnet_server) check all files (server-result, job/knpair-files, client results) for completeness!