[quote=mdettweiler;207043]NO! Don't do that. :smile: If anything, we should change the Windows server to be like the Linux one. The database is already set up to parse and import stuff in the Linux format.

BTW, clients and servers are completely separate as far as date formats go. They don't have to be synced up; the only thing that really needs to be consistent is on the server end, so that when we import it into the DB it works right. On the client end, it's entirely for human reading since there'd be no reason to have that imported into the DB.[/quote]

I'm completely lost. You and Karsten fight it out about the dates. I'm tired of hearing about the issue. I thought I understood Karsten with his latest PM about it. That's why I asked him specifically about it.

You have to trust me on this. Karsten knows what he is talking about. He does a lot of behind-the-scenese conversion of results.

Perhaps you are misunderstanding. We will NOT be changing the Linux date formats as they are written to files anywhere. We will just be changing the CODE that formats them. In other words:

1. We will change Linux CODE to be like CORRECTED Windows CODE.
2. We will NOT change Linux results files output. (In other words, there are at least 2 different ways to display Linux dates in the same format.)
3. Karsten will change the Windows CODE and RESULTS so that that the Windows results will look like the current Linux results.

Karsten, am I right on this?

Max, are you clear yet?

I wish this issue had started in this thread instead of PMs. I'm sure we would not still be discussing it.

(caps for emphasis)


Gary

My script is working like a charm now for base 2. No problem with strange results output from teeny tests for large k's and no problem when there are rejected results. I even tried canceling pairs when there were rejected results lines within the most recent batch. That was a good test of the code. It worked perfectly!

Now, we have another issue for base 3 (and for other bases) and it's really unrelated to LLRnet: Rounding errors. Rounding errors apparently go into an infinite loop on LLR. Here is an example of my recent test:

[quote]
100542584*3^323-1 is not prime. RES64: B516DDAC3C2DA591. OLD64: C681FF88490C1762 Time : 1.233 ms.
100542584*3^324-1 is not prime. RES64: B9EAF440B2630D00. OLD64: A89CF68775EDE106 Time : 1.250 ms.
100542584*3^325-1 is not prime. RES64: 57B2D8B312BADF70. OLD64: E84124B970CCFA7F Time : 2.213 ms.
Submitted to server at [2010-03-01 22:06:55]
100542584*3^326-1 is not prime. RES64: FDCBF11CBD949DB8. OLD64: 9CDDA336E292EDB7 Time : 5.832 ms.
100542584*3^327-1 is not prime. RES64: 8828751C67D74542. OLD64: 98795F553785CFC3 Time : 1.112 ms.
100542584*3^328-1 is not prime. RES64: AF0F967B018B1A05. OLD64: 0D2EC37104A14E0C Time : 23.712 ms.
100542584*3^329-1 is not prime. RES64: 0BC6192E5B9C413B. OLD64: 612B383CFC4DEEE0 Time : 1.098 ms.
100542584*3^330-1 is not prime. RES64: 858763DE10557C9C. OLD64: 4A20F1AFED6BF763 Time : 1.736 ms.
100542584*3^331-1 is not prime. RES64: F62BB139C363FC20. OLD64: 78D33CCDE4CD36B6 Time : 1.118 ms.
100542584*3^332-1 is not prime. RES64: 16D3F5810CDB888A. OLD64: CA5CD746C65A27AD Time : 1.086 ms.
100542584*3^333-1 is not prime. RES64: D161A7B4A11D97EA. OLD64: BCF6694353041CD4 Time : 1.102 ms.
100542584*3^334-1 is not prime. RES64: 8AC3F5B355AAD00E. OLD64: 7AC0378A50026F70 Time : 1.708 ms.
100542584*3^335-1 is not prime. RES64: 6736C9D7E97D4CCF. OLD64: C50160D8A97DE443 Time : 1.125 ms.
100542584*3^336-1 is not prime. RES64: B65E42722A31566F. OLD64: D131D14945A5FCD3 Time : 1.146 ms.
100542584*3^337-1 is not prime. RES64: CDFAB4C26501BE9F. OLD64: 7E7A59F7D971150C Time : 32.833 ms.
100542584*3^338-1 is not prime. RES64: 9CCAA4CC820D465F. OLD64: F52F47EE85C998E3 Time : 1.927 ms.
100542584*3^339-1 is not prime. RES64: 068CBF65D4E28A02. OLD64: 13A63E317EA79E03 Time : 1.445 ms.
100542584*3^340-1 is not prime. RES64: A2DAE029B63CEEF2. OLD64: FDDAC64E1F66C0DC Time : 2.059 ms.
100542584*3^341-1 is not prime. RES64: 615F2CD76A644608. OLD64: 63FBF7F9353CAE2E Time : 1.331 ms.
100542584*3^342-1 is not prime. RES64: 5996C0C773651BB4. OLD64: 8BFAEB081E8E7BAB Time : 1.789 ms.
100542584*3^343-1 is not prime. RES64: 4813E4310051A928. OLD64: D83BAC9300F4FB75 Time : 1.383 ms.
100542584*3^344-1 is not prime. RES64: 49FF3A27FFC4CA16. OLD64: 56E99CB7E6A6CB51 Time : 1.402 ms.
100542584*3^345-1 is not prime. RES64: ABC93293D1BC0131. OLD64: 6E1F627B2B3D4AC2 Time : 1.322 ms.
Iter: 27/575, ERROR: ROUND OFF (0.5) > 0.40
Continuing from last save file.
Disregard last error. Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.
Disregard last error. Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.
Disregard last error. Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.
Disregard last error. Result is reproducible and thus not a hardware problem.
For added safety, redoing iteration using a slower, more reliable method.
Continuing from last save file.
[/quote]

The last 3 lines appear to continue forever. Although I didn't let it continue more than a minute or 2, it doesn't really matter if it's an infinite loop or just a very long one. The lresults.txt file was 4.8 MB by the time that I stopped it, which would be unacceptable to many users.

Interestingly, this was the test where I returned untested pairs to the server. 20 of the 25 pairs in my batch had been tested. Despite 1000s of lines of extra output from the loop, it correctly returned the 20 good results and the 5 untested pairs beginning with n=346.

Thoughts anyone?

My feeling: Forget about other bases with this release. There's more involved than we should care to get into at this point. As far as I know, base 2 does not have rounding errors. Someone correct me if I'm wrong there. If base 2 does have rounding errors, we have to ask: Are they only on small tests or where k is > 2^n? If so, we should probably ignore them. But if anyone can come up with a quick solution for coding around such a loop, then that would be OK. I just don't want to get into any special code for base 3 and other bases.

Other than the rounding error issue, it looks like base 3 was testing well. I did Riesel base 3 for the above k for n=1 thru 345. Unfortunately I believe that rounding errors occur regardless of n-size on other bases. They are quite common in PFGW; although Mark has coded an automatic -a1 test that retests them. I suppose we could see if Jean and co. could do the same for LLR. But pursuing that with this release of LLRnet, IMHO, is a waste of time for an excellent tool that is badly needed.

Edit: Karsten, so you have something to refer to, attached is the latest do.pl script. It should be 100% bugfree for base 2 for Riesel and Proth tests of all sizes. I now anticipate that this will be the final version of it.

I'll be looking at the incorrect statements in README.txt and the date issue next. I'll also do a code review of the Windows script to see if it close to synced up with the Linux do.pl script.


Gary

[quote=gd_barnes;207048]I'm completely lost. You and Karsten fight it out about the dates. I'm tired of hearing about the issue. I thought I understood Karsten with his latest PM about it. That's why I asked him specifically about it.

You have to trust me on this. Karsten knows what he is talking about. He does a lot of behind-the-scenese conversion of results.

Perhaps you are misunderstanding. We will NOT be changing the Linux date formats as they are written to files anywhere. We will just be changing the CODE that formats them. In other words:

1. We will change Linux CODE to be like CORRECTED Windows CODE.
2. We will NOT change Linux results files output. (In other words, there are at least 2 different ways to display Linux dates in the same format.)
3. Karsten will change the Windows CODE and RESULTS so that that the Windows results will look like the current Linux results.

Karsten, am I right on this?

Max, are you clear yet?

I wish this issue had started in this thread instead of PMs. I'm sure we would not still be discussing it.

(caps for emphasis)


Gary[/quote]
Okay, yes, that makes sense now. :smile:

[quote=gd_barnes;207049]My feeling: Forget about other bases with this release. There's more involved than we should care to get into at this point. As far as I know, base 2 does not have rounding errors. Someone correct me if I'm wrong there. If base 2 does have rounding errors, we have to ask: Are they only on small tests or where k is > 2^n? If so, we should probably ignore them. But if anyone can come up with a quick solution for coding around such a loop, then that would be OK. I just don't want to get into any special code for base 3 and other bases.

Other than the rounding error issue, it looks like base 3 was testing well. I did Riesel base 3 for the above k for n=1 thru 345. Unfortunately I believe that rounding errors occur regardless of n-size on other bases. They are quite common in PFGW; although Mark has coded an automatic -a1 test that retests them. I suppose we could see if Jean and co. could do the same for LLR. But pursuing that with this release of LLRnet, IMHO, is a waste of time for an excellent tool that is badly needed.[/quote]
Actually, LLR has already had an automatic next-FFT retest for many versions (since in many ways it's more closely related to Prime95, which has that feature as well, than is PFGW). In fact it's a bit more advanced than PFGW's rendition of that feature since it just re-does the affected portion of the test, rather than starting it over, potentially saving a lot of time on big tests.

What this looks like here is a bug in LLR 3.8.0. My guess is that it's backing up and retrying, but forgetting to raise the FFT in the process--so it keeps running into trouble over and over. Hence an infinite loop. I'm pretty sure that earlier versions of LLR have handled this correctly, so it would definitely be worth reporting to Jean even though we're not going to hold up the LLRnet client release on it.

BTW, base 2 can have rounding errors once in a while but they're a lot less common than other bases; these kind of things are fixed by trial-and-error tuning AFAIK, so the fact that the majority of the base 2 code in gwnum has been around a lot longer than the non-base-2 code makes for a lot less roundoff errors in base 2. Note that since this is a gwnum issue rather than an LLR one, LLR should theoretically error on exactly the same tests in exactly the same ways that PFGW does (ditto for Prime95).

[B]On another subject[/B]: the latest do.pl is working well and I'll upload the latest version to the no-IP pages shortly. The only thing I'd change is to add a test condition in convertResults() for the text "probably prime"; as it is it can, ironically enough, catch a positive result from the little-used Frobenius PRP but not a "regular" PRP test. :smile:

[quote=mdettweiler;207054][B]On another subject[/B]: the latest do.pl is working well and I'll upload the latest version to the no-IP pages shortly. The only thing I'd change is to add a test condition in convertResults() for the text "probably prime"; as it is it can, ironically enough, catch a positive result from the little-used Frobenius PRP but not a "regular" PRP test. :smile:[/quote]

I removed the phrase "probable" from the results conversion and changed it to "Frobenius PRP" because it appears the term "probable" or "probable prime" is no longer used with LLR version 3.8. Also, keep in mind that we only want to consider the result of the FINAL test of a pair. If we get "probable prime" followed by "Frobenius PRP" or simply "is prime!", then we have to ignore the first test.

That was the big challenge: Having it skip results lines when the same pair would be more thouroughly tested with a subsequent line.

Can you demonstrate any cases where the term "probable" is ever output? And if so, is there another line for the same pair where it shows "is PRP" or "is Frobenius PRP" or "is prime!"?

I did a whole lot of teeny base 2 and base 3 prime tests and didn't encounter one. I also did medium and large-sized base 2 tests. I haven't done medium and larger base 3 tests after encountering the roundoff loop.

If you have time, you might run a bunch of small and medium base 2 and base 3 tests yourself and attempt to break the code.

You know where the applicable forums/threads are. Can you report the issue to Jean and co.?


Gary

[quote=gd_barnes;207056]I removed the phrase "probable" from the results conversion and changed it to "Frobenius PRP" because it appears the term "probable" or "probable prime" is no longer used with LLR version 3.8. Also, keep in mind that we only want to consider the result of the FINAL test of a pair. If we get "probable prime" followed by "Frobenius PRP" or simply "is prime!", then we have to ignore the first test.

That was the big challenge: Having it skip results lines when the same pair would be more thouroughly tested with a subsequent line.

Can you demonstrate any cases where the term "probable" is ever output? And if so, is there another line for the same pair where it shows "is PRP" or "is Frobenius PRP" or "is prime!"?

I did a whole lot of teeny base 2 and base 3 prime tests and didn't encounter one. I also did medium and large-sized base 2 tests. I haven't done medium and larger base 3 tests after encountering the roundoff loop.

If you have time, you might run a bunch of small and medium base 2 and base 3 tests yourself and attempt to break the code.[/quote]
From what I'm seeing here and from reading LLR's readme, it sounds like it does the following:

Base 2:
-LLR/Proth test if possible, producing a final result.
-If not (due to k>b^n or whatever), do a Fermat SPRP test (i.e., the "regular" PRP test we're used to), then a Frobenius PRP if that comes back positive. Presumably Frobenius is a stronger PRP test.

Other bases:
-N-1/N+1 test (final result)

The LLR/Proth or N-1/N+1 tests, respectively, can be bypassed and a PRP forced with the ForcePRP=1 option, which should make it do Fermat then Frobenius if positive.

At any rate, you're right, I don't believe 3.8 uses "probable prime" lingo any more. Now it just says "base 3-Strong Fermat PRP!" or "Frobenius PRP!" instead. Essentially "base 3-Strong Fermat PRP!" is the equivalent of 3.7.1c's "is probably prime" (since it just did the Fermat PRP test when appropriate).

The only reason why we'd possibly want "is probably prime" to be supported is in case somebody wanted to use 3.7.1c with the script. Not likely, I'll admit...probably not worth the bother.

[quote]You know where the applicable forums/threads are. Can yuo report the issue to Jean and co.?[/quote]
Okay, I'll poke around with it a bit more and then report my findings.

[quote=mdettweiler;207057]From what I'm seeing here and from reading LLR's readme, it sounds like it does the following:

Base 2:
-LLR/Proth test if possible, producing a final result.
-If not (due to k>b^n or whatever), do a Fermat SPRP test (i.e., the "regular" PRP test we're used to), then a Frobenius PRP if that comes back positive. Presumably Frobenius is a stronger PRP test.

Other bases:
-N-1/N+1 test (final result)

The LLR/Proth or N-1/N+1 tests, respectively, can be bypassed and a PRP forced with the ForcePRP=1 option, which should make it do Fermat then Frobenius if positive.

At any rate, you're right, I don't believe 3.8 uses "probable prime" lingo any more. Now it just says "base 3-Strong Fermat PRP!" or "Frobenius PRP!" instead. Essentially "base 3-Strong Fermat PRP!" is the equivalent of 3.7.1c's "is probably prime" (since it just did the Fermat PRP test when appropriate).

The only reason why we'd possibly want "is probably prime" to be supported is in case somebody wanted to use 3.7.1c with the script. Not likely, I'll admit...probably not worth the bother.


Okay, I'll poke around with it a bit more and then report my findings.[/quote]

I suppose that wouldn't hurt to add the term "probably prime". But is that really the term? I thought it was "probable prime". If you can confirm that for sure, I could put it in without testing it because I don't want to mess with testing 3.7.1.

But...on the other hand, this script is unlikely to work well at all with 3.7.1 because the tests done on small tests, PRPs, and primes of different bases are so much different. So using that argument, it isn't particularly helpful to add the probably prime lingo in there because the script would not work in many other situations.

They really did a lot of modifications for 3.8 on the smaller and PRP tests. I have to say they are excellent. There was just a big learning curve on them when coding the script.

I forgot to address one of the main things you brought up. You said that checking for "Frobenius PRP!" wouldn't hit other kinds of PRPs. But we can't just check for the more simple "PRP!" because that hits "Fermat PRP!" and as you stated, after a Fermat PRP, it does another better test for the "Frobenius PRP". We would end up causing the pair to be submitted twice causing a rejected pair and possibly an incorrect prime being reported if the Frobenius test comes back composite after the Fermat test came back as PRP.

I'm now testing medium and larger base 3 tests. All kinds of interesting output being put out by LLR on the medium sized base 3 primes. A lot of lines that show "may be prime," followed finally by "is prime!". Fortunately only the "is prime!" line shows up in the results. But even if it didn't, "may be prime," would not hit any of the results conditions and the line would be ignored as it should.

Whew, this is a lot more involved on testing than I had originally imagined. We can blame Karsten. Ha-ha. :smile:


Gary

Max and Karsten,

I thought it might be helpful if I showed the actual lresults_hist.txt files from a couple of my tests recently. They are attached. These particular tests really show how many different scenarios that we are dealing with and that I have coded around.

I think quite a few of the scenarios came up in the PMs that Karsten sent but I'm not sure if all of them did.


Gary

[quote=gd_barnes;207058]I suppose that wouldn't hurt to add the term "probably prime". But is that really the term? I thought it was "probable prime". If you can confirm that for sure, I could put it in without testing it because I don't want to mess with testing 3.7.1.[/quote]
Hmm, you're right:
[code]$ ./cllr371c.exe -d -q"4436*28^6242-1"
LLR tests only k*2^n+/-1 numbers, so, we will do a PRP test of 4436*28^6242-1
Starting probable prime test of 4436*28^6242-1
Using generic reduction FFT length 3072
4436*28^6242-1 is a probable prime. Time : 4.946 sec.
Please credit George Woltman's PRP for this result![/code]
Note that the "Please credit..." line is printed to lresults.txt as well; it would of course be skipped over by convertResults().

[quote]But...on the other hand, this script is unlikely to work well at all with 3.7.1 because the tests done on small tests, PRPs, and primes of different bases are so much different. So using that argument, it isn't particularly helpful to add the probably prime lingo in there because the script would not work in many other situations.[/quote]
Yeah, I'd suggest just putting in the probable prime lingo check as the last thing it checks--why not. :smile: Beyond that, though, if users want to use 3.7.1c or other pre-3.8 versions, they do so at their own risk.

[quote]They really did a lot of modifications for 3.8 on the smaller and PRP tests. I have to say they are excellent. There was just a big learning curve on them when coding the script.

I forgot to address one of the main things you brought up. You said that checking for "Frobenius PRP!" wouldn't hit other kinds of PRPs. But we can't just check for the more simple "PRP!" because that hits "Fermat PRP!" and as you stated, after a Fermat PRP, it does another better test for the "Frobenius PRP". We would end up causing the pair to be submitted twice causing a rejected pair and possibly an incorrect prime being reported if the Frobenius test comes back composite after the Fermat test came back as PRP.

I'm now testing medium and larger base 3 tests. All kinds of interesting output being put out by LLR on the medium sized base 3 primes. A lot of lines that show "may be prime," followed finally by "is prime!". Fortunately only the "is prime!" line shows up in the results. But even if it didn't, "may be prime," would not hit any of the results conditions and the line would be ignored as it should.[/quote]
Yeah, the "may be prime" messages are a result of how LLR 3.8 has to regularly re-do its N-1/N+1 tests at least once, as analyzed in great detail in the LLR 3.8 thread I started at CRUS. I'd forgotten that it put those in the lresults file too, though.

Speaking of which, I wonder how PRPnet handles those extra "may be prime" messages (not to mention all the Frobenius stuff)--I should check that when I get the chance. It obviously has some coding to avoid extraneous lines in results files since in my experience it hasn't had a problem cutting through various other dross (such as roundoff errors from PFGW or "non-base-2 number, doing PRP test" messages from LLR), so it will be interesting to see how it handles these.

And I thought we had -c working...
 

Just now I tried running do.pl -c on a queue of 2 completed and 3 incomplete workunits. Here's what happened:
-The 2 completed WUs were submitted correctly.
-The first two incomplete WUs were canceled correctly.
-The last incomplete WU was abandoned and the server still lists it as in progress.

Here's the console output I got from the script:
[code][2010-03-02 03:08:10]
Cancelling : 195/142804 (600000000000:M:1:2:258)
[2010-03-02 03:08:14]
Cancelling : 195/142825 (600000000000:M:1:2:258)
[2010-03-02 03:08:15]
Cancelling : 195/142826 (600000000000:M:1:2:258)[/code]
So, all three incomplete pairs apparently made it at least as far as the LLRnet client. The strange thing is why the last one didn't actually get canceled on the server. Any idea why this happened?

FYI, this was with the latest do.pl and llrnet.lua files.

[quote=mdettweiler;207070]Just now I tried running do.pl -c on a queue of 2 completed and 3 incomplete workunits. Here's what happened:
-The 2 completed WUs were submitted correctly.
-The first two incomplete WUs were canceled correctly.
-The last incomplete WU was abandoned and the server still lists it as in progress.

Here's the console output I got from the script:
[code][2010-03-02 03:08:10]
Cancelling : 195/142804 (600000000000:M:1:2:258)
[2010-03-02 03:08:14]
Cancelling : 195/142825 (600000000000:M:1:2:258)
[2010-03-02 03:08:15]
Cancelling : 195/142826 (600000000000:M:1:2:258)[/code]So, all three incomplete pairs apparently made it at least as far as the LLRnet client. The strange thing is why the last one didn't actually get canceled on the server. Any idea why this happened?

FYI, this was with the latest do.pl and llrnet.lua files.[/quote]


Hum. Very good catch. I had not closely inspected the joblist.txt when canceling pairs previously. My mistake there. I'm looking into it now.

This looks like the same type of problem that we were getting before where the final results of a dried server would just go out into cyberspace with nothing to show what happened to them. But I'll have to do some file prints to verify why it's happening.

Do you REALLY want to add "probable prime"? Is this script going to work in all circumstances for 3.7.1 and before? Keep in mind that we'll automatically put LLR 3.8 in the client that we send people or that we post in our threads.

Tell you what: If you can run some 3.7.1 tests for all n=1 to 1000 for a large k (perhaps the one that I did) and everything works except for the "probable prime" tests, I'll add it. The run takes around 3-5 mins. If there are other problems, then I think it's just added code for an LLR version that people should not be using for this release. The bottom line is that we should be able to tell people that it works for only 3.8 or that it works for 3.8 and 3.7.1 ALL of the time; not all of the time for one and some of the time for the other.


Gary