PRPNet 5.5 Released

[rogue]

Create a .bat file (on Windows) and put this line into that .bat file. Specify the .bat file in prpclient.ini.

I believe that is how it was done by others.

[gd_barnes]

Originally Posted by rogue

Create a .bat file (on Windows) and put this line into that .bat file. Specify the .bat file in prpclient.ini.

I believe that is how it was done by others.

That's what I've done but it seems cumbersome. Your statement:

If there is a command line option to llr to generate different residues, then specify that in prpclient.ini.
In other words instead of "llr.exe" use "llr.exe -xxx" where xxx is the command line option.

seems to imply that it could be put directly into the prpclient.ini file.

I had hoped that option had been added. I guess I was mistaken so I'll continue using the .bat file.

[rogue]

Originally Posted by gd_barnes

That's what I've done but it seems cumbersome. Your statement:

If there is a command line option to llr to generate different residues, then specify that in prpclient.ini.
In other words instead of "llr.exe" use "llr.exe -xxx" where xxx is the command line option.

seems to imply that it could be put directly into the prpclient.ini file.

I had hoped that option had been added. I guess I was mistaken so I'll continue using the .bat file.

I will look into make a change in the client to handle this.

[gd_barnes]

Originally Posted by rogue

I will look into make a change in the client to handle this.

That would be excellent. Thanks!

[lalera]

hi,
https://www.mersenneforum.org/showpo...&postcount=142

[mdettweiler]

Hi Mark,

I just upgraded a couple of Gary and I's personal PRPNet servers from v5.3.2 to the latest SVN head from Sourceforge (r107, which bumped the version number to 5.4.7 on account of some client-only changes but is equivalent to the officially-released 5.4.6 as far as prpserver is concerned), and we noticed an interesting little SQL issue when attempting to sort by base (b) in Sierpinski/Riesel searches.

Specifically, Gary loaded a server with three one-k CRUS bases, and set sortoption=n,b in prpserver.ini. When clients connected to request work, the server would produce error messages like this on its console:

[2023-03-23 05:59:05 CDT] 5: ODBC Information: SQL_ERROR: [MySQL][ODBC 3.51 Driver][mysqld-5.0.75-0ubuntu10.5]Column 'b' in order clause is ambiguous
[2023-03-23 05:59:05 CDT] 5: ODBC Information: SQL Statement: select c.CandidateName, c.k, c.b, c.c, c.n   from Candidate c, CandidateGroupStats cgs where c.HasPendingTest = 0    and c.CompletedTests = 0    and c.DecimalLength > 0    and c.k = cgs.k    and c.b = cgs.b    and c.c = cgs.c    and (c.n < cgs.SierpinskiRieselPrimeN or cgs.SierpinskiRieselPrimeN = 0) order by n,abs(b) limit 500

Gary tried a few variations on the SQL statement directly in the MySQL console and confirmed that abs(b) is the problem: it worked fine if he just used b instead. (I'm guessing MySQL is deciding it can be more forgiving there since the bare column names have been rendered unambiguous by the earlier equality constraints, e.g. "c.b = cgs.b".)

I identified the relevant source code change as having been in SVN r38 (between releases 5.3.2 and 5.4.0), which tweaked the sorting code as part of adding support for generic and cyclotomic searches for PRPNet 5.4.0. The references to "b" and "c" in the sorting clause were changed to "abs(b)" and "abs(c)" respectively (prpserver.c lines 651 and 660 in the current SVN HEAD, r107).

Changing those two lines to "abs(c.b)" and "abs(c.c)", respectively, seems to have done the trick and rendered the SQL query unambiguous. With this fix, Gary's server is now working properly in sortoption=n,b mode.

(Probably, "order by c.n,abs(c.b)" would be the ideal syntax here as it would completely remove any potential ambiguity; but "order by n,abs(c.b)" seems to be working well enough given that, as noted above, MySQL seems to have collapsed the identically-named columns as a result of the equality constraints.)

I should note that the NPLB server we're running this on is living with positively ancient software, i.e. its MySQL version is very old (about a decade); so it may be that this issue didn't turn up in previous testing/usage because newer MySQLs might've been A-OK with it. That said, sorting by "b" or "c" is a relatively niche use case, so I can believe that we may be the first ones to actually try it since the 5.4.0 release.

[rogue]

Originally Posted by mdettweiler

Hi Mark,

I just upgraded a couple of Gary and I's personal PRPNet servers from v5.3.2 to the latest SVN head from Sourceforge (r107, which bumped the version number to 5.4.7 on account of some client-only changes but is equivalent to the officially-released 5.4.6 as far as prpserver is concerned), and we noticed an interesting little SQL issue when attempting to sort by base (b) in Sierpinski/Riesel searches.

Specifically, Gary loaded a server with three one-k CRUS bases, and set sortoption=n,b in prpserver.ini. When clients connected to request work, the server would produce error messages like this on its console:

[2023-03-23 05:59:05 CDT] 5: ODBC Information: SQL_ERROR: [MySQL][ODBC 3.51 Driver][mysqld-5.0.75-0ubuntu10.5]Column 'b' in order clause is ambiguous
[2023-03-23 05:59:05 CDT] 5: ODBC Information: SQL Statement: select c.CandidateName, c.k, c.b, c.c, c.n   from Candidate c, CandidateGroupStats cgs where c.HasPendingTest = 0    and c.CompletedTests = 0    and c.DecimalLength > 0    and c.k = cgs.k    and c.b = cgs.b    and c.c = cgs.c    and (c.n < cgs.SierpinskiRieselPrimeN or cgs.SierpinskiRieselPrimeN = 0) order by n,abs(b) limit 500

Gary tried a few variations on the SQL statement directly in the MySQL console and confirmed that abs(b) is the problem: it worked fine if he just used b instead. (I'm guessing MySQL is deciding it can be more forgiving there since the bare column names have been rendered unambiguous by the earlier equality constraints, e.g. "c.b = cgs.b".)

I identified the relevant source code change as having been in SVN r38 (between releases 5.3.2 and 5.4.0), which tweaked the sorting code as part of adding support for generic and cyclotomic searches for PRPNet 5.4.0. The references to "b" and "c" in the sorting clause were changed to "abs(b)" and "abs(c)" respectively (prpserver.c lines 651 and 660 in the current SVN HEAD, r107).

Changing those two lines to "abs(c.b)" and "abs(c.c)", respectively, seems to have done the trick and rendered the SQL query unambiguous. With this fix, Gary's server is now working properly in sortoption=n,b mode.

(Probably, "order by c.n,abs(c.b)" would be the ideal syntax here as it would completely remove any potential ambiguity; but "order by n,abs(c.b)" seems to be working well enough given that, as noted above, MySQL seems to have collapsed the identically-named columns as a result of the equality constraints.)

I should note that the NPLB server we're running this on is living with positively ancient software, i.e. its MySQL version is very old (about a decade); so it may be that this issue didn't turn up in previous testing/usage because newer MySQLs might've been A-OK with it. That said, sorting by "b" or "c" is a relatively niche use case, so I can believe that we may be the first ones to actually try it since the 5.4.0 release.

Thanks for the bug report. I hadn't used "b" in the sorting, so I hadn't seen it. This actually breaks others servers. I need to make other changes in preserver (using c. on both n and k) and to add the table alias in PrimeWorkSender.cpp in the same function that triggered this issue. I will do some testing and update sourceforge once I know it works.

[mdettweiler]

Originally Posted by rogue

Thanks for the bug report. I hadn't used "b" in the sorting, so I hadn't seen it. This actually breaks others servers. I need to make other changes in preserver (using c. on both n and k) and to add the table alias in PrimeWorkSender.cpp in the same function that triggered this issue. I will do some testing and update sourceforge once I know it works.

Ah yes, I see what you mean. PrimeWorkSender::SelectCandidates() is plugging the order-by constraint into a different select statement in the Sierpinski/Riesel case than in the generic k*b^n+c case, but only the S/R case provides (and needs) the "c." table alias (since the generic case doesn't need to involve the CandidateGroupStats table in the first place). So the change I made would fix the S/R case but break the more common case - whoops!

Glad you pointed this out before I attempted to use my "fixed" build at NPLB! (There's no urgency for me to upgrade the NPLB servers, since 5.3.2 has everything we need for non-conjecture searches, but I'll probably upgrade them all in the near future to avoid technical-debt pileup...)

[mdettweiler]

Gary and I just noticed another quirk of the new server version: it doesn't refresh the stats on the web pages as often as it used to. Looking at the code, I see this was a change in r104 (5.4.5):

--- a/source/prpserver.cpp
+++ b/source/prpserver.cpp
@@ -256,7 +256,7 @@
          ReprocessINIFile("prpserver.ini");
 
       // Send e-mail every 1024 seconds (if new primes/PRPs have been reported)
-      if (!(counter & 0x3ff))
+      if (!(counter & 0x3fff))
       {
          timerDBInterface->Connect(3);

Since r104, the stats are updated (and emails sent, etc.) every 16,384 seconds (~4.5 hours) instead of every 1,024 seconds (~17 minutes). Since the comment wasn't changed along with the bitmask, I'm wondering whether this was intentional.

Given that r104 introduced the recent improvements to Sierpinski/Riesel primed-k handling (such that the server now immediately stops handing out tests above a prime on an S/R k, rather than waiting until the next stats refresh to notice this), I can see how it might be less crucial now that the stats be updated frequently. Was there a concern that some heavier-loaded servers might not have time to complete a stats update in the original 17-minute timeframe?

For our own internal usage, I've rebuilt the server with the old bitmask (0x3ff i.e. 1024 seconds), since we liked having the stats more up-to-date and our candidate loads are small enough for the refresh performance to not be an issue. I suppose ideally, this would be a configurable setting in prpserver.ini; but changing the hardcoded value is working well enough for us in the meantime, so this wouldn't be high priority.

[rogue]

Originally Posted by mdettweiler

Gary and I just noticed another quirk of the new server version: it doesn't refresh the stats on the web pages as often as it used to. Looking at the code, I see this was a change in r104 (5.4.5):

--- a/source/prpserver.cpp
+++ b/source/prpserver.cpp
@@ -256,7 +256,7 @@
          ReprocessINIFile("prpserver.ini");
 
       // Send e-mail every 1024 seconds (if new primes/PRPs have been reported)
-      if (!(counter & 0x3ff))
+      if (!(counter & 0x3fff))
       {
          timerDBInterface->Connect(3);

Since r104, the stats are updated (and emails sent, etc.) every 16,384 seconds (~4.5 hours) instead of every 1,024 seconds (~17 minutes). Since the comment wasn't changed along with the bitmask, I'm wondering whether this was intentional.

Given that r104 introduced the recent improvements to Sierpinski/Riesel primed-k handling (such that the server now immediately stops handing out tests above a prime on an S/R k, rather than waiting until the next stats refresh to notice this), I can see how it might be less crucial now that the stats be updated frequently. Was there a concern that some heavier-loaded servers might not have time to complete a stats update in the original 17-minute timeframe?

For our own internal usage, I've rebuilt the server with the old bitmask (0x3ff i.e. 1024 seconds), since we liked having the stats more up-to-date and our candidate loads are small enough for the refresh performance to not be an issue. I suppose ideally, this would be a configurable setting in prpserver.ini; but changing the hardcoded value is working well enough for us in the meantime, so this wouldn't be high priority.

I did this because I was running S3 and had many thousands of sequences in the server.

The proper solution would be a configuration setting in prpserver.ini to specify how frequently to do it.

[gd_barnes]

Originally Posted by rogue

I did this because I was running S3 and had many thousands of sequences in the server.

The proper solution would be a configuration setting in prpserver.ini to specify how frequently to do it.

A configuration setting would be great! When running PRPnet, I'm often running just a few k-values with tests that take from 15 mins. to 1-2 hours so I'd probably set it to update every 5 mins. I like seeing it up to date. For the public servers at NPLB, updates every 15-20 minutes are good.