OFFICIAL "SERVER PROBLEMS" THREAD

[Madpoo]

Originally Posted by chalsall

A couple of quick things...

First... While I support presenting this knowledge in a proper HTML rendering, I think some additional UX work might be needed. For example, the previous report had visual breaks every ten (10#) million ranges, while this new report is just one long 1,000-row table.

If I may please reference the Low Composite Candidate Cleanup report that came out of a great deal of collaboration during Wayne's project. Also, James' Mersenne.ca site is a case study in how to present complex data using contemporary HTML / CSS.

Secondly... Yes, this is going to break a couple of my spiders. I /think/ there is now a proper API to get this type of meta-data from the Primenet server. Correct? Could you please point me to the threads discussing this, and I'll delta my code paths.

One of the great things about hanging out here is seeing all the great things people do!

No promises on how long this will be available, but the old format will still be available with this link:
GIMPS Summary - TEXT

James does good work with his site and he's been amazing at improving the Primenet pages as well. I'm just a hack fixing up the pages here and there. LOL What looks good to me tends to be more utilitarian than pretty. My own brother gave me a hard time back when I first did the redesign of the Primenet site... no respect.

The work on redesigning this particular page was more from a standpoint of cleaning up the SQL query that generates the page. I'd done the same work on other report pages but this particular one was so ugly that I put a pin in it 4 or 5 years ago, and finally decided to dive into it since I had a week vacation to burn.

[chalsall]

Originally Posted by Madpoo

... and finally decided to dive into it since I had a week vacation to burn.

Understood. But... Change management is a thing... 9^)

[lycorn]

Originally Posted by Madpoo

or maybe they had a custom crawler setup that would parse it all out and gather data...

For those folks, I apologize in advance.

Guilty.
Apologies accepted . Back to the drawing board...

[Madpoo]

Originally Posted by chalsall

Understood. But... Change management is a thing... 9^)

Change whatnow?

I updated the table to replace zero with a blank.

As far as how the old text page would add a blank line every 10M range (so it would have 10 rows, then a blank, then 10 more)... I guess that would make more sense for a text-based table. Kind of arbitrary though and I guess just serves to break it up visually.

It would also generate a new "header" section every 50M because it was trying to keep that somewhere on the page at any given time, wherever you happened to scroll to. Hopefully the floating header row on the new table is a better way of doing that.

I do plan to add some tooltips to the header row so that if you hover over it, it'll have a description of what the columns are. The old page has a legend of sorts at the very bottom.

And it might make sense to have the column sections visually distinct. They're broken down into 5 basic groups:
- Exponent range (start/end) and # of known primes
- Composite (either verified LL/PRP or factored)
- Unknowns (single tests or untested)
- Assigned
- Available

Right now they all kind of blend together, whereas the text format had pipes as a bit of a marker between sections.

Those are all spit-and-polish types of things though and hopefully won't be an issue to tweak. Just fixing up the report generation in the first place and optimizing it was a larger undertaking. Without going into too much detail, let's just say that the way the old report was generated (and still is, since it's still spitting out that text format hourly) was using SQL cursors. We recently ran some analysis of our inefficient SQL queries and the cursors sprinkled here and there were significantly slower.

The rewrite means the HTML data generates in a fraction of the time which I'm pretty happy about, so you should expect to see the old text based output disappear at some point since that was also a main factor.

Bear in mind that these reports can take a while to generate. The server it's running on was pretty fresh at the time we set it up but it's now starting to show some age, and we've significantly grown the amount of data stored on it since then, with JSON results now, the PRP proof handling, etc. so trying to optimize our SQL is crucial.

On the other hand, we are trying to add some interesting new things and do some data cleanup. James has been working hard on a project to start showing the application name/version in the exponent reports. Part of that has meant going through hundreds of millions of old records and assigning them to the correct application lookup values, where maybe we'd used a generic one when the result came in.

Just things like that, having accurate version info on results may make it easier for researchers to look at bad results and see that some app and version had buggy results more often, or whatever.

[harlee]

Want to report an issue with the server not accepting P-1 results with a factor found. This is happening on two different systems running 30.8 build 15:

[Work thread Dec 3 14:01] P-1 found a factor in stage #2, B1=320000, B2=243437040.
[Work thread Dec 3 14:01] M2935157 has a factor: 6538660032507417043406897 (P-1, B1=320000, B2=243437040)
[Comm thread Dec 3 14:01] Sending result to server: UID: harlee/i5-5250U_1600, M2935157 has a factor: 6538660032507417043406897 (P-1, B1=320000, B2=243437040)
[Comm thread Dec 3 14:01]
[Comm thread Dec 3 14:01] PnErrorResult value missing. Full response was:
[Comm thread Dec 3 14:01] [Dec 3 14:01] Visit http://mersenneforum.org for help.
[Comm thread Dec 3 14:01] Will try contacting server again in 70 minutes.

[James Heinrich]

Originally Posted by harlee

Want to report an issue with the server not accepting P-1 results with a factor found

Thanks, sorry, that was my fault. I fixed a different issue and in doing so made this broken. I think it should be working properly now. Prime95 should auto-retry after [70] minutes and continue as normal.

[harlee]

James,

Thanks, the server has accepted results from both clients.

[chalsall]

Originally Posted by Madpoo

The rewrite means the HTML data generates in a fraction of the time which I'm pretty happy about, so you should expect to see the old text based output disappear at some point since that was also a main factor.

Bear in mind that these reports can take a while to generate. The server it's running on was pretty fresh at the time we set it up but it's now starting to show some age, and we've significantly grown the amount of data stored on it since then, with JSON results now, the PRP proof handling, etc. so trying to optimize our SQL is crucial.

Aaron... I totally understand.

And we all deeply appreciate the amount of work you do to maintain the hub of the GIMPS infrastructure. And on Winblows, no less! 9^)

One suggestion, if I may...

It shouldn't cost much additional compute to generate a raw CSV/TSV file at the same time you are generating the human-facing HTML. This could then be made available to those weird people who write code to fetch and parse this data. Once an hour. And this "data product" is orthoginal to the human facing HTML code, which will likely change over time.

To share... I have found that Pointy-haired Bosses tend to specify JSON IPC for even simple vectors like this. And then they can't understand why they can't import the dataset into Excel... 8^)

[James Heinrich]

Originally Posted by chalsall

It shouldn't cost much additional compute to generate a raw CSV/TSV file at the same time you are generating the human-facing HTML.

The text version is currently still being generated in the same format (at the same URL) that it has been for decades, and I would like to request that this be maintained in the same place in the same format for legacy reasons (I know I'm not the only one spidering it). I have been archiving a copy of this file daily since 2012-Feb. If you want to write additional output in HTML, CSV, JSON, whatever, that's great too, give the consumer choice of input suitable to their needs.

[Reboot It]

Originally Posted by James Heinrich

Thanks, sorry, that was my fault. I fixed a different issue and in doing so made this broken. I think it should be working properly now. Prime95 should auto-retry after [70] minutes and continue as normal.

I also had a result that failed to upload (factor found in Stage 1 P-1 of M70434877) at around 18:08 GMT, with the same error message.

Prime95 did retry every 70 minutes several times, and while that factor did eventually upload at 21:38, the message returned was:

[Comm thread Dec 3 21:38:42] PrimeNet success code with additional info:
[Comm thread Dec 3 21:38:42] Already have factor 9261248336162888422114297 for M70434877
[Comm thread Dec 3 21:38:42] CPU credit is 15.5069 GHz-days.

However, it appears I have not received the credit for that P-1 work. Are you able to add the credit, or is this an issue for George?

[James Heinrich]

Originally Posted by Reboot It

Are you able to add the credit, or is this an issue for George?

I do not have access to edit user credit, you'll need to talk to George.