OFFICIAL "SERVER PROBLEMS" THREAD

[Prime95]

I'll let madpoo render a verdict. I did not see anything in the PHP logs, madpoo knows more places to look for evidence.

[chalsall]

Originally Posted by Prime95

I'll let madpoo render a verdict. I did not see anything in the PHP logs, madpoo knows more places to look for evidence.

OK... Happy to provide additional data.

20140911_221714 ERR :  Error from Network. Message: "500 read timeout"
20140911_221714 INFO:  100: 77003071,74,0 (5EB14E057AC239FC6FAC60A52E09FC76) -- Keep: 0
20140911_221714 INFO:  100: 77003089,74,0 (F8EA35171419A456EB6A8CFB9C3B4B13) -- Keep: 0
20140911_221715 INFO:  100: 77003281,74,0 (FA81467FDA9A0EFDA212F09FA2EA8438) -- Keep: 0
20140911_221715 INFO:  100: 77003383,74,0 (1A6931D8D909D8055A44D1F05B106EB0) -- Keep: 0
20140911_221715 INFO:  100: 77003389,74,0 (974DD223DA94D833EB2A82078A1E4CF2) -- Keep: 0
20140911_221715 INFO:  100: 77003429,74,0 (1F82E3F6C1A9250C6843AF41EF5573ED) -- Keep: 0
20140911_221715 INFO:  Spider has finished.  Exiting.


20140911_223002 INFO:  Get Manual Work spider starting...
20140911_223003 INFO:  100: 67397093,74,0 (55FE20CE3DBD0498B4F4DD8A105B78D1) -- Keep: 0
20140911_223003 INFO:  100: 67397173,74,0 (957FD57F810D316AA6FBB58404D11154) -- Keep: 0
20140911_223003 INFO:  100: 67397203,74,0 (BC8B07D85C0AEDA1CDC37FD6FC4F8632) -- Keep: 0
20140911_223003 INFO:  100: 67397527,74,0 (C0F81B5653280EBAACDDADA04D170ED4) -- Keep: 0
20140911_223003 INFO:  100: 67397581,74,0 (42548F86FA4818B8CB91AEF57CBDE5C4) -- Keep: 0
20140911_223003 INFO:  100: 67397599,74,0 (063794E6F371632F15870A979EF732B1) -- Keep: 0
20140911_223003 INFO:  100: 67397611,74,0 (B66DBCD1924D88035FB1A904E3FF4133) -- Keep: 0
20140911_223004 INFO:  100: 67397623,74,0 (6502753AA0366EF5317B7B503A4DE043) -- Keep: 0
20140911_223004 INFO:  100: 67397777,74,0 (CA885A1E502909D62B47FDF602B91E82) -- Keep: 0
20140911_223004 INFO:  100: 67397879,74,0 (A3FB956BB5663B88C7BCADC74C741E9B) -- Keep: 0
20140911_223004 INFO:  Category 3...
20140911_223004 INFO:  100: 64262729,74,1 (FCD4A161C2919226C1DCAD57DA7D14B4) -- Keep: 0
20140911_223004 INFO:  100: 64262893,74,1 (ADB390B47BC5B31245B52EB96EF2C436) -- Keep: 0
20140911_223204 ERR :  Bad response:
500 read timeout

20140911_223204 ERR :  Error from Network. Message: "500 read timeout"
20140911_223204 INFO:  100: 64262917,74,1 (4129E70FAE1AD408CA497C3AFCB7BC87) -- Keep: 0
20140911_223204 INFO:  100: 64262953,74,1 (7DB2832AA03D02911855437B8A18C58C) -- Keep: 0
20140911_223404 ERR :  Bad response:
500 read timeout

20140911_223404 ERR :  Error from Network. Message: "500 read timeout"
20140911_223405 INFO:  100: 64262981,74,1 (7842FA7F68D3117DA13153E7C258C163) -- Keep: 0
20140911_223405 INFO:  100: 64263593,74,1 (EBB6AE72DB86E8A231573C19D77785FA) -- Keep: 0
20140911_223405 INFO:  Category 4...
20140911_223405 INFO:  100: 70109747,73,0 (FA04968641DF730781E733CD87617E00) -- Keep: 0
20140911_223405 INFO:  100: 70109749,73,0 (B4FC385769FDFE3AE9639F1BA46CC4E2) -- Keep: 0
20140911_223405 INFO:  100: 70109777,73,0 (B9F8EA48B510115645791AA2BAB56DA5) -- Keep: 0
20140911_223405 INFO:  100: 70109821,73,0 (5D929F0408DFAD2AF65860C324F2E89A) -- Keep: 0
20140911_223605 ERR :  Bad response:
500 read timeout

[Madpoo]

Originally Posted by tha

The graph of the PrimeNet activity has become remarkably flat at 160 TFlops for the past 28 hours.

Yeah, that was from me doing something and keeping those hourly stats from running for a while. The graph is filling in with a flat line during those missing hours. It'll clear itself up in time.

[Madpoo]

Originally Posted by chalsall

OK... Happy to provide additional data.

20140911_221714 ERR :  Error from Network. Message: "500 read timeout"...

What timezone would those timestamps correlate with? I can try to match it up with server logs to see if there was anything going on.

FYI, the server is set to run in UTC, just to keep it all standard no matter it's location and without the hassles of converting when reading in different apps that handle local time conversions differently.

EDIT: I'm assuming your times are also UTC.

All I can see in the IIS logs are the hits coming from your system around the times you mentioned. The ones being logged were all status 200 and the time-taken field are all pretty normal, in the 300-400 millisecond range. There are some hits to the "/report_factoring_effort/" page that do take longer on average, like 7.5 seconds. That might be where the server is doing some factor checking on reported results? Not sure.

Nothing looks funny in the IIS or PHP logs around those times though. There's some of the typical PHP warnings and info that we've been seeing due to increased log levels, but I don't know for sure that any of them are related since they don't match those particular time periods and are from other parts of the website in most cases. Just more fun things to eventually crack down on. The PHP is a little loosey goosey with some old syntax, so it does tend to throw a lot of warnings about deprecated syntax. Kind of a chore to go through.

Let me know if the issue is recurring and I'll look some more.

In regards to some of the specific exponents from your own logs, for example, 67397093. I see that exponent is assigned out to 2 people, but the first expired on July 8 and the second assigment for it just expired on Sep 7, but it hasn't been assigned to anyone else yet.

Whatever happened must have kept your spider from getting those assignments, I gather.

500 errors (the "500 read timeout" from your snippet) seems to imply it timed out on a connection to the server itself. I don't know how the manual assignment really works, like if it's possible to start getting assignments back but then if there's a server error/timeout, it won't actually reserve them for you?

Nothing on the server looks out of place. Network, CPU, memory, disk space, etc. have all been pretty normal today, and aside from me tinkering with an update to Google charts on a test page, I haven't done anything else on there, and I don't believe George or James have either, so that wouldn't explain it.

More info on what I see is that at 22:17 between :14 and :15 there are 3 hits to the server from your spider:

2014-09-11 22:17:14	/manual_assignment/	cores=1&num_to_get=2&pref=100&exp_lo=67675370&exp_hi=&B1=Get+Assignments/
2014-09-11 22:17:14	/manual_assignment/	cores=1&num_to_get=2&pref=100&exp_lo=67675370&exp_hi=&B1=Get+Assignments/
2014-09-11 22:17:15	/manual_assignment/	cores=1&num_to_get=2&pref=100&exp_lo=67675370&exp_hi=&B1=Get+Assignments/

[NBtarheel_33]

Barbados time = US EDT = UTC-4 if that is indeed the time zone to which his servers are synced. So 2217 would actually be 0217 GMT on 2014-09-12.

[error]

Manual results form does not recognize ECM factor found. It responds with

Notice: Undefined variable: b2 in C:\inetpub\www\manual_result\manual_result.inc.php on line 282
Error: Missing checksum. Correct the problem or email results to woltman at etc

[chalsall]

Originally Posted by Madpoo

FYI, the server is set to run in UTC, just to keep it all standard no matter it's location and without the hassles of converting when reading in different apps that handle local time conversions differently.

Yup, ditto GPU72; UTC kept in sync via NTP.

Originally Posted by Madpoo

Let me know if the issue is recurring and I'll look some more.

Seems to be fine again since early this morning.

Originally Posted by Madpoo

500 errors (the "500 read timeout" from your snippet) seems to imply it timed out on a connection to the server itself.

If you're not seeing any related errors in the logs, then it definitely could have been a connectivity issue between the two servers. Although my spiders were seeing the issue from 1and1 (in the eastern US of A), as well as myself from my workstation (Barbados).

Anyway, thanks for drilling down. Will keep an eye on it, and let you know if I see anything more (and try to get deeper info on the connectivity situation during the event).

[James Heinrich]

Originally Posted by error

Manual results form does not recognize ECM factor found.

Fixed.

[ET_]

Originally Posted by James Heinrich

Fixed.

Is there a way to address the issue for GMP-ECM results files having B2 values other than 100*B1?

Luigi

[James Heinrich]

Originally Posted by ET_

Is there a way to address the issue for GMP-ECM results files having B2 values other than 100*B1?

Probably, but I'm afraid I don't understand the question. Can you give me an example of what you mean?

[error]

Originally Posted by error http://www.mersenneforum.org/images/...s/viewpost.gif
Manual results form does not recognize ECM factor found.

Originally Posted by James Heinrich

Fixed.

Thanks, but ...
still complains about the checksum and the factor is not accepted.