Primenet Server - Official Maintenance Thread

Madpoo · 2016-10-11, 21:25

I'm starting a new thread to announce server maintenance tasks.

To kick things off, our collocation host has given us a 3-day window during which they will be physically moving our server from one place to another on the same floor.

That maintenance is scheduled to occur during any of these 3 windows:

Code:

Tue 2016-10-11 between 23:00 UTC and 07:00 UTC the following day
Wed 2016-10-12 between 23:00 UTC and 07:00 UTC the following day
Thu 2016-10-13 between 23:00 UTC and 07:00 UTC the following day

Unfortunately that's as specific as it gets, so I'm just expecting it on any of those days. The actual length of outage is estimated to be 60-100 minutes once they start.

Once the maintenance is complete I'll follow up so we know it's done.

Mark Rose · 2016-10-11, 22:35

Oooo.... server migration :D

Madpoo · 2016-10-12, 01:01

Quote:

Originally Posted by Mark Rose

Oooo.... server migration :D

Yeah, a real hoot and a holler.

Just got forwarded an email from George that apparently *tonight* is the night. I'm figuring on whether I should do a graceful shutdown ahead of time or have them hit the power button and let it save the virtual machine's state on it's own. Decisions decisions...

I'll be kicking off a fresh backup of the data and hope it syncs up to the cloud beforehand, at any rate. We'll see. It should all be good though.

snme2pm1 · 2016-10-12, 06:39

Quote:

Originally Posted by Madpoo

*tonight* is the night. I'm figuring on whether I should do a graceful shutdown ahead of time or have them hit the power button and let it save the virtual machine's state on it's own.

Hell of a way to test fail safe provisions!
My notion of "tonight" is somewhat different to yours I would expect.
mersenne.org seems quiet now...
Ed: Assuming decent arithmetic, UTC 07:02 apparently resumed sometime earlier. (I wasn't watching closely).

Madpoo · 2016-10-12, 14:20

Quote:

Originally Posted by snme2pm1

Hell of a way to test fail safe provisions!
My notion of "tonight" is somewhat different to yours I would expect.
mersenne.org seems quiet now...
Ed: Assuming decent arithmetic, UTC 07:02 apparently resumed sometime earlier. (I wasn't watching closely).

I got tired of waiting and went to bed... looks like it happened between 11:30 PM and midnight'ish in my timezone (PDT). (EDIT: 11:23 PM to 11:57 PM according to my Pingdom monitor)

It worked out pretty well. The virtual machine saved state and came back up okay. I'm now doing a reboot of the server since I don't always trust the saved-state as far as OS stability... when it works it works usually but I've had experiences with hung services or other oddities.

EDIT: And we're back up. Over all a pretty decent process. It helped that it was apparently just from one spot on the same floor to another. I've done entire cabinet/cage relocations and it's not fun.

Madpoo · 2016-12-22, 22:39

The Primenet server is going to get a reboot shortly. The online backup service had stalled out and it just bugged me about it, so to be sure I'm going to reboot the server. I've seen that happen before when the service (Carbonite in this case) updates the client software and although it doesn't say so, it's best to reboot it since files were in use, blah blah. Windows stuff.

It should be a quick thing... I'm also updating the AV and a few other odds and ends as long as I'm at it.

Time is now 22:38 UTC and I'm planning to hit the button around 22:40 UTC. Should only be about 5-10 minutes of downtime.

EDIT: Back online at about 22:50 UTC.

Madpoo · 2016-12-23, 16:08

I'm doing a quick SQL update on Primenet. It failed to apply automatically yesterday during the maintenance... I already knew I can never trust Windows Update to do SQL updates... it always seems to fail. Don't know why I thought it'd work yesterday either.

EDIT: Back online. Easy breezy.

Madpoo · 2017-01-18, 21:39

I added a certificate to Primenet and now I need to adjust the horrible default cipher suite that Microsoft uses. You'd think they'd disable things like sslv3 in some security hotfix, but I guess they're afraid that some website out there would freak out if that happened.

I'm shooting for an "A" score on ssl labs... we'll see.

Outage will begin now and should only last a few minutes.

Madpoo · 2017-01-18, 21:44

Quote:

Originally Posted by Madpoo

...
I'm shooting for an "A" score on ssl labs... we'll see.
...

www.mersenne.org SSL test

Dubslow · 2017-01-19, 07:27

Quote:

Originally Posted by Madpoo

www.mersenne.org SSL test

Is it possible to force all incoming connections to https by default?

S485122 · 2017-01-19, 20:27

Quote:

Originally Posted by Dubslow

Is it possible to force all incoming connections to https by default?

Yes it is. But for the moment HTTPS is in test phase (I do use it and it works perfectly for me.)

2016-10-11, 21:25	#1
Madpoo Serpentine Vermin Jar Jul 2014 5·677 Posts	Primenet Server - Official Maintenance Thread I'm starting a new thread to announce server maintenance tasks. To kick things off, our collocation host has given us a 3-day window during which they will be physically moving our server from one place to another on the same floor. That maintenance is scheduled to occur during any of these 3 windows: Code: Tue 2016-10-11 between 23:00 UTC and 07:00 UTC the following day Wed 2016-10-12 between 23:00 UTC and 07:00 UTC the following day Thu 2016-10-13 between 23:00 UTC and 07:00 UTC the following day Unfortunately that's as specific as it gets, so I'm just expecting it on any of those days. The actual length of outage is estimated to be 60-100 minutes once they start. Once the maintenance is complete I'll follow up so we know it's done.

2016-12-22, 22:39	#6
Madpoo Serpentine Vermin Jar Jul 2014 6471₈ Posts	Server reboot coming up The Primenet server is going to get a reboot shortly. The online backup service had stalled out and it just bugged me about it, so to be sure I'm going to reboot the server. I've seen that happen before when the service (Carbonite in this case) updates the client software and although it doesn't say so, it's best to reboot it since files were in use, blah blah. Windows stuff. It should be a quick thing... I'm also updating the AV and a few other odds and ends as long as I'm at it. Time is now 22:38 UTC and I'm planning to hit the button around 22:40 UTC. Should only be about 5-10 minutes of downtime. EDIT: Back online at about 22:50 UTC. Last fiddled with by Madpoo on 2016-12-22 at 22:55

2016-12-23, 16:08	#7
Madpoo Serpentine Vermin Jar Jul 2014 5×677 Posts	Minor SQL update occurring now I'm doing a quick SQL update on Primenet. It failed to apply automatically yesterday during the maintenance... I already knew I can never trust Windows Update to do SQL updates... it always seems to fail. Don't know why I thought it'd work yesterday either. EDIT: Back online. Easy breezy. Last fiddled with by Madpoo on 2016-12-23 at 16:15

2017-01-18, 21:39	#8
Madpoo Serpentine Vermin Jar Jul 2014 5·677 Posts	Quick server reboot coming up - prepping for SSL I added a certificate to Primenet and now I need to adjust the horrible default cipher suite that Microsoft uses. You'd think they'd disable things like sslv3 in some security hotfix, but I guess they're afraid that some website out there would freak out if that happened. I'm shooting for an "A" score on ssl labs... we'll see. Outage will begin now and should only last a few minutes.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
OFFICIAL "SERVER PROBLEMS" THREAD	ewmayer	PrimeNet	2873	2023-01-30 01:14
Official AVX-512 programming thread	ewmayer	Programming	31	2016-10-14 05:49
Primenet maintenance announcements	Madpoo	PrimeNet	7	2015-11-12 05:50
Official Peeved Pets Thread	Prime95	Lounge	32	2015-10-02 04:17
Server maintenance thread	Madpoo	PrimeNet	9	2015-07-25 07:47

2016-10-11, 22:35	#2
Mark Rose "/X\(‘-‘)/X\" Jan 2013 3²·7³ Posts	Oooo.... server migration :D