mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > PrimeNet

Reply
 
Thread Tools
Old 2017-02-19, 18:01   #56
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

216308 Posts
Default

Quote:
Originally Posted by Madpoo View Post
I'm not sure how that would have worked before though. The only change I made was redirecting http -> https on the website.
OK. I'm just reporting what was observed.

Again, when you announced the migration last month I added a regex to redirect any "http://" requests to be "https://", and it worked. Literally "$URL =~ s/http:/https:/;" was the only change to the proxy script.

This morning I observed the proxy wasn't working; removing this transform resulted in the proxy working again.

If it would help at all, I could provide you (privately) with the transaction logs just before and just after the proxy started seeing the 404 errors.

Edit: Just saw your edit (LOL)... As mentioned in my post above, everything my proxy sees is targeted to "http://v5.mersenne.org/v5server/". The $URL variable is derived from $ENV{"REQUEST_URI"} (in Perl).

Last fiddled with by chalsall on 2017-02-19 at 18:06
chalsall is offline   Reply With Quote
Old 2017-02-19, 18:42   #57
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

23·409 Posts
Default

Quote:
Originally Posted by chalsall View Post
OK. I'm just reporting what was observed.

Again, when you announced the migration last month I added a regex to redirect any "http://" requests to be "https://", and it worked. Literally "$URL =~ s/http:/https:/;" was the only change to the proxy script.

This morning I observed the proxy wasn't working; removing this transform resulted in the proxy working again.

If it would help at all, I could provide you (privately) with the transaction logs just before and just after the proxy started seeing the 404 errors.

Edit: Just saw your edit (LOL)... As mentioned in my post above, everything my proxy sees is targeted to "http://v5.mersenne.org/v5server/". The $URL variable is derived from $ENV{"REQUEST_URI"} (in Perl).
LOL... well, cool then. I should have been more explicit about this only affecting the website URL and not the API.

I poked around on the website and saw a few hits from the GPU_to_72 submission spider ... they were all POST to /account/default.php on HTTP and they were getting the 301 redirect to HTTPS in response, as expected, however I didn't see a followup POST to the https link... it's not resubmitting after the redirect as far as I could tell.

I also saw Misfit doing insecure POSTs to the manual result, Misfit-specific URL, and although I'm not seeing a follow up POST resubmittal to the secure URL, I do see a GET to the same URL securely. That's odd.

Unfortunately in the case of Misfit. the data it's sending should be encrypted from the start, and while it's great that it's resubmitting securely, the fact that it still tried going to HTTP first means the info in there is still out there bubbling around the intertubes in the clear.

I think he'd said Misfit would be updated to use SSL but I suppose then it's a matter of making sure that change is there and that anyone using it is updating to the latest/greatest?

For now I think I'll have to exclude the misfit specific URL from being redirected. The SSL will still work so the transition on the client side can happen over time.

TL;DR:
--------
I'm searching for any other POSTs to HTTP that don't seem to be handled correctly, but for now the ones I saw are GPU72 submitting something to /account/default.php, and then Misfit. I'll exclude the Misfit hits from redirecting but the GPU72 posts, to me anyway, may seem to be harmless if they're not re-posting? Leaving those alone for now, hoping for a quick fix from your spider?
Madpoo is offline   Reply With Quote
Old 2017-02-19, 19:24   #58
chalsall
If I May
 
chalsall's Avatar
 
"Chris Halsall"
Sep 2002
Barbados

23·17·67 Posts
Default

Quote:
Originally Posted by Madpoo View Post
I poked around on the website and saw a few hits from the GPU_to_72 submission spider ... they were all POST to /account/default.php on HTTP and they were getting the 301 redirect to HTTPS in response, as expected, however I didn't see a followup POST to the https link... it's not resubmitting after the redirect as far as I could tell.
That's a bit funny, informative, and enlightening... The "GPU_to_72 Submission Spider" UserAgent string is presented by a Perl script I wrote about three years ago or so and released under the GPL. It was written very quickly largely as a proof of concept.

I had no idea anyone was still using it! Much better tools are now available (MISFIT et al).

So you know, the GPU72 site's observation spiders are using SSL, and all come from the same IP address. There don't appear to be any warnings being issued by them after this morning's transition.

I will continue to have the GPU72 proxy not transform any HTTP requests to be HTTPS unless and until you ask for that to be done.

Are we having fun yet? (I am! Complexity is interesting.)
chalsall is offline   Reply With Quote
Old 2017-02-19, 22:00   #59
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

327210 Posts
Default

Quote:
Originally Posted by chalsall View Post
That's a bit funny, informative, and enlightening... The "GPU_to_72 Submission Spider" UserAgent string is presented by a Perl script I wrote about three years ago or so and released under the GPL. It was written very quickly largely as a proof of concept.

I had no idea anyone was still using it! Much better tools are now available (MISFIT et al).
Weird... yeah, someone is using it, but for whatever reason the only hits I saw coming from it so far today were POSTs to that /account/default.php

Meh... well, if it's not you then I'm inclined to ignore it then. People do weird things.

Okay, I think I have the redirect for those misfit hits disabled. I'm just tracking any other peculiar POSTs. I see some script kiddiez (I assume) posting to weird things, looking for holes no doubt. Punks.
Madpoo is offline   Reply With Quote
Old 2017-02-19, 22:32   #60
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

327210 Posts
Default

Quote:
Originally Posted by Madpoo View Post
Weird... yeah, someone is using it, but for whatever reason the only hits I saw coming from it so far today were POSTs to that /account/default.php
I think the person using that spider is AirSquirrels.
Madpoo is offline   Reply With Quote
Old 2017-02-20, 06:51   #61
Dubslow
Basketry That Evening!
 
Dubslow's Avatar
 
"Bunslow the Bold"
Jun 2011
40<A<43 -89<O<-88

160658 Posts
Default

Quote:
Originally Posted by Madpoo View Post
Those were working great, *except* if it was to the root directory. For whatever reason, (.+) does NOT match a path of nothing (although in truth the path is "/".

I ended up changing things around on both rules to do a negative match of cgi-bin/(.+) instead which achieves the same goal and makes that home page redirect as needed.
The + modifier matches 1 or more repetitions of the previous pattern.
The * modifier matches 0 or more repetitions of the previous pattern.
Code:
In [1]: import re

In [2]: re.search("(.+)", "thing")
Out[2]: <_sre.SRE_Match object; span=(0, 5), match='thing'>

In [3]: re.search("(.+)", "")

In [4]: re.search("(.*)", "thing")
Out[4]: <_sre.SRE_Match object; span=(0, 5), match='thing'>

In [5]: re.search("(.*)", "")
Out[5]: <_sre.SRE_Match object; span=(0, 0), match=''>

Last fiddled with by Dubslow on 2017-02-20 at 06:52 Reason: inb4 aaron knows this and I've completely missed the point
Dubslow is offline   Reply With Quote
Old 2017-02-21, 02:11   #62
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

23×409 Posts
Default

Quote:
Originally Posted by Dubslow View Post
The + modifier matches 1 or more repetitions of the previous pattern.
The * modifier matches 0 or more repetitions of the previous pattern.
Oh geez... I should have thought of that. Send me back to Regex 101...
Madpoo is offline   Reply With Quote
Old 2017-02-23, 21:59   #63
Brain
 
Brain's Avatar
 
Dec 2009
Peine, Germany

33110 Posts
Default

Quote:
Originally Posted by chalsall View Post
That's a bit funny, informative, and enlightening... The "GPU_to_72 Submission Spider" UserAgent string is presented by a Perl script I wrote about three years ago or so and released under the GPL. It was written very quickly largely as a proof of concept.

I had no idea anyone was still using it! Much better tools are now available (MISFIT et al).

So you know, the GPU72 site's observation spiders are using SSL, and all come from the same IP address. There don't appear to be any warnings being issued by them after this morning's transition.

I will continue to have the GPU72 proxy not transform any HTTP requests to be HTTPS unless and until you ask for that to be done.

Are we having fun yet? (I am! Complexity is interesting.)
I am still using it and it has broken:
Code:
20170223_215245 INFO:  Submission spider starting...
20170223_215245 INFO:  Attempting to log into PrimeNet.  This can take a little while...
20170223_215245 ERR :  Bad response:
301 Moved Permanently
Adding the "s" to http fixed it: my $PrimeNetURL = "https://www.mersenne.org/";

Could you please update version 0.24 on GPU72: http://www.gpu72.com/software/submit_spider

Tnx, Sebastian

Last fiddled with by Brain on 2017-02-23 at 22:00
Brain is offline   Reply With Quote
Old 2017-02-24, 03:45   #64
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

23×409 Posts
Default

Quote:
Originally Posted by Brain View Post
I am still using it and it has broken:
Code:
20170223_215245 INFO:  Submission spider starting...
20170223_215245 INFO:  Attempting to log into PrimeNet.  This can take a little while...
20170223_215245 ERR :  Bad response:
301 Moved Permanently
Adding the "s" to http fixed it: my $PrimeNetURL = "https://www.mersenne.org/";

Could you please update version 0.24 on GPU72: http://www.gpu72.com/software/submit_spider

Tnx, Sebastian
Sorry about that Brain.

After I got in touch with AirSquirrels (who was using it to auto-submit results), he switched it to "https" and it worked. I checked again later yesterday and saw someone else was also trying to submit to http and not having any luck but unfortunately I couldn't tell who it was (looking up the reverse IP didn't help at all).

I was hoping the unlucky soul would reach out for help or see this thread, so I'm glad you did. As far as I know, it was probably just the two of you using that, but now I'll check again and see if some other hits come in sporadically.

I think it may take a while for some people who are using it to make themselves known... it only tries when it has something to check in, I guess, but once it does, it'll retry every xx minutes since the POST to http would always fail.

Fortunately it seems like adding the "s" is all it needs and it'll submit whatever it needed to.
Madpoo is offline   Reply With Quote
Old 2017-02-24, 04:55   #65
Madpoo
Serpentine Vermin Jar
 
Madpoo's Avatar
 
Jul 2014

23·409 Posts
Default

Quote:
Originally Posted by Madpoo View Post
I think it may take a while for some people who are using it to make themselves known... it only tries when it has something to check in, I guess, but once it does, it'll retry every xx minutes since the POST to http would always fail.

Fortunately it seems like adding the "s" is all it needs and it'll submit whatever it needed to.
Actually it looks like the other person I noticed using that submission spider is our own "petrw1". I might be wrong but that's the best I could suss out.

I'll have to PM him and mention this thread.
Madpoo is offline   Reply With Quote
Old 2017-02-24, 06:57   #66
Brain
 
Brain's Avatar
 
Dec 2009
Peine, Germany

14B16 Posts
Default Tnx to Madpoo

Thanks a lot for all your work and improvements, finder of the great M49, ruler of the DB.
Brain is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Big milestone coming up schickel Aliquot Sequences 8 2011-07-29 10:54
Mersenne BOINC coming? frmky Software 27 2011-02-20 08:52
Dark times may be coming...? OmbooHankvald mersennewiki 10 2005-10-24 06:26
And the hits just keep on coming..... R.D. Silverman Factoring 13 2005-10-04 10:02
Coming to a DC project near you P4 2.4B/GA8SQ800 /pc3200 dragongoddess Hardware 0 2003-03-22 15:49

All times are UTC. The time now is 02:33.

Wed Aug 5 02:33:28 UTC 2020 up 18 days, 22:20, 1 user, load averages: 1.84, 1.77, 1.69

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.