Number of forum active members - Page 2

CRGreathouse · 2009-07-14, 18:26

Quote:

Originally Posted by retina

Oh.I guess the security policies are quite strict here.

More's the pity. Can we have that one changed, at least to allow archive.org?

Xyzzy · 2009-07-14, 21:21

The robots.txt file, we think, has been fixed. Or we may have shut off access to the forum to everyone in Romania. Who knows?

cheesehead · 2009-07-14, 23:16

I don't suppose the Wayback Machine can retrieve those past pages now ...

or is it simply

"They're gone, Jim."

CRGreathouse · 2009-07-14, 23:41

Quote:

Originally Posted by cheesehead

I don't suppose the Wayback Machine can retrieve those past pages now ...

It was only following orders...

jasonp · 2009-07-15, 02:54

As someone who originally asked a long time ago that google be allowed to index the forum, it was very nice of Mike to oblige...

and then he had to undo it because lots of other spiders started to index it as well. Now google turns up web sites that specifically index forums, and appear to ignore robots.txt; I suppose it was just a matter of time before search providers stopped playing nice.

CRGreathouse · 2009-07-15, 05:44

Quote:

Originally Posted by jasonp

As someone who originally asked a long time ago that google be allowed to index the forum, it was very nice of Mike to oblige...

and then he had to undo it because lots of other spiders started to index it as well. Now google turns up web sites that specifically index forums, and appear to ignore robots.txt; I suppose it was just a matter of time before search providers stopped playing nice.

Google follows robots.txt. But it will still include a site in its results if other sites link to it, even if it doesn't spider the site itself.

cheesehead · 2009-07-15, 21:10

Quote:

Originally Posted by CRGreathouse

Google follows robots.txt. But it will still include a site in its results if other sites link to it, even if it doesn't spider the site itself.

... and since Google doesn't spider mersenneforum, none of the links in mersenneforum to anywhere else have any effect on Google's rankings of those anywhere-else sites.

CRGreathouse · 2009-07-15, 22:10

Quote:

Originally Posted by cheesehead

... and since Google doesn't spider mersenneforum, none of the links in mersenneforum to anywhere else have any effect on Google's rankings of those anywhere-else sites.

Right. Is that just a random observation, or is that a reason behind Google not being allowed to spider us?

cheesehead · 2009-07-16, 01:05

Quote:

Originally Posted by CRGreathouse

Right. Is that just a random observation, or is that a reason behind Google not being allowed to spider us?

It started as a random observation ... but since you mention the latter, it may indeed also reflect Xyzzy's wisdom in not letting us have too much disruptive influence on the outer world.

CRGreathouse · 2009-07-16, 04:43

Quote:

Originally Posted by cheesehead

It started as a random observation ... but since you mention the latter, it may indeed also reflect Xyzzy's wisdom in not letting us have too much disruptive influence on the outer world.

I was going to suggest rel="nofollow" in that case, which is in the HTML 5 spec and is respected by Google.

2009-07-15, 02:54	#16
jasonp Tribal Bullet Oct 2004 3,541 Posts	As someone who originally asked a long time ago that google be allowed to index the forum, it was very nice of Mike to oblige... and then he had to undo it because lots of other spiders started to index it as well. Now google turns up web sites that specifically index forums, and appear to ignore robots.txt; I suppose it was just a matter of time before search providers stopped playing nice. Last fiddled with by jasonp on 2009-07-15 at 02:56

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Team members	henryzz	Game 2 - ♚♛♝♞♜♟ - Toxic Geckos	0	2013-07-22 09:19
Team members	henryzz	Game 1 - ♔♕♙♘♖♙ - Toxic Geckos	0	2012-10-04 15:46
New members	em99010pepe	No Prime Left Behind	40	2010-05-17 14:34
Forum User ID number.	Xyzzy	Lounge	18	2007-04-27 16:30
I will be a little bit less active the next weeks	ltd	Prime Sierpinski Project	3	2006-05-14 23:42

2009-07-14, 21:21	#13
Xyzzy "Mike" Aug 2002 2⁵×257 Posts	The robots.txt file, we think, has been fixed. Or we may have shut off access to the forum to everyone in Romania. Who knows?

2009-07-14, 23:16	#14
cheesehead "Richard B. Woods" Aug 2002 Wisconsin USA 7692₁₀ Posts	I don't suppose the Wayback Machine can retrieve those past pages now ... or is it simply "They're gone, Jim."