You Must Now Exclude Search Engine Results Pages From Google’s Index

If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!

If you run a “related search” type of setup on any of your websites, you’re probably going to need to exclude the results pages from Google’s results. This is a pretty common trick, so there’s no doubt in my mind there’s probably tons of websites out there that employ this technique to help get “extra” search engine traffic. Well now, Matt Cutts, has cut out this loophole

Google has been making addendums to their webmaster guildelines lately, with an emphasis on clearly explaining what you can or cannot do in certain circumstances.

Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.

The language certainly appears concise and clear enough. Hopefully Google plans on enforcing this new rule across the board, and they don’t just focus on the “little guy”. There are actually a ton of huge companies who benefit from this very setup, and undoubtedly they’ll be affected in a major way with this news.

As Danny Sullivan’s post explained: there are many, many keyword categories affected by this. In particular, “product and price comparision search engines” would seemingly be dead, now. Unless they really do have a huge roster of repeat visitors.

On a personal note, I can say that Sootle was banned for this very offense in 2004. I don’t mean I was “Warned”. I mean I got an instant and brutal death penalty for the main domain and all sub-domains. I “cleaned up my act” by removing the offending pages and got back in. Now it appears that this offense isn’t automatically bannable, but it probably should be. There’s probably no good reason pages like that are in for most searches. I can admit my intention was to get traffic from Google, and I’m sure that’s the case with these large companies doing the same thing.

I consider this to be a very significant change to policies. Do you?

If you enjoyed this post, subscribe to the Sootle RSS feed!.