One of the coolest features in the Google Webmaster Toolkit is the Sitemaps. The best thing I learned using this tool was about how your Crawl Rate affects your real life experience in gaining revenue from a website. The two are much more closely related than I would have thought. The faster your Crawl Rate, the better. If your Crawl Rate is very slow, your revenue has probably already plummeted.
Incoming links increase your Crawl rate
This comes as no surprise. Incoming links have long been the mantra of search engine optimizers, and with very good reason. The links mean your website is “important”, and much like anywhere in life, the more important your website is deemed to be, the better.
I assume you’re updating your website relentlessly
When I make this post I’ll assume my readers are doing the same things I am. And that is, updating website relentlessly with the full knowledge that content creation is but a small part of my overall job, but one that CANNOT be overlooked, no matter what else I do. If I don’t create some sort of value on the internet, in the form of usability and features, or with content, I don’t stand a chance of truly moving forward. So let’s assume you are creating a lot of new pages. Then the only thing left to do is to get them indexed in Google.
Increase your Crawl rate by updating frequently and export your HTML in RSS and ping
Update frequently, and offer an RSS feed that pings your changes to the “blogosphere”. It’s a relatively painless process to turn any HTML web page into RSS. I wrote a small script in PHP to do it, but there are probably many such utilties already available. This is a must. Export all your pages into RSS and setup and account at Feedburner. Link the RSS Chicklet from every page of your website. Set your Feedburner account to Ping all 10 services it allows. Also, just for good measure, make sure to download this new account into your own RSS feed reader. This will alert the spiders as your pages update. Your website does not have to be a blog to do this, and it helps immensely.
These simple tips should help improve your Crawl Rate within a matter of weeks.
Always a good idea to use the resources from google’s webmastersite a.k.a google sitemaps to better understand your organic traffic and the way it involves the google spiders and crawlers
I submitted a few sitemaps when google first came out with it. I failed to update the sitemap for a few months then updated, I noticed a big increase in visitors after I updated. Definitly something everyone should be doing.
It’s definitely something that can’t hurt, and can possibly help. So why not use it?