|
Here's the lowdown on Google and it's bots.
The freshbot crawls your popular pages on a frequent basis - for some sites like Amazon it crawls the site every 10 minutes: on less popular sites, it may only come once every 2 weeks.
What happens on the freshbot crawls is that it finds any new links and indexes them.
When the once-a-month deppbot comes, it will then crawl each page that the freshbot found.
That's why once you get the sitemap in place, you need to give it about a month to see the impact.
Hope this helps!
|