What Does Google’s Improved Crawling of Geotargeted Sites Actually Mean?

Ecommerce sites most commonly use geo-targeting, an optimization technique that helps them serve different content to audiences in different countries. In an effort to reduce challenges these sites face when it comes to organic searches, Google continues to make improvements in the way they crawl these geo-targeted sites. These changes enable more multinational sites to rank better in search results.

Geo-targeting is an accepted website feature and strategy that works to improve customer experience. Unfortunately, some of the techniques and features of its implementation can sometimes prevent and hinder search engines from crawling and indexing site content holistically. In an effort to resolve crawling and indexing issues, Google created new and improved locale-aware crawling configurations for Googlebots, which are automatically activated when Google detects language/locale-based content, algorithmically. These configurations include language-dependent crawling, through which search bots can start crawling pages with “Accept-Language” HTTP header in the request, and geo-distributed crawling, through which bots can recognize IP addresses that appear to be coming outside of the US in addition to those that they currently use. These new configurations are deployed automatically as Google’s algorithm detects the need for more thorough indexing.

These improvements in Google’s crawling configurations are designed to be deployed automatically, which means no notifications or changes are required on your part. However, they do not eliminate the need to optimize your site locally. While this is a step towards improving geo-targeting issues, search engines still have a long way to go in optimizing and fine-tuning algorithms and configurations to support geo-targeting better. Google, for one, still strongly recommends creating separate URLs and rel-alternate-hreflang annotations for different locales.

The other side of this is, there’s still no way to request Google to engage its locale-aware crawling configurations on your website and still no way to determine if it is already being used on your site.