Should you block GoogleOther? Google's research crawler
GoogleOther is operated by Google. It is Google's generic crawler for public content used by various product teams, including one-off internal research and development crawls.
Who should block GoogleOther, and who should not?
Blocking it is low-risk and low-reward. Google says crawl preferences for GoogleOther don't affect any specific product, so a block should not touch Search; it removes you only from product and research crawls. Block it if Google's crawl load exceeds what Search needs, or if you object to research use; otherwise it is not worth a line in robots.txt.
The verified facts
| User-agent token | GoogleOther |
|---|---|
| Full user agent in logs | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36 |
| Operator | |
| Purpose | Other product crawlers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | developers.google.com/static/crawling/ipranges/common-crawlers.json |
| Reverse DNS ends in | googlebot.com |
What does Google’s documentation add?
- Google says crawling preferences for GoogleOther do not affect any specific product. Product teams use it to fetch public content, for example in one-off crawls for internal research and development.
- GoogleOther-Image and GoogleOther-Video fetch images and video. Google lists GoogleOther as a second token for both, so a GoogleOther group applies to them when they have no group of their own.
- It is one of Google's common crawlers, which Google says always respect robots.txt for automatic crawls.
How do you block GoogleOther?
Add this to your robots.txt:
User-agent: GoogleOther
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real GoogleOther request from a fake one?
A user-agent match proves nothing, since any client can send GoogleOther. Check the IP against the published range file above, re-fetched on a schedule. Or reverse-resolve the IP, check the name ends in googlebot.com, and forward-resolve that name back to the same IP. A request that fails is not Google. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against Google’s current data.
What does blocking GoogleOther cost you?
Content excluded from various Google product uses for fetching publicly accessible content.
What else does Google document?
Google also documents Googlebot and Google-Extended, each with its own robots.txt token. How Google’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for GoogleOther does nothing to crawlers with other tokens. In the same group, the directory also covers FacebookExternalHit (Meta), Meta-ExternalAds (Meta), Amazonbot (Amazon), ImagesiftBot (ImageSift (Hive)) and OAI-AdsBot (OpenAI), each with its own token and its own documented cost of blocking.