Should you block DataForSeoBot? DataForSEO's SEO crawler
DataForSeoBot is operated by DataForSEO. It crawls for DataForSEO's SEO and marketing database; readers never see its fetches.
Who should block DataForSeoBot, and who should not?
DataForSEO mainly sells data through APIs and databases: the links its bot records feed a Backlinks API and Backlink Database, so they can end up in products built on that data. Blocking it costs readers nothing and removes the links on your own pages from that supply; it does not hide links pointing to you. With a 5-second default gap it is gentle, so block it for data reasons rather than load. Its small, published subnets make it one of the easiest bots to verify.
The verified facts
| User-agent token | DataForSeoBot |
|---|---|
| Full user agent in logs | Mozilla/5.0 (compatible; DataForSeoBot; +https://dataforseo.com/dataforseo-bot) |
| Operator | DataForSEO |
| Purpose | SEO and marketing tool crawlers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Reverse DNS ends in | dataforseo.com |
What does DataForSEO’s documentation add?
- DataForSEO says the bot adds the links it finds on your pages to its backlink database and collects no other data, triggers no ads, and does not add to your Google Analytics traffic.
- Its default gap between requests is 5 seconds; a Crawl-delay line in its robots.txt group changes that.
- It publishes three IPv4 subnets and three IPv6 subnets, and reverse DNS names of the form crawling-gateway-136-243-228-176.dataforseo.com.
- DataForSEO lists a Backlinks API and a Backlink Database among its products, the commercial outlets for this data.
How do you block DataForSeoBot?
Add this to your robots.txt:
User-agent: DataForSeoBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real DataForSeoBot request from a fake one?
A user-agent match proves nothing, since any client can send DataForSeoBot. Reverse-resolve the IP, check the name ends in dataforseo.com, and forward-resolve that name back to the same IP. A request that fails is not DataForSEO. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against DataForSEO’s current data.
What does blocking DataForSeoBot cost you?
DataForSEO stops adding the links on your pages to its backlink database. Links other sites point at you are still found on those sites.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for DataForSeoBot does nothing to crawlers with other tokens. In the same group, the directory also covers SemrushBot (Semrush), AhrefsBot (Ahrefs) and MJ12bot (Majestic), each with its own token and its own documented cost of blocking.