Should you block AhrefsBot? Ahrefs's SEO crawler
AhrefsBot is operated by Ahrefs. It crawls for Ahrefs's SEO and marketing database; readers never see its fetches.
Who should block AhrefsBot, and who should not?
Unlike the other SEO crawlers in this directory, AhrefsBot also feeds a search engine, Yep, so a block costs a little search visibility as well as tool coverage. For most sites the better lever is Crawl-delay, which Ahrefs honors for HTML fetches. Block it when crawl load is a real cost, or when you do not want your pages' content in a competitor-research database. A block does not hide links pointing to you: AhrefsBot records those when it crawls the linking sites.
The verified facts
| User-agent token | AhrefsBot/7.0 |
|---|---|
| Full user agent in logs | Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) |
| Operator | Ahrefs |
| Purpose | SEO and marketing tool crawlers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | api.ahrefs.com/v3/public/crawler-ip-ranges |
| Reverse DNS ends in | ahrefs.com or ahrefs.net |
What does Ahrefs’s documentation add?
- AhrefsBot feeds two products: the Ahrefs marketing-intelligence database and Yep, a search engine Ahrefs runs. Blocking it removes you from both.
- Ahrefs' Site Audit tool crawls under a separate token, AhrefsSiteAudit, limited to 30 URLs a minute. Owners who verify a site in Ahrefs can let that audit crawler ignore robots.txt on their own site.
- Crawl-delay is followed for HTML pages but not while rendering JavaScript, when the bot may fetch several page assets at once.
- Returning 4xx or 5xx status codes makes it slow down automatically, which Ahrefs suggests for outages or maintenance windows.
How do you block AhrefsBot?
Add this to your robots.txt:
User-agent: AhrefsBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real AhrefsBot request from a fake one?
A user-agent match proves nothing, since any client can send AhrefsBot/7.0. Check the IP against the published range file above, re-fetched on a schedule. Or reverse-resolve the IP, check the name ends in ahrefs.com or ahrefs.net, and forward-resolve that name back to the same IP. A request that fails is not Ahrefs. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against Ahrefs’s current data.
What does blocking AhrefsBot cost you?
Content excluded from Ahrefs backlink database and Yep search engine index.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for AhrefsBot does nothing to crawlers with other tokens. In the same group, the directory also covers MJ12bot (Majestic), DataForSeoBot (DataForSEO) and SemrushBot (Semrush), each with its own token and its own documented cost of blocking.