Should you block DuckDuckBot? DuckDuckGo's search crawler
DuckDuckBot is operated by DuckDuckGo. It crawls pages for DuckDuckGo search results.
Who should block DuckDuckBot, and who should not?
Almost no public site should block it. It identifies itself with a checkable IP list and feeds DuckDuckGo's own indexes. If the goal is to keep your text out of DuckDuckGo's AI-assisted answers, block DuckAssistBot instead: DuckDuckGo says that opt-out does not change organic rankings. Block DuckDuckBot only for sections no search engine should crawl, and give Bingbot the same rule, because Bing supplies most of DuckDuckGo's ordinary links.
The verified facts
| User-agent token | DuckDuckBot |
|---|---|
| Full user agent in logs | DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html) |
| Operator | DuckDuckGo |
| Purpose | Search and AI-search indexers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | duckduckgo.com/duckduckbot.json |
What does DuckDuckGo’s documentation add?
- DuckDuckGo says it maintains DuckDuckBot and its own indexes to support its results, but sources the traditional links and images in those results largely from Bing.
- Its IP list is the same as DuckAssistBot's: on 2026-09-26 duckduckbot.json and duckassistbot.json were identical files of 486 addresses each, so an IP match proves a request came from DuckDuckGo, not which of the two bots sent it.
- DuckDuckGo states only that it respects WWW::RobotRules, the name of a Perl robots.txt parser. It documents no crawl-delay support and no time for a robots.txt change to take effect.
How do you block DuckDuckBot?
Add this to your robots.txt:
User-agent: DuckDuckBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real DuckDuckBot request from a fake one?
A user-agent match proves nothing, since any client can send DuckDuckBot. Check the IP against the published range file above, re-fetched on a schedule. A request that fails is not DuckDuckGo. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against DuckDuckGo’s current data.
What does blocking DuckDuckBot cost you?
Your pages drop out of the indexes DuckDuckGo builds with its own crawler. DuckDuckGo says it sources most of the traditional links in its results from Bing. It does not say whether a DuckDuckBot block touches those; expect them to follow your Bingbot rules.
What else does DuckDuckGo document?
DuckDuckGo also documents DuckAssistBot, which has its own robots.txt token. How DuckDuckGo’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for DuckDuckBot does nothing to crawlers with other tokens. In the same group, the directory also covers Meta-WebIndexer (Meta), Amzn-SearchBot (Amazon), YouBot (You.com), Diffbot (Diffbot), PetalBot (Huawei) and MistralAI-Index (Mistral), each with its own token and its own documented cost of blocking.