Should you block MJ12bot? Majestic's SEO crawler
MJ12bot is operated by Majestic. It crawls for Majestic's SEO and marketing database; readers never see its fetches.
Who should block MJ12bot, and who should not?
MJ12bot is the hardest SEO crawler here to verify: no IP list and no documented reverse-DNS check, only an ident string you request by email. Anyone can claim its user-agent, so a firewall rule on that string blocks impostors and the real bot alike, and robots.txt stays the practical control. Throttle it with Crawl-delay if its load is acceptable; block it if a place in Majestic's link index is not worth the requests. The block removes the links on your pages from Majestic, not the links other sites point at you.
The verified facts
| User-agent token | MJ12bot/v1.4.8 |
|---|---|
| Operator | Majestic |
| Purpose | SEO and marketing tool crawlers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Verification | No IP list: a distributed crawler. Majestic adds a pre-arranged ident string to its requests when asked by email |
What does Majestic’s documentation add?
- MJ12bot runs as a distributed community crawler with no fixed IP range, and Majestic asks sites not to block it by IP.
- To verify it, email bot@majestic12.co.uk. Majestic will send a private ident string with its requests to your site, in a CRAWLER-IDENT header or the user-agent.
- It honors Crawl-delay up to 20 seconds. An MJ12bot section in robots.txt replaces the * section rather than adding to it, so copy any crawl-delay into it.
- It stores the link graph, not page content. If it cannot fetch robots.txt it assumes crawling is allowed, but some failures, such as 403 Forbidden, count as a full disallow.
How do you block MJ12bot?
Add this to your robots.txt:
User-agent: MJ12bot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
What does blocking MJ12bot cost you?
Majestic stops mapping the links on your pages. Links that other sites point at you are still recorded when it crawls those sites.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for MJ12bot does nothing to crawlers with other tokens. In the same group, the directory also covers DataForSeoBot (DataForSEO), SemrushBot (Semrush) and AhrefsBot (Ahrefs), each with its own token and its own documented cost of blocking.