Should you block LinerBot? Liner's search crawler
LinerBot is operated by Liner. It gathers information from the web and indexes it for Liner's search engine.
Who should block LinerBot, and who should not?
The stakes are small either way: LinerBot feeds one company's search engine. Allow it if you want your pages findable in Liner; block it, or only its private paths, if you do not. Liner's page does not say whether the index is used for model training, so treat that as unknown. Verify against the published IP file before blaming it for load, since the user agent is easy to copy.
The verified facts
| User-agent token | LinerBot/1.0 |
|---|---|
| Full user agent in logs | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; LinerBot/1.0; +https://docs.getliner.com/docs/linerbot) |
| Operator | Liner |
| Purpose | Search and AI-search indexers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | docs.getliner.com/linerbot.json |
What does Liner’s documentation add?
- Liner describes LinerBot as a crawler that gathers information from the internet and indexes it for its search engine.
- It accepts path-level rules, so you can allow /public/ and disallow /private/ rather than blocking the whole site.
- Liner says its IP pool may change over time and points to linerbot.json for the current list.
How do you block LinerBot?
Add this to your robots.txt:
User-agent: LinerBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real LinerBot request from a fake one?
A user-agent match proves nothing, since any client can send LinerBot/1.0. Check the IP against the published range file above, re-fetched on a schedule. A request that fails is not Liner. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against Liner’s current data.
What does blocking LinerBot cost you?
Your pages drop out of Liner's search index.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for LinerBot does nothing to crawlers with other tokens. In the same group, the directory also covers OAI-SearchBot (OpenAI), Claude-SearchBot (Anthropic), Googlebot (Google), Bingbot (Microsoft), Applebot (Apple) and PerplexityBot (Perplexity), each with its own token and its own documented cost of blocking.