Lints robots.txt for the machine-readable TDM reservation EU law expects: dead crawler tokens, shadowed groups, missing AI trainers, and rules that never apply. Runs entirely in your browser — nothing is uploaded.
Same engine as the VS Code extension, byte for byte.
This page is the working piece. The full pack has everything below.
Finds the robots.txt lines that reserve nothing — retired crawler tokens, shadowed groups, rules above the first User-agent.
A media and IP lawyer reviewing a TDM reservation bills from €200 an hour; this reads the file in about a second.
Buy the full version — $29It parses the robots.txt open in your editor and reports every line that does not do what it looks like it does: retired crawler tokens, invented tokens, rules sitting above the first User-agent line, paths missing a leading slash, empty Disallow lines under training crawlers, and groups that cancel the User-agent: * block. Fifteen rules, each finding carrying the replacement line.
Publishers, news and magazine developers, agencies and web teams inside the EU who have to make an Article 4(3) text-and-data-mining reservation machine-readable, and anyone outside the EU who wants AI training crawlers to actually receive the instruction they wrote. It is aimed at the person who edits robots.txt, not at the legal department.
Syntax validators check grammar. A group naming anthropic-ai is perfect grammar and matches no crawler, because Anthropic sends ClaudeBot. A Disallow above the first User-agent line parses fine and is discarded. Validators pass both. This checks the published token registries and RFC 9309 group semantics, which is where reservations fail.
Free covers the whole job for one file: the robots.txt you have open, all fifteen rules, every finding with its fix. The licence key changes the axis rather than the depth — it scans every robots.txt in the workspace and writes files you keep: a corrected robots.txt, a matching /.well-known/tdmrep.json, and a dated Markdown evidence report.
A media and IP lawyer reviewing whether a reservation is machine-readable bills from around 200 euro an hour, and they will still hand the token list to a developer to implement. The extension costs 29 dollars once and reads a file in about a second, which is the difference between a review and a check you can run on every commit.
One question, answered by the person who built it. Your email only if you want the answer sent.