· ReadyStack
Worked example
Real numbers from this tool, line by line.
Fifteen findings in one 32-line robots.txt — six errors, seven warnings, two notes — for EU publishers and web teams who believed they had opted out.
L1 warn no_tdm_signal nothing points at /.well-known/tdmrep.json
L1 error trainer_gap 8 of 9 training crawlers still unreserved
L2 error rule_before_group Disallow above first User-agent — dropped
L7 info crawl_delay not in RFC 9309; GPTBot ignores it
L8 warn noindex_line unhonoured by Google since 1 September 2019
L13 error legacy_token anthropic-ai retired — use ClaudeBot
L16 error unknown_token ChatGPT-Bot: no such crawler
L19 error trainer_allowed CCBot + empty Disallow: is permission
L19 warn star_group_shadowed CCBot can now reach /wp-admin/, /checkout/
L22 info agent_bot_blocked ChatGPT-User blocked — costs the citation
L25 warn star_group_shadowed Googlebot can now reach /wp-admin/, /checkout/
L26 error path_no_slash private/ has no leading slash — never fires
L28 warn duplicate_group GPTBot already has a group on line 10
L31 warn unknown_directive "allow all" is not a robots.txt field
L32 warn sitemap_relative /sitemap.xml is relative and is discardedThe file parsed. Any validator returns it clean, because validators check grammar — and grammar was never the problem.
Article 4(3) of Directive (EU) 2019/790 lets a rightsholder reserve text-and-data-mining rights over content published online, and requires that reservation to be machine-readable; Germany transposed it as § 44b UrhG. Since 2 August 2025, Article 53(1)(c) of the EU AI Act makes providers of general-purpose AI models run a copyright policy that identifies and respects those reservations. A reservation a machine cannot parse is, to the machine, an absence — and an absence reads as consent.
Five of the six errors sit on a line you can point at.
Line 2 is a Disallow: /wp-admin/ above the first User-agent: line. RFC 9309 defines rules as belonging to a group, and a rule outside one is dropped. It was never in force.
Line 13 is User-agent: anthropic-ai, a retired token. Anthropic sends ClaudeBot, and robots.txt matches the token literally, so the group below it matched no crawler at all. The same trap catches Claude-Web and FacebookBot, replaced by meta-externalagent.
Line 16 is User-agent: ChatGPT-Bot. No such crawler exists — it is the kind of token a generated config file produces: plausible, symmetrical, wrong.
Line 19 is User-agent: CCBot followed by an empty Disallow:. That is not a reservation. It is full permission, in the shape of a restriction, to the crawler feeding the corpus most model builders start from.
Line 26 is Disallow: private/. Paths are matched from the site root; without the leading slash it never fires.
The sixth error is the file as a whole. Of the nine training crawlers tracked by name — GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider, meta-externalagent, Applebot-Extended, PerplexityBot, Amazonbot — eight are unreserved. Each is matched on its own token, so a group for GPTBot says nothing to the other eight.
TDM Reservation Lint is a VS Code extension running 15 rules over the robots.txt you have open. On a corrected version of the same file it reports nothing.
Nothing points at a TDMRep signal either: robots.txt instructs crawlers, while the W3C TDM Reservation Protocol expects /.well-known/tdmrep.json.
The free layer does the whole job for one file: the robots.txt in front of you, all 15 rules, every finding with its replacement line. The licence key moves the axis, not the depth — the whole workspace, and files you keep: a corrected robots.txt, a matching tdmrep.json, a dated evidence report. A media and IP lawyer reviewing whether a reservation is machine-readable bills from €200 an hour, and then hands the token list to a developer anyway.
Questions people ask
What does TDM Reservation Lint actually do?
It parses the robots.txt open in your editor and reports every line that does not do what it looks like it does: retired crawler tokens, invented tokens, rules sitting above the first User-agent line, paths missing a leading slash, empty Disallow lines under training crawlers, and groups that cancel the User-agent: * block. Fifteen rules, each finding carrying the replacement line.
Who is this for?
Publishers, news and magazine developers, agencies and web teams inside the EU who have to make an Article 4(3) text-and-data-mining reservation machine-readable, and anyone outside the EU who wants AI training crawlers to actually receive the instruction they wrote. It is aimed at the person who edits robots.txt, not at the legal department.
Why is a free robots.txt validator not enough?
Syntax validators check grammar. A group naming anthropic-ai is perfect grammar and matches no crawler, because Anthropic sends ClaudeBot. A Disallow above the first User-agent line parses fine and is discarded. Validators pass both. This checks the published token registries and RFC 9309 group semantics, which is where reservations fail.
What is free and what needs a licence key?
Free covers the whole job for one file: the robots.txt you have open, all fifteen rules, every finding with its fix. The licence key changes the axis rather than the depth — it scans every robots.txt in the workspace and writes files you keep: a corrected robots.txt, a matching /.well-known/tdmrep.json, and a dated Markdown evidence report.
What would this cost done by a person?
A media and IP lawyer reviewing whether a reservation is machine-readable bills from around 200 euro an hour, and they will still hand the token list to a developer to implement. The extension costs 29 dollars once and reads a file in about a second, which is the difference between a review and a check you can run on every commit.
Why not just ask ChatGPT or another AI chat?
A general AI chat answers from training data with a cutoff date, cannot read your repository and names no rule version. TDM Reservation Lint checks the file you open against 15 rules from a rule set dated 2026-09-20, and points at the exact line with the fix. For a filing, an audit or a client you need that dated result on your own files.
Ask about this tool
One question, answered by the person who built it. Your email only if you want the answer sent.