Robots.txt Policy Writer: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Wrote a correct robots.txt in all 22 policies: the private paths repeated in every group that names a bot, GPTBot blocked while OAI-SearchBot stays allowed, Google-Extended blocked without touching Googlebot, no Host line, absolute sitemaps, longest-match exceptions for a search folder, PDFs and WordPress, and a comment instead of a pretend rule when the policy asked to remove a page from Google, to slow Google down, or to cover a staging host and a Cloudflare setting.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Got 20 of 22 right. It wrote a Host line when the policy asked for a preferred host, and for a page the policy asked to block but keep out of Google results it left the page open and explained noindex in a comment instead of writing the Disallow. The group rule, the AI bot tokens, the sitemaps and the pattern cases were right.

With and without the skill

Tested 2026-10-08.

The same request on both sides, asking for the robots.txt only; a fence around the answer is removed first, and the comment check looks for a comment that says the thing, not a fixed label. Sonnet already writes nearly every policy right alone, group rule included, so its measurable gain is one line: without the skill it wrote a Host line in the one case that asked for a preferred host. Haiku without the skill also wrote Host, blocked a page the policy wanted removed from Google, opened the private folder to the user-triggered agents, and gave no noindex comment. With the skill the first two were fixed and the comment was written, yet it wrote Host again and left one page open that the policy asked to block. A strong model gains little here; the smaller one gains more.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-two crawler policies written by us, all in English: about two thirds with a trap (a bot group that must repeat the private paths, training blocked while AI search stays open, Google-Extended against Googlebot, a Host request, a relative sitemap, a request to remove a page from Google, a staging host, longest-match exceptions) and the rest plain policies where the obvious file is right. Each answer was parsed as a robots.txt by the RFC 9309 rules (the most specific group replaces the star group, longest match wins, Allow wins a tie, * and $) and every named bot was asked whether it may fetch the stated paths; sitemaps had to be absolute, a Host line was refused, and cases that ask for something robots.txt cannot do also needed a comment that says so. The rules and bot names in the skill were read on the vendors' own pages on 8 October 2026. One run per model and policy.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.