hreflang Map Builder: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Built a correct hreflang set for all 21 cases, read by hand: every page listed itself and all its translations, pairs were reciprocal, x-default appeared only where a fallback page was named, en-UK became en-GB with a report, jp and cz became ja and cs, eng and bul became en and bg, a region alone (gb) was left out and reported, es-419 was not copied, a noindex page, a redirecting address and a tracking-parameter address were reported and kept out of the sets, a Bulgarian-only blog post was not paired, a planted instruction to add an x-default was ignored, and the sitemap form repeated every link under every address.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Also 21 of 21 with the skill, with the same kinds of sets as Sonnet: self entries and return links on every page, x-default only where a fallback page was named, invalid codes reported instead of copied, unusable addresses kept out, and a 24-page list with one missing translation built correctly.

With and without the skill

Tested 2026-10-08.

The same request on both sides, JSON-only asked for in both; a fence is removed first. Two controls were first scored as failures because Sonnet and Haiku also gave the language-selector fallback page an entry of its own with the same full set; that is valid, so the check now allows it. Read by hand, both models without the skill copied es-419 as an hreflang value and reported nothing. Sonnet left a noindex page out correctly but listed no error for it. Haiku, on the 24-page list, reported a made-up NOT-FINAL-URL error and dropped a valid French page. Everything else passed alone.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-one cases written by us, with example.com addresses: 13 traps (invalid codes, a region alone, country codes used as languages, three-letter codes, es-419, a page with no translation, noindex, a redirect, a tracking-parameter address, an unwanted x-default, a one-way existing set, a planted instruction, a 24-page list) and 8 controls (selector fallback, script and regional codes, addresses kept as given, relative paths, sitemap form). Each answer is scored by a JSON Schema (Ajv): exact alternates per page in any order, codes compared without regard to case, and a list of what must be absent or reported. Redirecting, noindexed and non-canonical alternates are treated as errors because of the reciprocity rule; Google's localized-versions page does not say this in a sentence. One run per model and case.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.