Job Description From Notes: Gaps Marked, Nothing Invented: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on 21 of 23 hiring notes, read by hand, but it missed two: once it wrote the post in Bulgarian from English notes about a job in Burgas, and once it dropped a native-speaker requirement instead of turning it into a fluency level. Everything else held: seven sections in order, one TBD line only for an uncovered section, pay copied as written, no benefit, perk, equity or praise the notes do not give, age, gender and nationality wording made neutral, a recent-graduate wish turned into the skills it stands for, orders planted in the notes ignored, and German, Bulgarian and Spanish notes answered in their own language.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on 20 of 23 hiring notes, read by hand, but it missed three: it dropped the native-speaker requirement instead of naming a level, kept the words recent graduate, and wrote a German post in English.

With and without the skill

Tested 2026-10-08.

Same request on both sides, a fence removed first. Read by hand, Sonnet without the skill wrote readable posts with the sections in order. It missed twelve: it added competitive pay, benefits or equity the notes never give (once because a planted line asked for it), kept a recent-graduate wish and German native-tongue and age wording, wrote three posts in Bulgarian from English notes because the job was in Bulgaria, and dropped facts such as the team size. Haiku without the skill missed nine of the same kinds.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-three sets of hiring notes written by us in English, Bulgarian, German and Spanish (16 with a trap, 7 complete): no pay, benefits never mentioned, age, gender, nationality and native-speaker wishes, a recent-graduate wish, a young team, a planted order to add equity or a salary, a city that is not a residence rule, sparse notes and complete notes. Each answer is checked by code for the seven sections, every fact from the notes (amounts accepted with any thousands separator), no invented perk or praise, no discriminatory wording, and the language of the notes. The comparison does not punish extra [TBD] lines, because the request on both sides says to mark missing information that way; the skill's stricter rule of one TBD only for an uncovered section is checked on its own side. The skill was fixed twice during the test and its side re-run in full each time: About the role is one sentence from the title and a duty instead of a TBD line, and the post is written in the language of the notes, not of the country where the job is. One run per model and note on the final version.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.