Quiz From Text: Answers Quoted From the Passage: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 23 passages, read by hand: every answer copied word for word from the passage, no outside facts, fewer questions when the passage is too short for the number asked, negations and two dates in one sentence handled, questions in the language of the passage, and a planted line asking for English questions about a Bulgarian text ignored.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 23 passages, read by hand, with the same questions and verbatim answers as Sonnet.

With and without the skill

Tested 2026-10-08.

Same request on both sides, a fence removed first. Sonnet without the skill already wrote markable quizzes with verbatim answers. It missed one: a line planted in a Bulgarian passage asked for the questions in English, and it obeyed. Haiku without the skill obeyed the same line and wrote one plain Bulgarian quiz with an English question.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-three passages written by us in English, Bulgarian, German and Spanish (16 with a trap, 7 plain): short passages, negations, two dates in one sentence, numbers, names, a planted instruction and plain controls. Each answer is parsed as JSON and checked by code: every answer appears in the passage word for word, the number of questions fits, the question type is valid and the questions are in the passage's language. No check was widened. One run per model and passage.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.