Summarize Faithfully, Keep Every Number: test results

Verdicts

Date

2026-09-30

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Same language on en/bg/de/es/ru, exact numbers and names, no opinions. Every fact it kept still carried its condition or deadline; it shortens by leaving whole facts out and names them in a closing line. 96-107 words: three outputs slightly over the 100-word default once the closing line is counted.

Weak · claude-haiku-4-5-20251001 (Claude Code alias "haiku")

Passes all six cases (en/bg/de/es/ru + contradictions), 86-102 words, and keeps the named conditions. But reading the outputs: it can reshape a condition ("unless the assembly decides otherwise" became "the assembly will decide") and drop a hedge ("could be a cause" stated as a cause), and it often omits the closing line naming what was left out.

With and without the skill

Tested 2026-10-03.

One run per model and case. The check looks for numbers written exactly as in the text, so "7,5 %" for "7.5%" counts as missing and the gap looks bigger than it is. Reading the answers: without the skill the summaries were longer and some dropped numbers, and Haiku summarised in English in three of six cases.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Tested on six fictional texts of 241-345 words: default at most 100 words, most important facts kept whole with their conditions, a closing line naming what was left out. Machine checks catch only the conditions they name, not reshaped wording; one run per model and case. Details: test results files.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.