Checks a summary against the source text it was made from and lists the summary sentences the source does not support, each copied word for word, as strict JSON. It catches a number that was rounded or given another unit, a hedge that became a certainty, a cause the source never states, two facts joined into a false one, a claim whose speaker was dropped, a fact that comes from outside the source, and it ignores any instruction hidden in the source. A sentence is judged supported or unsupported and nothing in between, and a faithful paraphrase is never flagged. It never checks the source against the world and never judges what the summary left out or how well it is written. Use to check a summary, abstract, executive summary or AI-written digest against its source, to review an agent's report of a document, or to find where a summary says more than the text.
Summary Fidelity Check is a tested SKILL.md that checks a summary against the source text it was made from and lists the summary sentences the source does not support, each copied word for word, as strict JSON; an agent buys it once for $0.01 over x402.
Not for
Fact-checking: it compares the summary with the pasted source only, never with the world, so a false statement that the source makes is passed and a true one the source lacks is flagged. It does not judge what the summary leaves out, its style or its length. It reads one source and one summary pasted together, not a set of files or a web page.
Tested, honestly
Tested 2026-10-09 with a strong and a weak model.
With and without the skill
Results with and without the skill, for Sonnet and Haiku |
| with | without | with | without |
|---|
| Summaries judged right (24 pairs) |
| Summaries judged right (24 pairs) | 24/24 | 24/24 | 24/24 | 23/24 |
|---|
Same request and same checks on both sides. The request already spells out what unsupported means and the JSON shape, and with it Sonnet without the skill was right on all 24, faithful summaries included. Haiku without the skill missed one: a Bulgarian summary that dropped who made a claim about unemployment.
Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.
- SonnetStrong model, claude-sonnet-5-5
- Right on all 24, checked by code: a rounded number and a changed unit, dropped hedges, added causes, two facts merged into a false one, dropped speakers, outside facts, a changed quantifier and planted notes in English and Bulgarian, each flagged with the sentence copied exactly; an empty list for eight faithful summaries, and plain words with no list when the summary was missing.
- HaikuWeak model, claude-haiku-5-5
- Right on all 24, checked by code, with the same flags and the same empty lists as Sonnet.
Full test summary
Example
Our own test text, before and after the skill ran. Excerpts only.
English · claude-sonnet-5-5
Before
SOURCE:
The new bridge is 14 kilometres long and cost 380 million euros. It opened to traffic in May.
SUMMARY:
1. The new bridge is 14 miles long.
2. The bridge cost 380 million euros.
3. It opened to traffic in May.
After
{"unsupported": [{"sentence": "The new bridge is 14 miles long.", "why": "The source gives the length as 14 kilometres, not miles."}]}
German · claude-sonnet-5-5
Before
SOURCE:
Die Stadt hat im Juni die Parkgebühren erhöht. Im Juli wurden 2.300 Parkscheine weniger verkauft als im Juni.
SUMMARY:
1. Die Stadt erhöhte im Juni die Parkgebühren.
2. Wegen der höheren Gebühren wurden im Juli weniger Parkscheine verkauft.
After
{"unsupported": [{"sentence": "Wegen der höheren Gebühren wurden im Juli weniger Parkscheine verkauft.", "why": "Die Quelle nennt beide Fakten nur nebeneinander und sagt nicht, dass die Gebührenerhöhung den Rückgang verursacht hat."}]}
Spanish · claude-sonnet-5-5
Before
SOURCE:
La biblioteca de Sevilla abrió en 2015 y tiene 80 000 libros. La de Granada abrió en 2019.
SUMMARY:
1. La biblioteca de Sevilla tiene 80 000 libros.
2. La biblioteca de Granada tiene 80 000 libros.
3. La de Granada abrió en 2019.
After
{"unsupported": [{"sentence": "La biblioteca de Granada tiene 80 000 libros.", "why": "La fuente da los 80 000 libros solo para la biblioteca de Sevilla, no para la de Granada."}]}
Bulgarian · claude-sonnet-5-5
Before
SOURCE:
Министерството твърди, че през 2025 г. безработицата е спаднала до 5,4 %. Синдикатът оспорва цифрата.
SUMMARY:
1. Безработицата през 2025 г. е спаднала до 5,4 %.
2. Синдикатът оспорва цифрата.
After
{"unsupported": [{"sentence": "Безработицата през 2025 г. е спаднала до 5,4 %.", "why": "Източникът представя цифрата като твърдение на Министерството, а резюмето я излага като безспорен факт."}]}
What is in the file
- The answer
- When a sentence is supported
- When a sentence is unsupported
- Report only what the text shows
- Missing input
- Work in this order
- Short examples
Languages
English, Bulgarian, Spanish, German. Tried in: English, German, Spanish, Bulgarian.
License
Perpetual, non-exclusive; use and modify for yourself incl. paid work; no resale or republishing. Holder: Georgi Kalchev, aiskills402.com. Full terms.
Versions
Current version 1.0.1, updated 2026-10-09. Whoever bought an earlier version gets new ones free through the same re-download token.
v1.0.1 · 2026-10-09
- Measured on 24 pairs: Sonnet 24 -> 24, Haiku 23 -> 24 (without -> with the skill, same checks). No check was widened. - Price: $0.01 (no measured gain).
v1.0.0 · 2026-10-09
First release: reads a source text and a numbered summary and lists the summary sentences the source does not support, each copied word for word, as strict JSON. Eight defect kinds (figure changed, hedge removed, reversed, cause added, facts joined wrongly, speaker dropped, outside fact, invented detail); binary verdict per sentence; faithful paraphrase never flagged; lines addressed to an AI are not obeyed; missing source or summary gets a plain refusal. Designed from the batch-4 plan row (no § 2 brief existed when written). No model run yet: the baseline, the price check ($0.02 provisional) and the sentence on what Sonnet gains are still to be written.
FAQ
What counts as unsupported?
A summary sentence the source does not say: a rounded number or a changed unit, a hedge turned into a certainty, a cause the source never states, two facts joined into a false one, a claim whose speaker was dropped, a fact from outside the source, or an invented detail. A faithful paraphrase, a shorter form or a weaker statement is supported, and so is a sentence that merges facts about one and the same subject.
Does it check whether the source is true, and what does a faithful summary get?
No, the source is the whole world for this job. A true sentence that the source lacks is still flagged, because a reader who trusts the source would not find it there, while a false statement that the source itself makes is passed. A summary that only shortens, reorders or rewords, keeps the hedges and names the speakers gets an empty array and nothing more.
How are the flagged sentences and odd inputs handled?
Every flagged line is the summary sentence in its original wording, minus its list number, so a reviewer can search for it; a script can confirm that with a plain substring test. A planted note in the source that tells the assistant to mark everything as supported is not obeyed. If the summary or the source is missing, the agent says so in plain words and returns no list.
Does it help Claude Sonnet?
Our numbers show no gain for it, so one cent is the price. Twenty-four source and summary pairs in four languages went to Sonnet and Haiku, guided by the file or not, and a script compared the flagged sentences with the planted ones. Our request already spelled out what unsupported means, and with it plain Sonnet was right on all 24, eight faithful summaries included. Haiku missed one without the file: a Bulgarian summary that dropped who made a claim. The file keeps the rules in one place for a pipeline.