Test results
Vocab From Text: test results
Tested 2026-10-08, skill version 1.0.0 at the time of loading this page. We run every skill on a strong and a weak model before it is listed, and publish both verdicts, including where the weak one fails.
Verdicts
- Date
- 2026-10-08
- Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")
- Right on 21 of 23 texts, read by hand, but it missed two: on an English kitchen sentence it listed the preposition above as a content word, and on a Bulgarian text with a sentence broken over a line it listed one item too many. Otherwise the word as written, its dictionary form, a translation in the sense of the text and the exact sentence as example were right, false friends were translated by their real meaning and separable German verbs were joined.
- Weak · claude-haiku-5-5 (Claude Code alias "haiku")
- Right on all 23 texts, read by hand, with the right words, dictionary forms, translations and example sentences.
With and without the skill
Tested 2026-10-08.
| Sonnet | Haiku | |||
|---|---|---|---|---|
| with | without | with | without | |
| Word lists right (23 texts) | 21/23 | 22/23 | 23/23 | 22/23 |
Same request on both sides, a fence removed first. Sonnet without the skill already did this task well and missed one list, where a sentence broken over a line led it to an extra item. With the skill it did one worse, so for Sonnet the skill measured no gain; Haiku went from 22 to 23.
Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.
Note
Twenty-three short texts written by us in English, German, Spanish and Bulgarian, each with the learner's language and a list of known words (16 with a trap, 7 plain): known words in other forms, false friends, separable verbs, irregular forms, a sentence broken over a line and a planted instruction. Each answer is parsed as JSON and checked by code: exactly the unknown nouns, verbs, adjectives and adverbs, the dictionary form, the translation and the example sentence copied from the text. Word classes follow the Universal Dependencies tags. No check was widened. One run per model and text.
What was not measured
- Models other than the two named above were not run.
- Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
- Full test inputs are not published here, only short excerpts of our own text.
- Results on your own texts, languages and domains can differ.