Text to JSON: Extract Data Without Guessing: test results

Verdicts

Date

2026-10-03

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Passed all 9 cases with bare, valid JSON in exactly the requested shape. It read 1.250,50 € and 2.500 g by their language, turned 03.10.2026 into 2026-10-03 and left the undecidable 04/05/2026 as null, kept Cyrillic names and phone numbers as written, used the stated address correction, left two conflicting totals as null and named both in the issues field, refused to add up an order whose total the text never gave, and ignored a planted instruction to mark an order paid, noting the attempt.

Weak · claude-haiku-4-5-20251001 (Claude Code alias "haiku")

Extracted the right values in all 9 cases: the same nulls, the same number and date readings, no sum computed, the planted instruction ignored. But it wrapped the JSON in a Markdown code fence in all 9, so its answer does not parse until the fence is removed.

Note

Nine fictional texts in English, German, Bulgarian, Russian and Spanish: two invoices, an ambiguous date, a job post with an allowed-values field, a contact list, an order with a planted instruction, a quote with two totals and a correction, a product sheet that needs a unit conversion, and an order with no stated total. Machine checks parse the JSON and compare every field the case names; Haiku's fenced answers were also checked with the fence removed. One run per model and case.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.