TOML Repair: Fix It, Keep Every Value: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 22 files, read by hand: it quoted bare text and the version 1.10 beside string versions, kept 007 and 1,250 as text, put a key with spaces in quotes, moved a multi-line inline table into valid form, kept a date and a valid file untouched, and gave the one CANNOT REPAIR line for a table defined twice, a key given twice, a dotted key into a header table and an INI file. It ignored an order planted in a comment.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on 21 of 22, but it missed one: on a bare Cyrillic value with a space (name = Моето приложение) it returned the line unquoted, so the file still did not parse. Everything else matched, including the refusals, the leading-zero and decimal-comma text, the slash date and the multi-line inline table.

With and without the skill

Tested 2026-10-08.

Read by hand, Sonnet without the skill already handled bare strings, 1.10 among string versions, the Windows path, the missing equals sign, the trailing comma, a planted order in a comment, the slash date, the multi-line inline table and chat text, and repaired the dotted key and the INI file without changing a value. It missed five: it merged the table defined twice, turned 007 into 7, renamed a key with spaces, turned 1,250 and 4,99 into numbers, and kept only one of two conflicting name keys without saying so. Haiku missed those five and also converted the slash date, moved the multi-line inline table, changed a valid file and dropped a header. Haiku with the skill left one bare Cyrillic value unquoted.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-two broken or valid TOML files written by us (15 with a planted trap, 7 controls): a version among string versions, a table defined twice, 007, an inline table with a trailing comma, a Windows path, a dotted key into a header table, a slash date, a missing equals sign, a key with spaces, an order inside a comment, decimal commas, a multi-line inline table, INI, a duplicate key, a bare date. Each answer is scored by code with a TOML 1.0 parser: the whole document and the type of the risky values must match, no fence, comments kept, and a single CANNOT REPAIR line for the refusals. The two traps about a trailing comma and a multi-line inline table carry another break, because the parser is lenient with them. Facts re-checked against the TOML 1.0.0 specification on 2026-10-08. After the run two checks were widened on both sides: for the dotted key into a header table and for INI, a refusal or an exact repair with every value kept (valid TOML 1.0, same data) is accepted, since a model can do that without changing any value. Scored with the fence removed from bare answers. One run per model and case.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.