Mojibake, CSV and JSONL repair: code against an AI model

Yes, for these tasks, and that is less of a win than it sounds. We wrote a program for each of six repair and conversion jobs and ran it through the 135 checks we had built for AI models. The programs passed all 135, the same as Claude Sonnet with our written rules loaded. Then a sentence that was in none of the cases showed one rule refusing perfectly good text.

Method

We picked six jobs where each input has one right answer that can be written down as rules: undoing mojibake (text whose bytes were read in the wrong encoding), repairing a broken CSV file, repairing broken JSONL, repairing a Markdown table, converting a cron schedule between five scheduler dialects, and writing Bulgarian in Latin letters under the official system.

Each job already had a test: between 21 and 24 cases, fixed before any model saw them, each with a check written in code. Days earlier, Claude Sonnet and Claude Haiku had answered every case once, from the bare request and again with our rules for that job loaded, with tools switched off.

This time there was no model. Each program is ordinary TypeScript with no outside library: text goes in, a result or a one-line refusal comes out. We fed the programs the same 135 inputs and scored their output with the same checks. Every test file also counts the cases it ran against the number in the case file, breaks one output on purpose to prove the checks can fail, and throws 60 KB of hostile input at the program, such as ten thousand open braces. A second reviewer then broke the programs and the payment code 21 times on purpose; each break made a named test fail.

Results

Job	Cases	Sonnet, bare → rules	Haiku, bare → rules	Program
Mojibake	21	16 → 21	16 → 20	21
CSV	23	16 → 23	17 → 22	23
JSONL	22	15 → 22	15 → 22	22
Markdown table	21	17 → 21	17 → 20	21
Cron dialects	24	20 → 24	12 → 24	24
Bulgarian to Latin	24	15 → 24	11 → 18	24
All six	135	99 → 135	88 → 126	135

The models needed the rules, and the rules were enough for Sonnet. From the bare request, Sonnet got 99 of 135 right and Haiku 88. Most of the misses were guesses: a mojibake text with two possible originals turned into one of them, a CSV repair that met a row carrying an extra field and decided on its own which two fields belonged together, a JSONL record cut off halfway returned as if it were whole. With the rules loaded, Sonnet got every case and Haiku 126.

The programs matched Sonnet with rules, case for case. On these 135 inputs there was nothing a model did that the code could not. The code also gives the same answer to the same input every time, needs no prompt and does not read a note hidden in the data as an instruction.

And then the first gap. The mojibake program had a rule from our skill: question marks between two letters mean the letters were lost, so refuse rather than guess. All 135 cases passed. Then we gave it "See https://example.com/page?id=1 for details." and it refused, calling a web address damaged text. "Is it?Yes, it is." was refused as well. The one case with question marks between letters had them in runs of two to four, in German words that lost their special letters. No case had a single question mark wedged between two letters, because the cases were built to catch a model's mistakes, and no model mistakes a web address for broken bytes. We changed the rule to two or more question marks between letters and added both sentences as a test. The price of the change is stated in the same test: a word that lost exactly one letter is no longer refused and passes through as it is.

The second gap came from breaking the program on purpose. The mojibake code ignores letter pairs that only happen to look like damaged bytes unless such pairs make up at least half of the characters outside plain ASCII. The reviewer removed that threshold and all the cases still passed. In correct Russian, the capitalised word for "her" is one such pair, so without the threshold a correct Russian sentence was refused as damaged. Again, no case contained one. Two Russian sentences are now tests.

Refusals stay refusals. Where the cron target cannot express a schedule exactly, the program says so in one line instead of sending something close. It never reads today's date, so the same request gives the same schedule next month.

The six programs run as pay-per-call APIs; the API notes list them. The skill files remain for agents that do the job themselves, for example mojibake-repair, csv-repair and jsonl-repair.

Writing the test

A case set built for a model is not a test of code. Cases aim at the places a model slips. A program slips in different places, usually on plain input that nobody thought to include.
Add ordinary inputs. A web address, a question in a sentence, correct text in another alphabet. A repair tool's worst mistake is changing or refusing text that was fine.
Break the code on purpose. Remove a rule and run the tests. If nothing fails, no test needs that rule, and you do not know what it protects.
Say what a fix costs. Each change to a refusal rule moves some inputs from refused to returned unchanged. Write which ones into the test.

What we did not measure

Running each case again. Each model saw each case once. The program's answers do not change between runs; the models' answers may.
Speed and cost. We did not time the models against the programs, so this report makes no claim about either.
Real files. The inputs are our own short cases. Large exports, rare encodings and mixed files were not tested beyond the hostile inputs above.
Other models. Only Sonnet and Haiku, both run from Claude Code with tools off.
How many ordinary inputs are still refused. We found two gaps by looking. There may be others; a larger set of ordinary text is the next test.

Mojibake Repair: Undo Broken Encoding Exactly
Repairs mojibake, text whose accents, Cyrillic letters, quotes, dashes or emoji turned into strings such as Ã¨, Ã¼, â€œ, Ð¿Ñ€ or Р—Рґ because UTF-8 was read as Windows-1252, Latin-1 or Windows-1251, once or twice. Gives back the original text exactly, character for character, by undoing the wrong reading byte by byte, and leaves text that is already correct untouched. When the damage is mixed with good text, mixes two encodings, or lost bytes, so that more than one original is possible, it answers with a fixed CANNOT REPAIR line instead of guessing. Use when text from a file, database, email, web page or export shows garbled characters, or when asked to fix, undo or decode broken encoding, double encoding or strange characters.
$0.01once, at the time of writing — your agent sees the live price in the 402.
Get skill
CSV Repair: Fix Broken CSV, Keep Every Cell
Repairs broken or messy CSV so a standard parser reads it, without changing the text of a single cell, and answers with the CSV only, no code fence. Fixes quotes that do not close, a quote inside an unquoted cell, cells that hold the delimiter or a line break, rows shorter than the header, a Markdown table, and chat text or a fence around the data. Numbers such as 1,250, 007 and 1.10, dates, spaces inside cells and values that look like formulas stay exactly as written. Semicolon and tab files keep their delimiter. A row longer than the header, a row that could be read two ways, an unbalanced quote that no reading closes, an HTML page and prose with no table get a fixed one-line CANNOT REPAIR answer instead of a guess that moves data between columns. Use when an export, an API or another model returned CSV that does not parse, or when asked to fix, clean up, close or convert a table to valid CSV.
$0.05once, at the time of writing — your agent sees the live price in the 402.
Get skill
JSON Lines Repair: One Record Per Line
Repairs broken JSON Lines (NDJSON) so that every line is exactly one JSON record and no value changes. Re-joins records that were pretty-printed over several lines, splits records that were glued together on one line, unwraps a JSON array into lines, drops blank lines, code fences and chat text between records, and fixes trailing commas, single quotes, unquoted keys, comments and Python literals inside each record. Numbers are copied as written, never re-serialised. A last record that was cut off, a record with a repeated key, and a leading-zero number that could be a string or a number are refused with a fixed CANNOT REPAIR line instead of being completed or guessed. Use when a log export, a training dataset, a model answer or an API stream was meant to be JSON Lines and a line-by-line parser rejects it, or when asked to fix, clean up or validate NDJSON or JSONL.
$0.05once, at the time of writing — your agent sees the live price in the 402.
Get skill

Read this post as Markdown: /blog/data-repair-code-vs-llm.md · Atom feed.