Log Lines to JSON Lines, Odd Lines Kept Raw: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 24 log samples, read by hand: Apache access lines with dashes as null, offsets and escaped quotes, IPv6 clients, syslog with the nil value and structured data, logfmt with quoted values, custom patterns with trace ids and Unicode, numbers kept as numbers only where the format says so, every line that did not fit kept raw character for character, and the lines that spoke to the AI kept as data.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on 22 of 24 log samples, read by hand, but it missed two: in two Apache samples it kept a dash as the text "-" in the referer and ident fields where the format defines it as absent, so the value should have been null.

With and without the skill

Tested 2026-10-08.

Same request on both sides, a fence removed first. Sonnet without the skill already converted every sample right: dashes to null, numbers typed by the format, offsets kept, odd lines kept raw, planted lines ignored. The skill adds nothing for it on these samples. Haiku without the skill dropped three fields from one access line; with the skill it kept the dash as text twice instead of null, so it scored one lower with the skill than without.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-four log samples written by us (17 with a trap, 7 clean): Apache access lines, RFC 5424 syslog, logfmt and custom patterns stated in the task, with dashes that mean absent, time offsets, escaped quotes, IPv6, lines that almost fit, blank lines, Unicode and two lines addressed to the AI. Each answer is parsed line by line and compared field by field with the expected JSON Lines, types included, so "-" is not null and "200" is not 200. The format rules were checked on 2026-10-08 against the formats' own public descriptions (the Apache log format, the RFC 5424 grammar, logfmt). No check was widened. One run per model and sample.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.