Email and DNS Cutover Review: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 24 cases, read by hand: it found the bracketed staging prefix that Gmail drops, the sender name cut after the brand, two rules for one address, the routing log and one early test mail taken as proof, Promise.all and a plain loop over recipients, the visitor mail a send binding cannot deliver, foreign MX records, two SPF records, the DKIM record deleted with no copy, the DMARC call made by API token, the noindex element that only Yandex reads, and a reject policy or a strict SPF ending published before any report. It answered exactly No findings. for all eight sound plans and ignored a planted comment saying the module was approved. On the cutover log it also named a missing copy of the deleted MX records, which the paste does show; that extra finding was accepted.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 24 cases by content, but it missed the exact format twice: on the bracketed staging prefix it described the problem and the fix correctly but left out the code in brackets, and on the planted-comment case it found the Promise.all fault and ignored the comment, but it added a note after the verdict line, which breaks the exact reply. It answered exactly No findings. for all eight sound plans. On the cutover log it also named a missing copy of the deleted MX records, accepted as a true, shown finding.

With and without the skill

Tested 2026-10-08.

Same request on both sides; the bare side is scored on the concept in any words, and sound plans are not scored for content. Read by hand, Sonnet without the skill already knew foreign MX records, two SPF records, the DKIM delete with no copy, the early test mail, the routing log, Promise.all, two rules for one address, noindex and the strict mail policy. It missed the Gmail bracket rule and the cut sender name, and it advised an API token for the DMARC call, which we measured as refused (owner-measured, not re-checked). Haiku missed the same three. Both added remarks to nearly every sound plan instead of No findings.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Twenty-four mail and DNS plans or code snippets written by us (16 with a planted fault, 8 sound), scored by fixed codes, one verdict line, an exact No findings. for sound plans, and no fence. Re-checked in the vendor documentation on 2026-10-08: a send binding reaches verified destination addresses in the account (the page does not say what verified means, and the vendor's newer email service may allow other recipients; no answer argued that, and the check was not changed for it). Owner-measured and NOT re-checked: error 2008 for foreign MX records, the 10000 refusal of tokens for DMARC and DNS edits, Gmail dropping a leading bracketed prefix, the sender column cut at about twenty characters, the log staying empty for delivered mail. Checks widened after the run, for both sides: the cutover-log case now accepts a finding about the missing copy of the deleted MX records (true, shown in the paste), and the no-skill check for a strict mail policy now also accepts the soft-fail form of SPF and a confirmed sender list. One run per model and case.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.