Cold Email: test results

Verdicts

Date

2026-09-30

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

6 of 6 cases pass (en, bg, de, es, ru, thin brief). Right language, formal address, 60-110 words, one ask, easy exit, and no invented facts: with almost no brief it wrote a role-based reason and said it was writing without prior contact.

Weak · claude-haiku-4-5-20251001 (Claude Code alias "haiku")

6 of 6 pass the mechanical checks, but reading the texts shows real flaws: invents claims not in the brief ("most bakery owners I work with", "no massive minimums", "no surprise bills"), flatters ("great milestone"), ignores the formal-address rule in de/es/bg greetings (Hallo, Hola Laura, first-name only), leaves the Russian email without a greeting and with an English "Subject:" label, and on the thin brief does not say it is a first contact. Review its drafts before use.

With and without the skill

Tested 2026-10-03.

One run per model and case. Mostly length: 57 to 102 words with the skill, 137 to 297 without it. Sonnet's content was as good without the skill; Haiku without it used one stock phrase and once addressed a Bulgarian reader informally.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

One run per model and case (run 2 of 2; run 1 failed only because of a mistaken length check in our own cases, fixed). Checks are mechanical (language, kept names, banned phrases, a subject line, at most 139 words); they cannot detect invented claims, which we read by hand.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.