Survey Question Bias Check: test results

Verdicts

Date

2026-10-26

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 3 cases, read by hand. On the customer-service list it flagged the tagged question, the fused price-and-delivery question and the always question, each with a neutral rewrite, and said nothing about the terms-and-conditions item or the usage-frequency item. On the office list it found the charged canteen word and the lopsided rating scale, and left out the matched agreement scale and the health-and-safety item. On the clean list it answered exactly No findings, so a never option and an easy-or-difficult stem passed.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 3 cases, read by hand. It found the same problems as the strong model on both planted lists and kept every safe look-alike out. On the clean list it answered exactly No findings. Its rewrites are longer, and one of them offers a scale of its own, which the check allows because the line still starts with the question number and the code.

Note

Light test: three cases (two planted, one clean), one run per model; no comparison without the skill. The three inputs were written by us; each planted one carries a safe look-alike and neither planted case has a comment pointing at the defect. Facts come from a public survey-methodology guide, read on 26 October 2026; the absolute-word and scale-balance rules are our own. There are no owner-measured facts.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.