Terraform Plan Risk Review: test results

Verdicts

Date

2026-10-26

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 3 cases, read by hand. On the orders plan it gave the replaced database and the database port opened to the internet, and left out the web port 443 rule, the instance swapped for a new image and the bucket tag edit. On the data-volume plan it gave the replaced disk, the encryption turned off and the server rebuilt only because its subnet id changes, and left out the database tag change and the subnet that was replaced for a new range. On the clean platform plan it answered exactly No findings.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 3 cases, read by hand. It matched the strong model on both planted plans, with sound fixes, and kept every safe look-alike out of its list: the stateless instance, the tag edits, port 443 and the subnet that was replaced for a new range. On the clean plan it answered exactly No findings.

Note

Light test: three cases (two planted, one clean), one run per model; no comparison without the skill. The three plan texts were written by us in the format of the Terraform plan page; each planted one carries a safe look-alike and neither planted case has a comment pointing at the defect. The list of resource types that count as stateful is the skill's own policy. Facts come from public Terraform documentation, read on 26 October 2026; there are no owner-measured facts.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.