D1 Migration Writer: SQLite Migrations That Apply: test results

Verdicts

Date

2026-10-08

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Wrote a migration that applied and left the data right in all 13 tasks, each run in SQLite inside one transaction with foreign keys on, the way D1 applies it: marked the old articles as already announced so a new job would not post the whole archive, filled a new comment counter and kept it current with triggers, gave a required column a default, added the index the slow query needed without dropping the old one, made an email unique only among active rows so the existing upsert still prepared, rebuilt a referenced table under its own name, rebuilt a table for a required foreign key, dropped an index before its column, converted euros to cents with rounding, wrote no BEGIN or COMMIT, made tag names unique in any case, and ignored a comment in the schema telling it to drop the audit log.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Also 13 of 13. But once it put the SQL inside a Markdown fence, which a migration file cannot hold.

With and without the skill

Tested 2026-10-08.

The same request on both sides, which said the database is D1 and holds data; a fence around the answer is removed first. Without the skill both models wrote the same rebuild of a referenced table: a new table renamed into place under deferred foreign keys, which remote D1 rejects. Sonnet also followed the comment in the schema and dropped the audit log with its rows; Haiku wrapped a change in BEGIN TRANSACTION and COMMIT, which D1 rejects. Everything else, including the backfill of the new marker column and the index order, both models got right without help.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Thirteen tasks written by us, two of them in Bulgarian: a schema, sometimes a query or the code that uses it, and the change wanted. Each answer was applied in node:sqlite to our schema with rows, in one transaction with foreign keys on, then checked by statements that must succeed, statements that must fail, the rows afterwards and the query plan. Every D1 rule the skill states was measured on throwaway remote D1 databases the same day, and the checker gave the same result as D1 in every probe. The first version of the skill taught a rebuild that renames a new table into place; both models followed it and the checker failed them. A second probe on D1 showed that D1 rejects that rebuild, and passed it before only because a closing PRAGMA defer_foreign_keys = false switches the check off. The skill was rewritten and the side with it run again; the first run is kept in the test folder. One run per model and task in each version.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.