Remotion Code Review: test results

Verdicts

Date

2026-10-29

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 3 cases, read by hand. On the stats scene it named the counter advanced by a timer inside an effect and the plain img tag of the chart, and left out the clamped interpolate, the seeded random bars and the Img component with staticFile. On the leaderboard it gave the fetch inside an effect with no render delay and the Audio source written as a bare public path, and left out the Math.random in the Node render script and the fetch inside calculateMetadata. On the clean promo it answered exactly No findings.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 3 cases, read by hand. It gave the same two lines as Sonnet on the stats scene and on the leaderboard, with the same safe look-alikes left out, and answered exactly No findings on the clean promo. One check was wrongly strict and is disclosed: on the leaderboard it suggested calculateMetadata as an alternative fix and named Root.tsx, and the first version of the check rejected any finding line that mentioned Root. The check was narrowed to the intro composition's own fetch and the six saved answers were re-evaluated without a new model call; Haiku's answer is correct.

Note

Light test: three cases (two planted, one clean), one run per model; no comparison without the skill. The code was written by us; each planted case carries a safe look-alike (a clamped interpolate, seeded random, Math.random in a Node script, a fetch in calculateMetadata), and neither planted case has a comment pointing at the defect. Facts come from the public Remotion documentation, read on 29 October 2026; there are no owner-measured facts.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.