Reviews ONE test file for the five causes that make a test pass on one run and fail on the next, and lists only those, one line each with a fixed code, the test, why it flips and the fix - or exactly No findings. It catches a fixed sleep used to wait for background work, an expectation tied to today's date or the current time, tests that share state and only pass in one order, random values with no fixed seed, and a call to a real network address. It leaves out the safe look-alikes, such as a fake clock set before the call, a random generator built from a fixed seed, a mocked network module and fake timers moved by hand. Use to find why a test fails once in a while in CI, to review a new test file before merging, or to harden a suite that people re-run until it goes green.
Flaky Test Review is a tested SKILL.md that reviews ONE test file for the five causes that make a test pass on one run and fail on the next, and lists only those, one line each with a fixed code, the test, why it flips and the fix - or exactly No findings; an agent buys it once for $0.01 over x402.
Not for
Tests that cannot fail, test speed, coverage, automatic reruns in CI and fixtures kept in other files. It reads one pasted test file and judges only what is written there, not the code under test.
Tested, honestly
Tested 2026-10-26 with a strong and a weak model.
- SonnetStrong model, claude-sonnet-5-5
- Right on all 3 cases, read by hand. On the pytest file it found the three-second pause and the module-level list that the second test relies on, and left out the seeded generator and the fake clock fixture. On the Jest file it found the expected month tied to today and the real rates address, and left out the seeded winner test. On the clean Jest file, with its frozen clock, mocked module, fixed seed and hand-moved timers, it answered exactly No findings.
- HaikuWeak model, claude-haiku-5-5
- Right on all 3 cases, read by hand. It matched the strong model on both planted files, with longer reasons that add the case of running a test alone or in random order, and kept every safe look-alike out. On the clean file it answered exactly No findings. One answer was indented by four spaces, which the check accepts.
Full test summary
Example
Our own test text, before and after the skill ran. Excerpts only.
English · claude-sonnet-5-5
Before
import random
import time
from app.jobs import start_export, export_done
from app.reports import build_report
from app.lottery import draw
items = []
def test_adds_first_item():
items.append("a")
assert items == ["a"]
def test_adds_second_item():
items.append("b")
assert items == ["a", "b"]
def test_export_finishes():
job = start_export()
time.sleep(3)…
After
FIXED-SLEEP — test_export_finishes, `time.sleep(3)` — on a slow runner three seconds may not be enough, and on a fast one it only wastes time — poll `export_done(job)` in a loop with a timeout instead of sleeping…
What is in the file
- What the public source says, as read on 26 October 2026
- The answer
- The codes
- Rules
- Work in this order
- Short example
Languages
English. Tried in: English.
License
Perpetual, non-exclusive; use and modify for yourself incl. paid work; no resale or republishing. Holder: Georgi Kalchev, aiskills402.com. Full terms.
Versions
Current version 1.0.0, updated 2026-10-26. Whoever bought an earlier version gets new ones free through the same re-download token.
v1.0.0 · 2026-10-26
First release, written in the second light-test batch of 26 October 2026 (three cases, price $0.01). The idea is ours; no foreign skill was opened, facts come from a public testing blog post (see notes/ideas.md). Output: see the skill description.
Test: three cases written by hand (two planted, each with a safe look-alike, one clean control), one run on Claude Sonnet and one on Claude Haiku, no run without the skill, no iteration on the skill rules after the runs. `test/control.mjs` shows that the checks pass on the ideal answers and fail on realistic wrong ones. See `test-summary.json` for the verdicts read by hand.
FAQ
Frozen time set in a setup hook: does it count as a clock problem?
No. A fake clock set before the call is the safe form, and so are fake timers that the test moves by hand. Only an expectation that reads the real date or time, with nothing earlier in the file to freeze it, gets a line.
Where do the five causes come from?
They are the skill's own list. The Google Testing Blog post names random numbers and absolute timing among the causes, but our fetch returned only the reader replies, so the text claims no more than that and gives no rate or number.
Which files went through the trial?
A pytest file with a list shared across tests and a three-second pause, a Jest file with a month in the expected text and a real rates address, and a clean Jest file with mocks and a fixed seed. Sonnet and Haiku each read all three once.
What does a stable file return?
Only the sentence No findings. A cause is reported on one line: the code, the test, the reason it flips and the repair.