# Flaky Test Review Flaky Test Review is a tested SKILL.md that reviews ONE test file for the five causes that make a test pass on one run and fail on the next, and lists only those, one line each with a fixed code, the test, why it flips and the fix - or exactly No findings; an agent buys it once for $0.01 over x402. - Page: https://aiskills402.com/skills/flaky-test-review - Category: Code & Engineering (https://aiskills402.com/categories/code) - Price: $0.01 once, USD-priced, paid in USDC on Base over x402. Price as loaded on this page. The 402 response your agent receives is authoritative. - Version: 1.0.0 - Card (JSON): https://api.aiskills402.com/v1/skills/flaky-test-review ## Use it when Reviews ONE test file for the five causes that make a test pass on one run and fail on the next, and lists only those, one line each with a fixed code, the test, why it flips and the fix - or exactly No findings. It catches a fixed sleep used to wait for background work, an expectation tied to today's date or the current time, tests that share state and only pass in one order, random values with no fixed seed, and a call to a real network address. It leaves out the safe look-alikes, such as a fake clock set before the call, a random generator built from a fixed seed, a mocked network module and fake timers moved by hand. Use to find why a test fails once in a while in CI, to review a new test file before merging, or to harden a suite that people re-run until it goes green. ## Not for Tests that cannot fail, test speed, coverage, automatic reruns in CI and fixtures kept in other files. It reads one pasted test file and judges only what is written there, not the code under test. ## Tested, honestly Tested 2026-10-26. - Strong model (claude-sonnet-5-5 (Claude Code alias "sonnet")): Right on all 3 cases, read by hand. On the pytest file it found the three-second pause and the module-level list that the second test relies on, and left out the seeded generator and the fake clock fixture. On the Jest file it found the expected month tied to today and the real rates address, and left out the seeded winner test. On the clean Jest file, with its frozen clock, mocked module, fixed seed and hand-moved timers, it answered exactly No findings. - Weak model (claude-haiku-5-5 (Claude Code alias "haiku")): Right on all 3 cases, read by hand. It matched the strong model on both planted files, with longer reasons that add the case of running a test alone or in random order, and kept every safe look-alike out. On the clean file it answered exactly No findings. One answer was indented by four spaces, which the check accepts. Note: Light test: three cases (two planted, one clean), one run per model; no comparison without the skill. The three inputs were written by us; each planted one carries a safe look-alike and neither planted case has a comment pointing at the defect. The public source was read only through its reader replies, not the article body, so the skill claims little from it; the five codes are our own list. There are no owner-measured facts. Full summary: https://aiskills402.com/skills/flaky-test-review/tests ## Example ### English (claude-sonnet-5-5) Before: > import random import time from app.jobs import start_export, export_done from app.reports import build_report from app.lottery import draw items = [] def test_adds_first_item(): items.append("a") assert items == ["a"] def test_adds_second_item(): items.append("b") assert items == ["a", "b"] def test_export_finishes(): job = start_export() time.sleep(3)… After: > FIXED-SLEEP — test_export_finishes, `time.sleep(3)` — on a slow runner three seconds may not be enough, and on a fast one it only wastes time — poll `export_done(job)` in a loop with a timeout instead of sleeping… ## What is in the file - What the public source says, as read on 26 October 2026 - The answer - The codes - Rules - Work in this order - Short example ## Languages English ## How to buy Agent (HTTP): 1. GET https://api.aiskills402.com/v1/skills/flaky-test-review/file without a payment header. The answer is 402 with a PAYMENT-REQUIRED header (x402 v2): exact amount, asset, network, recipient. 2. Sign `accepts[0]` with an x402 client (for example @x402/core + @x402/evm). 3. Repeat the GET with the signature in the PAYMENT-SIGNATURE header. The answer is 200 with the file, its sha256 and a re-download token. Agent (MCP): https://mcp.aiskills402.com/mcp — free tools search_skills, get_skill, redownload_skill. Buying itself is over HTTP. Full flow: https://aiskills402.com/docs ## The file - Version: 1.0.0 - Size: 5.7 KB (5868 bytes) - SHA-256: 694db5737df5f70a66c724954574c1ac38e069393d6618af782c84ed03af0acb - Updated: 2026-10-26 - New versions are free through your re-download token. ## Versions ### 1.0.0 (2026-10-26) First release, written in the second light-test batch of 26 October 2026 (three cases, price $0.01). The idea is ours; no foreign skill was opened, facts come from a public testing blog post (see notes/ideas.md). Output: see the skill description. Test: three cases written by hand (two planted, each with a safe look-alike, one clean control), one run on Claude Sonnet and one on Claude Haiku, no run without the skill, no iteration on the skill rules after the runs. `test/control.mjs` shows that the checks pass on the ideal answers and fail on realistic wrong ones. See `test-summary.json` for the verdicts read by hand. ## License Perpetual, non-exclusive; use and modify for yourself incl. paid work; no resale or republishing. Holder: Georgi Kalchev, aiskills402.com. Terms: https://aiskills402.com/docs#license ## FAQ ### Frozen time set in a setup hook: does it count as a clock problem? No. A fake clock set before the call is the safe form, and so are fake timers that the test moves by hand. Only an expectation that reads the real date or time, with nothing earlier in the file to freeze it, gets a line. ### Where do the five causes come from? They are the skill's own list. The Google Testing Blog post names random numbers and absolute timing among the causes, but our fetch returned only the reader replies, so the text claims no more than that and gives no rate or number. ### Which files went through the trial? A pytest file with a list shared across tests and a three-second pause, a Jest file with a month in the expected text and a real rates address, and a clean Jest file with mocks and a fixed seed. Sonnet and Haiku each read all three once. ### What does a stable file return? Only the sentence No findings. A cause is reported on one line: the code, the test, the reason it flips and the repair. ## Measurement limits - Models other than the two named above were not run. - Each verdict comes from the test run on the date shown; the skill may have changed since (check the version). - Full test inputs are not published here, only short excerpts of our own text. - Results on your own texts, languages and domains can differ. Offer note: Paid in USDC (USD-pegged) over x402 by an AI agent; one-time.