A test that can never fail measures nothing

A test is a question you put to your code, and the only useful question is one whose answer could be no. If a check stays green whether the code works or not, it tells you nothing, and it does something worse than nothing: it ends the search. When something is wrong in production and the tests are green, people stop looking at the code that the tests claim to cover. Checks like that are easy to write by accident, so below are the usual ways a check ends up in that state, followed by a plain method for proving that yours is not.

The one question to ask of every check

Pick any assertion and picture the guarded behaviour broken at this very moment. Would the assertion still pass? That is the whole method. It sounds trivial, yet most checks of this kind were written by careful people who tried the happy path, saw green, and moved on. Green on a working system carries no information, because a check that always passes also passes there.

The formal version of the idea is mutation testing. You make a small change to the program, and each changed copy is called a mutant. If a test fails on it, the test noticed the difference and the mutant is said to be killed. A suite is judged by how many mutants it kills (Wikipedia, Mutation testing). A tool is optional. Break the code by hand, in precisely the way the test claims to prevent, and look for red. If nothing turns red, the check is decoration.

Ways a check cannot fail

A completeness guard that reads its own list

A frequent guard promises that every route, module or table is covered. Inside the test sits an array of names, the test loops over it, and a final assertion fixes the number of entries. Then a developer adds a new route. The application changes, but the array and its size stay as they were, so the guard remains green in exactly the situation it was built to catch. The guard measures itself, not the application.

The repair is to let the list be the real thing: read the directory, ask the router for its table, or use a glob that the bundler expands at build time. Keeping a count is fine as a second, deliberate step, a moment where a person confirms the new item was looked at. Prove it by creating the new item and watching the guard turn red.

A pattern that looks only at one level

A guard that finds its files with a pattern like src/*.ts looks at one level. In Python, the double-star form descends into subdirectories only when recursion is switched on (Python documentation, glob). Other tools have their own rules, so read the manual of yours. The danger shows up later, when a subfolder appears and the guard keeps passing because it never sees inside. Place a sample file in a nested folder and confirm the guard reports it.

A not-null test on a value that can be blank

Asserting that a token is not null feels like asserting that a login worked. But an empty string is not null, and neither are the number zero or a list with no members. A failed login that returns the field with nothing in it still passes. Measure how long the value is, or compare it with the full expected form, and look at the neighbouring fields too, since a half-filled record is the typical broken state and it sails past a null test.

An unauthenticated request to a protected address often receives a login form with a success code, particularly when the client follows redirects. The code tells you that some page was served, not which one. Attach the session, stop the client from following redirects, and look for a phrase that only the genuine page contains.

A waiting loop with no pause

Consider a loop that polls a deployment a fixed number of times and then reports that ten minutes passed. If every pass finishes almost instantly and nothing sleeps between passes, the loop is over in moments. It measured how quickly the network answered, not how much time elapsed. Pause in every pass, read a clock for the deadline, and print the elapsed time you actually measured.

A test that walks around the call site

You repair a bug in a pure function and write a test that feeds it the corrected input. That proves the arithmetic, which was never in doubt. The bug lived in the place that decides what gets passed in. Restore the old wrong argument there and the test is still green. Exercise the handler that makes the call, and add a check on the source text so the right arguments are required by name.

A control that never creates its condition

A counterfactual control says that with the wrong fix, this test must fail. It works only when the seeded data produces a difference. If two clocks differ by a few seconds while events are a minute apart, ordering by one clock or the other yields an identical sequence, so broken and repaired code behave alike. Before relying on such a control, assert that the difference you depend on is present in the data.

One fix in two branches, one test

When the same protection exists in a manual path and a scheduled path, or in two routes, a test of only one path lets the other lose the protection unnoticed. Run the test against each path, then remove the protection from each in turn and expect red every time.

A limit exercised far from its edge

A cap, a batch size or a platform ceiling is rarely reached by everyday test values. Feed an absurd one: a cap of five instead of the production number, a list longer than the batch size, one more event arriving from an already-seen source. If the guard never fires inside your test, nothing shows that it exists.

A catch block that converts failure to success

A try block that swallows an error and returns an empty list, a zero or true makes an unreachable host look healthy. Let the error count as an error, or pass it upward, and write down which item failed.

Breaking things on purpose, without losing your work

Breaking code deliberately is the right proof, but the script that does it has its own trap. Suppose it puts the file back after each mutation with a version-control command. The documentation for git restore says that the content comes from the index by default, or from the last commit when the staged option is used (Git documentation, git-restore). A repair that is not staged is overwritten by a plain restore, because the index still holds the old text; and if the script restores from the last commit, as the staged option does, even a staged but uncommitted repair is lost. Every later mutation then runs against code that lacks the repair, and the mutants that survive look like proof of a well-covered guard.

A safer routine keeps a duplicate made at the start and puts it back between mutations. Let the script stop with an error whenever a mutation left the file untouched, and finish by comparing file fingerprints so you know the file you started with is the file you finished with.

A short routine you can follow

Name what the check guards, in one sentence.
Break exactly that, in the smallest way you can.
Run the check. If it stays green, stop: you have found one that cannot fail.
Put the code back from a duplicate, never from the repository, and confirm it is back.
If the check covers several places, break it in every place, not only the one you tried.
For a completeness guard, add a new item inside a subfolder and expect red.

When this does not apply

Our advice targets checks that exist to catch regressions. It does not cover exploratory tests, benchmarks or demonstration scripts, where a pass is not offered as evidence. Some weak points above exist only in certain tools: how a glob treats nested folders, what a restore command takes its content from, and whether redirects are followed all differ, so confirm against the manual of your own stack. Breaking code by hand is slow on a large suite, and automated mutation testing can produce mutants that no test could ever tell from the original, so a surviving mutant is a question, not always a defect. We ran no measurements for this piece: the list is drawn from public documentation and from patterns we met in our own work, and it is not exhaustive.

Taking a green message as proof is the same mistake an agent makes when it claims success from a tool reply alone, a pattern we looked at in our report on agents that declare victory too early.

Reviewing a check before you trust it

When you want an outside read on a test, a smoke script, a script that breaks code deliberately or a pipeline step, the test-guard-review skill takes what you paste and looks only for the weaknesses covered above. Each finding comes with a short code, the location, the reason the check would stay green on broken code, and a specific input that turns it red. When it finds nothing it answers with one line saying so. Style, naming and the amount of code covered are outside what it judges.