An agent loop is anything that keeps going without a person pressing the button: a timer that wakes a model, an agent that tries again until the task is complete, a batch that walks down a list. In testing such a loop looks fine, because testing happens when everything cooperates. Trouble comes when the scheduler hands over the same tick twice, a remote service turns down every call, or the event the loop waits for never occurs. This guide covers three structural properties that decide whether a loop can be left alone: whether a repeated step does harm, whether the loop has a hard ceiling, and who decides the work is complete. A section near the end is a set of experiments that are able to fail.
Repeats: assume every step can run twice
A step that has an outward effect, such as sending a mail, publishing a post, inserting a row or charging a card, will eventually run again. The trigger may fire twice, a worker may restart halfway, or a client may time out and retry. The Amazon Builders' Library makes the key point plainly: a timeout or failure does not prove that the side effects did not happen, and an interface with side effects is not safe to retry unless it provides idempotency, meaning the effect happens once no matter how often the call is repeated (Amazon Builders' Library).
The usual tool is an idempotency key: one unique value per intended action, sent with every attempt, which the receiving side remembers so it can recognise a repeat. Stripe documents this pattern: with the first version of its interface, a retry that carries the same key gets the saved result of the first attempt back, and keys are allowed to be as long as 255 characters (Stripe API reference). Two details matter for your own loops.
First, create the key one time, before the retry loop begins. A key generated inside it is new for every attempt, which defeats the purpose. Second, the idea carries over when you own both sides. Before acting, claim the unit of work, for example the pair of a time slot and a task name, using a database write that is ignored if the claim exists. If that write changed no row, somebody already holds it, so skip. Give the claim back when the action fails, otherwise a failed run blocks its own retry.
Order matters for progress markers too. If the loop records "done" only after the effect, a crash between those two steps makes the restart do the effect again. Put the marker first when losing one action is acceptable, or save the marker and the effect in one atomic operation. Whichever you pick, write it down, because "at least once" and "at most once" are different promises and a loop unsure of its own promise will break one of them.
Ceilings: an end condition is only a hope
Most loops have a way to finish: the tests pass, the list is empty, the model announces it is done. That is a wish about the future. It ends the loop only if it comes true, and the failures worth worrying about are those where it never does. Anthropic's write-up of agent design says the work usually ends when the task is complete, yet teams often add a cap, for instance on the number of rounds, to keep control (Anthropic, Building effective agents). Treat that cap as required equipment.
A useful ceiling has three parts: a count of attempts, a deadline in elapsed time, and a message saying why the loop stopped. The last part is easy to skip and it is the one you will want in the middle of the night. A loop that silently leaves after hitting its limit looks identical to one that finished, so the exit has to say which happened.
Retries deserve their own ceiling because they multiply. The Builders' Library gives an example of a call chain five layers deep with three retries at each layer; if every layer retries on its own, the load on the database at the bottom grows 243 times, which can delay recovery (Amazon Builders' Library). Their advice, for cheap operations, is to retry at one point in the stack, and to limit retries with a token bucket that lets calls retry while tokens last and slows to a fixed rate afterwards. For an agent, the layers are the model client, the tool wrapper and the outer loop. If all three retry, the total multiplies instead of adding.
Batches need a different kind of stop. Some errors belong to one item, such as a malformed record, and skipping it is correct. Other errors belong to the whole run: the account is blocked, the login does not work, the allowance is used up. Pressing on after one of these earns the same refusal for every remaining item, and may repeat a charge or a warning each time. Separate the two kinds in code, halt the whole batch when the latter happens, and say how many items finished, were passed over, or are still waiting.
Self-grading: the worker should not be the judge
The third property is who ends the loop. When the test is to put the question "are you done?" to the model, the same system that did the work declares it good. It has no new information, and a confident wrong answer ends the loop as well as a right one. Even a loop with a flawless ceiling can be judged this way.
Anthropic's guide names when a model-as-evaluator loop fits: the criteria are clear, and each extra round brings value you can measure. The same article stresses that an agent should obtain ground truth from its surroundings after every step, from tool output or executed code, to judge how far it has got (Anthropic, Building effective agents). In practice, finish on evidence that no amount of persuasion from the model can change: a test suite that ran, a count of items left in a queue, a value read back from the system that was changed, or a second reader that sees the output cold. A model can still help by drafting and critiquing, but the stop signal should come from evidence.
Two neighbouring risks belong here because they look like grading. If the loop carries more access than its job needs, such as a token that can delete when the job only reads, one wrong step does damage the job never needed. And if the loop does something irreversible or outward-facing, such as publishing or mailing many people, without any approval step or earlier consent saved where it can look, then nobody judged the result before it left. A person who starts a script by hand and watches its output is already a gate; an unattended loop needs the gate built in, for example by writing proposals and acting only on those a person marked approved.
One more failure: silence that looks healthy
A loop whose source has died often keeps running and reporting no errors. A blank reply, a count of nothing or a swallowed exception gets treated as success. The fix is to distinguish a call that failed from a reply that was legitimately empty, store when each source last gave a real reply, and sound an alarm once that moment is too old. Measure replies received, not attempts made. Parallel workers add their own version: two writers touching one shared file or counter can overwrite each other, so give each worker its own output and merge once, or use a single atomic update for counters.
Experiments that can fail
Do not ask the loop whether it is safe; break it deliberately and count.
- Fire one trigger two times and count the effects. The count should be one.
- Kill the process after the effect but before the marker, restart it, and count again.
- Give the loop an objective that is impossible. It should halt at its ceiling and explain.
- Let the first batch item fail with an error that affects the whole run. The batch should halt and report.
- Aim the monitor at a source you switched off. It should raise the alarm. A monitor you have never watched complain tells you nothing, because quiet and unable to speak look the same.
Run each of these once against the real code path, never a replica of the logic, since a repeat guard written as a separate function can be disconnected from the place that should call it without any test turning red.
When this does not apply
A script you launch by hand and watch needs far less of this; you are the ceiling and the gate. A step with no outward effect, such as a read or a local calculation, can safely repeat, so idempotency work there is wasted. Some systems already give the guarantees for you, for instance a queue that deduplicates by message identity, and adding a second layer may only add moving parts. The sources we cite describe one payment interface, one cloud provider's practice and one vendor's view of agent design; your platform may behave differently, so confirm how your scheduler and your external services treat repeats before relying on any of this. We ran no new experiments here; the text combines public documentation with common review practice, so read the steps as a method and not as measured results.
How a loop ends is closely related to how an agent reports that it ended, and our earlier article on a report of completed work that was never completed looks at the same problem from the reporting side.
Reviewing a loop before it runs alone
If you want an outside reader for a loop, our agent-loop-review skill takes a pasted description, configuration or code and reports structural problems, each tagged with a fixed code. The families it covers: effects that happen again on a repeated trigger, progress saved late, loops without a cap, a model marking its own work, batches that continue past a refusal, permissions wider than the job, irreversible steps without approval, dead sources that look alive, and parallel workers sharing one file. Every finding gives the location, the cause and a remedy, and the reply closes with a single verdict. Prompt wording and model choice are outside its scope.
Read this post as Markdown: /blog/agent-loop-repeats-and-ceilings.md · Atom feed.