Remove the model and inspect what remains
A coding-agent workflow often looks healthy until the account reaches a usage limit. Then the queue disappears into chat history, nobody knows which change landed, a half-finished task has no owner, and recovery consists of prompting another model to reconstruct the story. The quota did not merely remove intelligence. It removed the process supervisor.
That is an architecture failure disguised as a billing event. Model access will always vary by provider incident, rate limit, budget, context window, account state, or deliberate cost control. The useful test is simple: turn off inference and ask what the system still knows. It should know what is queued, leased, waiting for approval, completed, failed, uncertain, and safe to resume.
The model should be a replaceable worker. It can reason about an ambiguous requirement, propose a patch, interpret a failure, or review an artifact. It should not be the only component remembering that the work exists.
Put the durable loop around the worker
The durable loop is ordinary infrastructure: admit a task, write its state, lease it to a worker, assemble bounded context, request judgment, validate the returned artifact, record a receipt, and advance or retry. Scheduling and recovery operate on the task record rather than on the model transcript.
A useful record includes the business job ID, workflow version, current state, lease owner and expiry, attempt count, input references, proposed artifact, approval status, external operation key, verification result, receipts, and next eligible action. The prompt may be disposable. The job record may not be.
State machines make the possible transitions inspectable. AWS Step Functions, for example, separates task states from retry and catch behavior and supports bounded backoff. The specific service is optional; the separation is not. A language model should not improvise retry policy from the emotional tone of the latest error message.
Let quota exhaustion degrade capability
When inference is unavailable, the orchestrator should stop leasing judgment-dependent steps and continue everything else it can prove safe. It can still receive work, deduplicate requests, expire stale leases, run deterministic tests, reconcile known external writes, assemble approval packets, notify an operator of a real exception, and preserve the exact restart point.
This is graceful degradation in business terms. The company temporarily loses some reasoning capacity, but it does not lose continuity. A code change may wait for diagnosis while its repository, failing check, last accepted artifact, and retry policy remain intact. A customer-facing action may remain held behind approval while intake and classification continue.
The stop condition matters. If the queue has no eligible deterministic work, the system should say that it is waiting for model capacity. It should not repeatedly spend calls asking a model whether model calls are available.
Verification is a separate job
Agent systems blur production and inspection when the same model creates an artifact and then declares it correct. Split the two. Mechanical checks belong to deterministic infrastructure: the test suite passes, the build artifact matches the commit, the expected record exists, the deployment reports the intended revision, or the API returned a known state.
Some verification still requires judgment. A product decision may be internally consistent but wrong for the customer. A migration may pass tests while creating an unacceptable operational burden. Make that a separate review step with its own budget, context, rubric, and receipt. It may use the same model family, another model, or a person, but it should not be hidden inside the production turn.
OpenAI's Agents SDK documents one concrete version of this boundary: a run can pause at an approval interruption, serialize its state, and resume later. The durable lesson is larger than the SDK. Waiting for judgment should be a first-class state, not an excuse to keep a server process or conversation alive.
Receipts make retries safe
A completed model response is not proof that an action landed. The workflow needs evidence from the system of record: a commit hash, check run, deployment ID, API object ID, message ID, or reconciliation result. Without that receipt, completed means only that the worker stopped talking.
External writes also need stable operation identities. Stripe's idempotency guidance shows the pattern: retries carrying the same key can return the first recorded result instead of creating a second object. Not every target supports idempotency, so the workflow may need a reconciliation state between unknown and retry. The dangerous design is a binary success-or-failure field that turns a lost acknowledgement into a duplicate action.
Receipts let a replacement worker continue without trust in the previous worker's narration. The new worker reads the job record, artifact, evidence, and unresolved question. It does not need to role-play the vanished conversation.
Run the model-loss drill
Test this architecture by exhausting the model budget on purpose. Seed the queue with work in several states: new, leased, waiting for approval, awaiting verification, externally uncertain, failed, and complete. Remove inference while one job is active. Restore it with a different worker or model.
The drill passes when every admitted task remains visible; expired work returns safely to the queue; deterministic checks continue; no external action is duplicated; approvals stay bound to the exact artifact; and the replacement worker receives bounded current context. Measure orphaned jobs, duplicate writes, time to honest degraded mode, successful resumptions, manual reconstruction minutes, and cost per verified completion.
Do not score the drill by whether the backup model can produce code. Score it by whether the business retained control of the work.
The model should reason about the work. It should not be the workflow.
When quota exhaustion pauses judgment but leaves state, safety, evidence, and recovery intact, the coding-agent loop has crossed the line from persistent prompting into operating infrastructure.
- Human-in-the-loopOpenAI Agents SDK, accessed August 7, 2026
- Handling errors in Step Functions workflowsAmazon Web Services, accessed August 7, 2026
- Idempotent requestsStripe, accessed August 7, 2026