Agent failure labs / Guide 56
AI Agent Spending Limits: Two Safe Purchases Break One Budget
Reproduce an AI agent spending limit breach from two individually valid requests, then enforce shared reservations and stable operation bindings before any spend.
Illustrative engineering guide. Examples are not client deployments, production-ready integrations, or evidence of measured results. Validate the design with your domain and security owners.
“Purchase both data exports, but stay within the team’s remaining budget.” In this hypothetical workflow, two agent branches each request sixty dollars. The team has one hundred dollars left. Each branch checks the same balance and concludes that its own purchase is affordable. Together they authorize one hundred twenty dollars, although neither branch violates its individual comparison.
This lab targets aggregate spending authority rather than invoice extraction or general cost optimization. The failure appears whenever several attempts share a budget: parallel tools, repeated retries, scheduled jobs, or multiple users. A spending limit is not a sentence in a prompt. It is an accounting invariant that must survive overlapping requests and unknown outcomes.
Reproduce the Shared-Balance Mistake
The unsafe mock evaluates each request against the unchanged initial balance. The fixed version tracks reservations keyed by operation, subtracts outstanding commitments, and rejects reuse of an operation identifier with a changed amount. The fixture uses integer USD minor units, with a ten-thousand-unit cap and two six-thousand-unit proposals.
This is a deterministic synthetic simulation, not an actual LLM experiment, concurrency benchmark, or payment test. The list represents overlapping intentions, but the guarded function runs sequentially in memory. Nothing calls a provider or spends money. Real multi-worker correctness requires a transactional implementation beyond this pedagogical ledger.
from copy import deepcopy
budget = {"limit": 10000, "settled": 0, "reserved": {}}
requests = [("export-a", 6000), ("export-b", 6000)]
def naive(state, proposals):
available = state["limit"] - state["settled"]
return [op for op, amount in proposals if amount <= available]
def reserve(state, operation, amount):
if type(amount) is not int or amount <= 0:
return "invalid_amount"
if operation in state["reserved"]:
return "already_reserved" if state["reserved"][operation] == amount else "operation_conflict"
available = state["limit"] - state["settled"] - sum(state["reserved"].values())
if amount > available:
return "budget_exceeded"
state["reserved"][operation] = amount
return "reserved"
assert naive(budget, requests) == ["export-a", "export-b"]
assert sum(amount for _, amount in requests) > budget["limit"]
fixed = deepcopy(budget)
assert [reserve(fixed, op, amount) for op, amount in requests] == ["reserved", "budget_exceeded"]
assert reserve(fixed, "export-a", 6000) == "already_reserved"
assert reserve(fixed, "export-a", 6500) == "operation_conflict"
assert reserve(fixed, "export-c", -1) == "invalid_amount"
assert sum(fixed["reserved"].values()) + fixed["settled"] <= fixed["limit"]
assert budget["reserved"] == {}
print("Aggregate cap preserved; duplicate reservation did not consume twice.")Rerunning the original two-request fixture produces one reservation and one denial. The repeated operation does not reserve another sixty dollars. A changed amount under the same identifier fails instead of silently increasing the reservation. These assertions expose the invariant directly, without relying on a model’s estimate of whether it has spent too much.
Why a Budget Prompt Cannot Coordinate Workers
A prompt can encourage economical choices but does not serialize shared state. Even two perfectly obedient agents can observe the same old balance. A malicious tool result can also suggest splitting a large task into small purchases, each below the per-call limit. The total authority must apply across the parent task and all delegated children.
Define the budget scope explicitly: tenant, cost center, currency, period, and responsible owner. A per-operation cap and a daily aggregate cap are different controls. Decide whether tax, shipping, retries, exchange-rate uncertainty, and cancellation fees consume the budget. If a provider cannot bound a call’s maximum charge, abstain or require a separately approved conservative ceiling rather than pretending the estimate is exact.
OWASP authorization guidance recommends least privilege and checking permissions for each request. The spending service should derive its budget scope from server-side identity and entitlements. A model-provided cost-center identifier is a candidate reference, not evidence that the requester may use that account or move funds between periods.
Make Reservation an Atomic Decision
In production, persist the operation binding and reserve capacity in one database transaction. Competing workers must not both pass a read-then-write check. Lock an appropriate budget row or use a carefully designed serializable transaction, with a unique operation constraint. Verify that every write path, including administrative imports, respects the same invariant.
PostgreSQL transaction isolation documentation explains that Read Committed can observe different snapshots across statements, while Serializable emulates serial execution for committed transactions and can require retries. The in-memory example does not implement those semantics. Database integration tests must deliberately interleave reservations and verify the final aggregate rather than merely exercise a single request.
Bind approval to the operation, provider, maximum amount, currency, budget period, and expiry. OWASP transaction authorization guidance recommends protecting significant transaction details and checking authorization at execution. A changed provider or increased maximum should require a new approval even when a reservation still fits the overall limit.
One accounting service should own reservations for all agents. A specialist cost estimator can provide bounded proposals for variable workloads, but it must not maintain an independent balance or issue spending permissions. More planning agents should increase the number of competing proposals, not multiply the budget. The smallest useful architecture centralizes authority while leaving estimation replaceable.
Unknown Does Not Mean Free Capacity
After submission, move the operation through explicit reserved, submitted, unknown, settled, or released states. A timeout means unknown, not failed. Keep the reservation until an authoritative provider lookup or documented expiry establishes the actual liability. Releasing immediately on timeout can authorize a second purchase while the first charge is still processing.
On confirmed settlement, convert the reserved ceiling into actual cost atomically and release only the difference. Reject a settlement exceeding the approved ceiling for investigation; do not hide it by modifying historical approval. A generic status: success is not an accounting receipt unless its origin and operation, merchant account, amount, and currency match the submitted request.
Cancellation has its own semantics. A request to cancel may still incur a fee or race with fulfillment. Likewise, a reservation abandoned before any submission can be released under a documented expiry policy, while an unknown submitted operation cannot be treated identically. Finance should own the reconciliation queue and aging policy, including what happens across a budget-period boundary.
The code omits the authentication source, database transactions, persistent approvals, state persistence, provider semantics, actual charging, settlement, and release. It also assumes one currency and accurate bounded amounts. Deterministic enforcement cannot prevent overspend caused by incomplete liabilities, external purchases absent from the ledger, or provider charges beyond the modeled ceiling.
Acceptance Tests
- Two sixty-dollar proposals against one hundred dollars must produce at most one reservation, including under concurrent database execution.
- Repeating an identical operation must return its existing reservation without consuming additional capacity.
- Reusing that operation with a different amount, provider, or currency must fail with a binding conflict.
- Splitting one task across child agents must still consume the same parent budget and period.
- A provider timeout must retain the reservation until matching authoritative reconciliation resolves liability.
- A settlement and reservation release race must not make capacity available twice or erase incurred cost.
Measure Both Leakage and Lockup
Establish a baseline from a labeled replay of proposed purchases, including overlapping attempts and delayed receipts. Measure approved liability above the limit over all evaluated budget windows. Also measure valid requests denied due to stale reservations over all genuinely affordable requests. A system can prevent overspend by permanently locking the budget, but that is not operational success.
Start with a shadow reservation ledger and compare it with finance’s authoritative balances. Then pilot a small fixed-price provider with human approval and an adapter kill switch. Suggested guardrails include no unreserved submissions and bounded unknown-operation age, with thresholds agreed by finance. These are proposed controls, not claimed production measurements.
The expense approval guide covers policy review, while the refund verification lab applies balance evidence in the reverse direction. Backend and API engineering can implement the atomic ledger. Start with the two-request interleaving test; individual affordability checks will not reveal the shared-budget defect.