The Hidden Cost of Uncapped AI Agent Loops

DEV Community

An AI agent with no step limit stops when its context window is full. On GPT-5.6 that's $160 for one run.

The first agent most of us write is a while True loop. Ask the model what to do, run the tool it picks, append the result to the history, ask again. It stops when the model says "done". Usually it does. Sometimes it doesn't: it reads the same file again, retries a failing test forever, or keeps searching for something that isn't there. If you have no step limit, that loop still ends eventually. It ends when the history no longer fits into the model. I simulated how much that one run costs, and the number is set mostly by the size of the context window. A few words first A token is a chunk of text the model reads as one piece; APIs bill per million of them. Input tokens are what you send, output tokens are what the model writes back, and output is priced higher. The context window is the most tokens one request can hold; a request that's bigger is rejected. An agent loop is code that calls the model repeatedly, letting it choose tools, and sends the full history with every call, because the API itself remembers nothing between calls. A step cap is a hard limit on how many times the loop may go around. The setup Every step resends everything so far and adds two things: the model's tool call, which I set at 200 output tokens, and the tool's result. For the result I measured a typical action, "read 100 lines of a file", by cutting four Python standard library modules into 100-line chunks and counting them with tiktoken (o200k_base, the encoding behind current GPT models). The median chunk was 802 tokens, so I used 800. The loop starts with 4,000 tokens of system prompt, tool descriptions and task. So each step makes the next request 1,000 tokens bigger. The loop runs until the next request plus its answer would overflow the window. Prices are official list prices from the catalog of the LLM gateway I work on, snapshot of 12 November 2026, with no discounts applied. Model (window) Steps until full Runaway run With a 25-step cap GPT-5.6 mini (400K) 396 $16.05 $0.09 GPT-5.6 (400K) 396 $160.54 $0.86 Claude Haiku 5.5 (1M) 996 $50.05 $0.04 Claude Sonnet 5.5 (1M) 996 $1,000.98 $0.85 Claude Opus 5.5 (1M) 996 $2,001.96 $1.70 On GPT-5.6 the runaway run costs 187 times what a capped one does. On the 1M-token models it's 1,178 times. The last step alone on Sonnet 5.5 costs $2, more than the whole capped run. Why the window sets the price Step n sends about n thousand tokens, so the total is 1 + 2 + 3 + … up to the last step. That sum grows with the square of the number of steps, and the number of steps is the window divided by the growth per step. Put together, a runaway run sends roughly window² ÷ (2 × tokens added per step) input tokens: 400,000² ÷ 2,000 = 80 million on GPT-5.6, almost exactly the 79.8 million the simulation counted. That's why a bigger window is a bigger risk. At the same $2/$12 price, I ran the loop with three window sizes: 200K costs $40.26, 400K costs $160.54, 1M costs $1,001.38. Doubling the window made the run 4.0 times more expensive; 2.5 times the window made it 6.2 times more. A model upgrade that "only" raises the window from 200K to 1M multiplies your worst case by 25 without touching the price per token. The fix is two counters Count steps, and count dollars from the usage field every response already returns. Here is the guard, written for OpenAI-style responses: PRICE_IN, PRICE_OUT = 2.00, 12.00 # $ per 1M tokens, your model's list price
MAX_STEPS, MAX_USD = 25, 1.00

def run_agent(client, model, messages, tools, execute_tool):
spent = 0.0
for step in range(MAX_STEPS):
r = client.chat.completions.create(model=model, messages=messages, tools=tools)
u = r.usage
spent += (u.prompt_tokens * PRICE_IN + u.completion_tokens * PRICE_OUT) / 1e6
msg = r.choices[0].message
messages.append(msg)
if not msg.tool_calls:
return msg.content, spent
if spent >= MAX_USD:
raise RuntimeError(f"budget hit after {step + 1} steps: ${spent:...

Powered by Readybot.io