Why inference isn't the whole cost
An "agent" typically plans, calls tools, reads results, and loops until it finishes a task. Each loop can involve several model calls, external API or search fees, and infrastructure to run it all. Budgeting only for the model's per-token price will understate the real total cost of ownership — often substantially.
Rule of thumb: model inference is a component of agent cost, not a synonym for it. Estimate every line below before you commit a budget.
The cost components
- Model inference: input and output tokens across every step of every run — usually several calls per completed task.
- Tools and search: external APIs, web search, code execution, or data providers the agent calls, each with its own fees.
- Orchestration and hosting: the servers, functions, or containers that run the agent loop and hold state.
- Storage: vector databases, caches, logs, and any documents or memory the agent retains.
- Evaluation: the model calls and engineering time spent testing that the agent behaves correctly before and after changes.
- Observability: tracing, monitoring, and logging tooling needed to debug non-deterministic behaviour.
- Human review: people checking, correcting, or approving agent output — frequently the biggest line for higher-stakes tasks.
Why agents multiply model calls
A single user request can trigger a planning call, one or more tool-use calls, calls to interpret tool results, and a final answer — plus any retries when a step fails. Because conversation history accumulates across steps, later calls in a run often carry larger prompts than earlier ones, so token cost per run can grow faster than the number of steps alone suggests.
Worked scenario (illustrative)
Illustrative assumptions — not a quote: a research agent completes 3,000 tasks/month; each task averages 6 model calls with growing context, 2 paid search calls, and a 90-second slice of a human reviewer's time.
The planning insight is where the money sits: 18,000 model calls, 6,000 search calls, and about 75 hours of human review a month. For many teams the review hours and search fees rival or exceed the token bill — so optimising the model alone would miss the biggest levers. Model out each component in the AI Agent Cost Calculator to see the full monthly total.
Controlling agent cost
- Cap loop iterations so a stuck agent cannot run up unbounded calls.
- Route by difficulty: use a cheap model for routine steps and a stronger model only for the hard ones.
- Prune context between steps instead of carrying the entire history forward.
- Reduce tool calls with caching and smarter planning — external fees add up fast.
- Design review out where quality allows, since human review is often the dominant cost.
Pricing and capabilities change. Provider rates, model line-ups, and features are updated by the providers over time. Any figures on this page are clearly labelled illustrative planning assumptions, not quotes. Official sources are checked on a recurring schedule. Unusual or structural changes are held for human review. See the Pricing Sources & Data Status page for verification dates, and always confirm current rates on the official provider pages below.
Official sources
Confirm current pricing and capabilities directly with each provider: