Plan what your AI support agent will cost to run
The short answer
A support agent doesn’t make one model call per reply. It rewrites the question, picks evidence, decides on tools, drafts and checks the answer, so a typical reply on a current mid-tier model costs about 3 to 11 cents in tokens alone. Model the whole loop before you choose a model, then add evaluation, hosting and engineering time, which a token estimate leaves out.
Count the loop, not the call
The usual first estimate takes one prompt, one answer and the price list. On GPT-6.1 Terra, a call with 6,000 input tokens and 400 output tokens costs $0.008. That number is correct, and it’s the wrong unit. A customer never sees a single call. They see a reply, and a reply that’s grounded, checked and allowed to use tools takes several calls.
A typical loop looks like this. A small step classifies the message and decides whether the agent should handle it. Another rewrites the question into a search query. After search, a call selects which passages actually support an answer. Another decides whether a tool is needed, such as an order lookup. Then the model drafts the reply, and a final call checks the draft against the evidence and your rules. That’s six calls, and each one carries its own copy of the instructions, the retrieved passages, the conversation so far and any tool results.
Context grows as the conversation goes on. The third reply in a chat carries the first two turns, plus whatever the tools returned. Timeouts, malformed tool arguments and failed checks cause retries, and a retried call costs the same as the first attempt. None of this is waste. It’s what makes the answer good. But it means the honest unit of cost is the reply, and the honest budget is replies per ticket times tickets per month.
A worked cost model you can redo
Here are the assumptions for a typical agent. Six model calls per AI reply. About 6,000 input tokens per call (instructions, retrieved passages, conversation and tool results) and about 400 output tokens. Add 10% for retries. That gives 6.6 calls, 39,600 input tokens and 2,640 output tokens per reply. For the cached rows, assume half of the input is a stable prefix served from the provider’s prompt cache. Ignore the cache write premium, which is small at steady volume when the same prefix is reused within its lifetime.
Prices come from the providers’ official pages, checked on 6 November 2026. They change often, so check them again before you rely on them. Claude Sonnet 5.5 costs $2 per million input tokens, $10 per million output tokens and $0.10 per million for cache hits. GPT-6.1 Terra costs $1 per million input tokens, $5 per million output tokens and $0.05 per million for cached input. Both are standard, short-context rates.
| Scenario (per AI reply) | Tokens in / out | Sonnet 5.5 | Sonnet 5.5, 50% cached | GPT-6.1 Terra | GPT-6.1 Terra, 50% cached |
|---|---|---|---|---|---|
| Lean: 2 calls, 4,000 in, 300 out | 8,800 / 660 | $0.024 | $0.016 | $0.012 | $0.008 |
| Typical: 6 calls, 6,000 in, 400 out | 39,600 / 2,640 | $0.106 | $0.068 | $0.053 | $0.034 |
| Heavy: 10 calls, 10,000 in, 500 out | 110,000 / 5,500 | $0.275 | $0.171 | $0.138 | $0.085 |
To check a cell, multiply input tokens by the input rate and output tokens by the output rate, then divide by a million. Typical on Sonnet 5.5 without caching: 39,600 × $2 plus 2,640 × $10, which is $0.0792 + $0.0264 = $0.1056. With half the input cached: 19,800 × $2 plus 19,800 × $0.10 plus the same output, which comes to $0.0680.
Tickets need more than one reply. Assume a chat ticket takes about three AI replies and an email ticket about one and a half, because email customers tend to put everything in the first message.
| Typical agent, per ticket | Sonnet 5.5 | Sonnet 5.5, 50% cached | GPT-6.1 Terra | GPT-6.1 Terra, 50% cached |
|---|---|---|---|---|
| Chat (3 AI replies) | $0.32 | $0.20 | $0.16 | $0.10 |
| Email (1.5 AI replies) | $0.16 | $0.10 | $0.08 | $0.05 |
Northstar’s team multiplied the $0.008 call by 30,000 replies. Their real agent ran the typical loop on GPT-6.1 Terra without caching, so each reply cost about $0.053 and the month cost about $1,580. Nothing was broken. The plan priced a call, and the invoice priced a reply.
Cut the bill without cutting quality
Not every step needs a mid-tier model. Classifying a message, rewriting a query and choosing passages are narrow tasks that a small model often handles well. GPT-6 Luna costs $0.10 per million input tokens and $0.50 per million output tokens on OpenAI’s pricing page, a tenth of Terra. If the first four calls of the typical loop run on Luna and only the draft and the check run on Terra, both with half the input cached, the typical reply falls to about $0.014. Move a step to a smaller model only when it passes the same tests as before.
Caching only works when the start of the prompt is identical from call to call. Put the stable parts first: instructions, tool definitions and policy text. Put the parts that change last: retrieved passages, the conversation and tool results. Both providers price cache reads at a small fraction of normal input, and both charge a premium to write the cache, so caching pays off when the same prefix is reused within its lifetime. Anthropic’s default cache lasts 5 minutes, OpenAI’s current models keep entries for at least 30 minutes, and each provider has a minimum prompt length below which nothing is cached.
Costs a token estimate misses
Token prices are the easiest line to estimate and rarely the largest. Before you compare a self-built agent with a vendor, list the rest.
| Cost | Why it adds up |
|---|---|
| Embeddings and reranking | Every knowledge change re-embeds content, and every reply may call a reranker on top of the model calls. |
| Evaluation runs | Each release reruns your test set, often with an LLM judge scoring every answer. That’s the full loop again, many times a month. |
| Logging and storage | You need traces of every call to explain a bad answer, and they grow with every reply. |
| Hosting | Workers, queues, a vector store and a database run whether or not traffic arrives. |
| Engineering and on-call time | Model deprecations, prompt regressions, helpdesk API changes and incidents need a person. At 0.2 of an engineer costing $7,500 a month, that’s $1,500 a month, or $0.05 per reply at 30,000 replies. |
Be fair about what building can achieve. A lean, well-cached agent on a mid-tier model can cost under a cent per reply in tokens. The point isn’t that model calls are expensive. It’s that the loop, retries, growing context, testing and engineering time multiply a small number into a large one, and most first budgets only count the small number.
Compare with a per-reply price fairly
My AskAI charges $0.05 per AI reply on the Scale plan and $0.06 on Pro, plus $0.02 per reply when a reply uses tasks or tools. That price covers the whole reply, not one call. Behind it, the agent makes around 10 model calls per reply, including query rewriting, reranking, groundedness checks and validation. Its cost model has been tuned for a long time: stable context is cached, and different steps go to different models, with small, fast models such as GPT-6 Luna or open-weight models for routing and classification, and larger models only where they improve the result. Hosting, retries, evaluation and maintenance are included.
Compare like with like. Put your own loop, retry rate and cache hit rate into the table above, then add the costs a token estimate misses. To run the full comparison with your ticket volume and team costs, use the build vs buy calculator on this playbook’s home page.
Ask AI to explain this guide
Build decisions and checks
Use these decisions to turn the worked case into your own implementation. A check is complete only when you have evidence for the stated result.
Model the cost per reply before choosing a model
Required. Pricing one model call hides the rewrite, evidence, tool, draft and check calls behind every reply.
Build. List every model call in one reply with its input and output tokens, add a retry allowance, and multiply by replies per ticket for chat and email. Price it on each candidate model from the provider's official page and record the date you checked.
Check. A worksheet that anyone can recompute shows tokens per reply, cost per reply and cost per ticket for each candidate, uncached and cached.
Route each step to the cheapest model that passes
When it applies. Running every step on a large model pays for capability that classification and query rewriting don't need.
Build. Give each step its own model setting. Try a smaller model for routing, classification, query rewriting and evidence selection, and keep a stronger model for drafting and checking where tests show it earns its cost.
Check. Each step's model passes the same held-out test cases as before the change, and the per-reply cost is recorded alongside the result.
Cache stable context
When it applies. Sending the same instructions and tool definitions in every call pays full input price for text that never changes.
Build. Order prompts with stable content first (instructions, tool definitions, policy text) and changing content last (passages, conversation, tool results). Enable the provider's prompt caching and check the minimum cacheable length and cache lifetime.
Check. Usage reports show cache reads on repeat calls, and the measured cache hit rate matches the rate in your cost model.
Set budgets, alerts and per-conversation caps
Required. A retry loop, a long conversation or a traffic spike can multiply spend before anyone notices.
Build. Set a monthly budget and spend alerts at the provider and in your own metrics. Cap model calls and tokens per reply and per conversation, and transfer to a person when a cap is reached. Include evaluation runs in the budget.
Check. A test conversation that hits the cap stops calling the model, transfers cleanly and raises an alert.
Ask AI to apply these decisions
Templates and examples
Every file uses synthetic data. Adapt it to your own policies and test it somewhere safe before it touches customers.
Keep it working
Launch is the start of the work, not the end. These are the recurring jobs this part of the agent needs, and who usually owns them.
| Cadence | Owner | Work |
|---|---|---|
| Monthly | Engineering | Recheck provider prices and update the worksheet whenever the loop or the model changes. |
| On change | Engineering | Retest routing choices when a new small model ships or a step's failure rate rises. |
| On change | Engineering | Watch the cache hit rate after every prompt or tool change, since a changed prefix stops reuse. |
| Weekly | Engineering | Compare actual cost per reply and per ticket with the model and investigate any drift. |
Test your decision
Your prototype makes one call per reply and costs $0.008. Before launch, you add query rewriting, evidence selection and a draft check. What should you do to the budget?
Show the expected decision
Recalculate it per reply, not per call. Count the new calls, the input each one carries and a retry allowance, then multiply by replies per ticket and tickets per month. Add the evaluation runs each release will need. A 4- to 6-call loop on the same model can cost four to seven times the prototype’s figure.
