Plan your AI support agent architecture
The short answer
A working AI support agent connects six things: helpdesk events, conversation state, authorized customer data, controlled tools, reply delivery and outcome measurement. Start with one support case you can handle end to end, and give every way it can fail a named owner and a defined next step.
Start with a service boundary
Before you compare models, write a one-page agreement about what the agent is for. It should say which questions the agent owns, which channels they arrive through, which languages it supports and how a customer reaches a person. Call out cases that look alike but need different authority. Reading an order’s status and changing its delivery address both start with “my order”, but the second one changes data and needs its own permission, checks and testing.
The fastest way to find the edges is to sit down with your support team and walk through five ordinary questions and five awkward ones. Awkward cases are where real conversations go sideways: a customer asks about two orders at once, writes from an old email address, or mentions a safety problem halfway through a delivery question. For each case, decide whether the agent answers, asks for more information, uses a tool or transfers the conversation. Anything the team can’t agree on yet becomes an explicit exclusion from the first launch, not a gap the agent fills by guessing.
Keep the first scope narrow on purpose. One intent, one channel and one language give you a launch where every failure is visible and traceable. A broad launch across many unrelated questions spreads the same number of failures thinly, and it becomes hard to tell whether a bad week came from knowledge, identity, tools or delivery.
Separate responsibilities before choosing services
The table below is a generic map of what the system has to do, not a prescribed pipeline. One application can own several rows, and a small team will often run all of them in a single service. The point is that each row has a decision someone must make and evidence you need to keep, so you can explain any conversation after the fact.
| Responsibility | Decision to make | Evidence to keep |
|---|---|---|
| Receive work | Which events are authentic and eligible? | Event receipt and no-action reason |
| Maintain state | Who owns the conversation and which message is current? | Conversation version and control state |
| Find evidence | Which sources may this person use? | Permitted source versions and tool results |
| Apply a procedure | Which steps and actions are allowed? | Task state and confirmed action outcomes |
| Deliver and inspect | Was the right message delivered? | Helpdesk message ID and decision record |
| Improve | Did the customer get a useful result? | Mature outcome, review decision and release version |
Notice that the model only appears inside some of these rows. Deciding whether a webhook is genuine, whether a customer may see an order or whether a reply actually reached the helpdesk is ordinary application logic. Keep those decisions in code you can test directly, and use model judgment for the parts that need it, such as understanding the question and wording the reply.
Capture a baseline before traffic changes
Once the agent goes live, the old workflow is gone and you can’t measure it again. Before launch, take a dated sample of conversations handled by people today. For each one, record the intent, channel, handling time where you have it, customer rating, whether it was transferred, how it closed and whether it reopened later. Write down what counts as resolved and how long you wait before scoring a conversation, because a ticket that looks closed on day one may reopen on day five. Where the evidence is missing, record it as unknown rather than filling it in.
Keep the sampling method alongside the numbers. If the first AI cohort contains only simple delivery questions, compare it with human-handled delivery questions from the same period, not with every historical ticket. Comparing easy AI cases against a mixed human workload will make the agent look better than it is. Store the baseline with your metric definitions so every later report uses the same denominator and the same idea of “resolved”.
Build the smallest complete service
A first version can be small: a model provider’s SDK, one durable worker (a background process whose jobs survive a restart), your existing database and your current admin tools may be enough. Add an agent framework when its state handling, recovery or tracing solves a problem you actually have. A framework won’t decide who may see an order, or whether a refund that timed out actually went through. Those rules are yours to write either way.
“Complete” matters more than “small”. The first release should handle one case all the way from the incoming event to a delivered reply and a recorded outcome, including the failure paths. A demo that answers questions in a test window but can’t recover from a failed send, or can’t tell you why it transferred a conversation, isn’t a smaller version of the service. It’s a different thing.
Count the ongoing work, not just the build
Model usage is only one line in the cost. Budget for keeping sources up to date, investigating failed deliveries, running evaluations, reviewing conversations and responding to incidents. Each of those needs a named owner, even if one person holds several roles in a small team. Someone has to own the knowledge, someone the code and security, and someone the quality data that tells you whether the agent is getting better.
Be wary of any universal “weeks to build” figure, including your own first guess. Estimate the first release from your real integration gaps and staffing, then build a thin end-to-end version and update the estimate from what you learned. The gaps that cost the most time, such as identity linking or a helpdesk event you didn’t expect, usually only show up once real events are flowing.
Ask AI to explain this guide
Build decisions and checks
Use these decisions to turn the worked case into your own implementation. A check is complete only when you have evidence for the stated result.
Choose the first support cases
Required. Launch too broadly and failures hide among many unrelated questions.
Build. Pick the intents (types of request), channels, languages and hours the agent will cover. List what's out of scope and how those cases reach a human. Start with one company and its existing helpdesk. Before AI changes anything, record a human baseline by intent and channel: volume, evidence of resolution, reopen rate, customer ratings and handling time. Freeze the metric definitions and note any gaps in the data.
Check. For any example request, an owner can say whether the agent should answer, take an action or transfer it. A dated baseline worksheet states the sample, the observation window, exclusions and outcomes you couldn't determine.
Choose a simple control flow
Required. A model can write a reply without knowing which business steps it's allowed to take.
Build. Map each step: receiving the message, permission checks, knowledge lookup, task execution, reply delivery and outcome logging. Decide which steps are fixed code and which need the model's judgment. Use the provider's SDK directly, and add a framework only where it solves a clear need.
Check. Walk through one normal case and one failure. Every step ends in a defined state.
Choose providers by evidence
Required. A strong score on a general benchmark doesn't tell you how well a model handles support for your business.
Build. Compare model families on your own questions, languages, tool schemas, latency, data terms and total cost. Record the exact version you pick and how you'd switch away from it. Hosting an open-weight model yourself adds capacity, patching and license work.
Check. Run the same held-out cases (examples kept out of tuning) against every candidate. Record the errors and which provider and model actually answered.
Name owners and count full cost
Required. Token spend is only one part of what the service costs.
Build. Name owners for policies, knowledge, code, security, incidents and launch, plus a quality owner for datasets, evaluation and metrics. In a small team, one person can hold several roles. Budget for hosting, retrieval, retries, monitoring, evaluation, human review and on-call time. Reuse systems the company already runs.
Check. The cost worksheet covers low, expected and peak volume. Every critical alert has an owner.
Ask AI to apply these decisions
Templates and examples
Every file uses synthetic data. Adapt it to your own policies and test it somewhere safe before it touches customers.
- service-plan.mdTemplate (Markdown or text)
- source-register.csvSpreadsheet worksheet (CSV)
Keep it working
Launch is the start of the work, not the end. These are the recurring jobs this part of the agent needs, and who usually owns them.
| Cadence | Owner | Work |
|---|---|---|
| Monthly | Support lead | Review new request types and human workload before you expand scope. |
| On change | Engineering | Review complexity whenever you add a branch or tool, and remove steps nobody uses. |
| Monthly | Engineering | Check for model deprecations monthly. Evaluate promising new versions before you release them. |
| Monthly | Service owner | Compare actual cost per verified outcome against the plan. Review capacity before seasonal peaks. |
Test your decision
A stakeholder wants to add refunds to the order-status launch. Can refunds use the same tool?
Show the expected decision
No. Keep the read tool separate. A refund changes money and needs its own action policy, checks on the order and the customer, limits, confirmation, an operation receipt and a recovery path for when you don’t know whether it went through. Add that capability only after those controls pass their own tests.
