Secure an AI support agent and handle service failures
The short answer
Secure the boundaries where data and authority enter your agent. Set one time and retry budget for the whole reply, keep accepted work safe through restarts, and decide in advance what a safe outcome looks like when each dependency fails. A gateway can reroute requests to another model, but that fallback still needs its own evaluation.
Treat external content as evidence, not authority
Prompt injection is text written to manipulate the model into ignoring its instructions. It can arrive anywhere the agent reads: customer messages, webpages, uploaded documents, text extracted from images, even tool responses. Your agent needs to read that content to do its job, so the defense isn’t to block it. It’s to keep it separate from the things that carry authority: your application’s policy and the customer’s verified identity. In the Northstar case, the text in the image is a claim from the customer, not an instruction the agent should follow.
Because no prompt alone can guarantee that hostile text is harmless, the real controls sit in code. Give the agent only the tools the current task needs, and enforce each tool’s permissions and allowed arguments on the server, where injected text can’t reach them. A refund tool that checks eligibility and limits itself, regardless of what the model asks for, stays safe even if the model is fooled. Validate any links or HTML the agent produces before they’re shown to a customer.
Test with realistic attacks: an instruction hidden in a retrieved help article, an attachment that asks for a secret, and an API field that tells the agent to send its reply somewhere else. Check the tool calls and data flows that result, not only whether the final reply contains a refusal. An agent can politely decline in its message while having already called a tool it shouldn’t have. Add new cases after any incident and when relevant security research appears.
Map every copy of customer data
Customer data spreads further than you might expect. To find out where yours goes, trace a synthetic record through the helpdesk, your application database, the model request, the gateway, logs, traces, learning candidates and evaluation exports. For each place it lands, record the purpose, the fields allowed there, who can access it, how long it’s kept and how deletion works. Include backups, and document any delay before deleted data is actually removed from them.
When you decide what goes into logs, list the fields that are allowed rather than the secret fields to strip out. With a removal list, a new API field that nobody added to the list goes straight into your logs. To check this, plant a canary value, a unique fake secret, under a key name you haven’t used before, run a request, and confirm the canary doesn’t appear in general logs or in the model’s context. Keep enough identifiers and timing data to investigate an incident without copying whole request bodies.
Plan each dependency failure
Your agent depends on a model provider, a knowledge index, business APIs, the helpdesk and its own workers. Each of them will fail at some point, and each failure needs a defined response decided in calm conditions, not during an incident. Some responses rely on reconciliation: checking the downstream system to find out what actually happened before doing anything else.
| Failure | Response policy | Proof |
|---|---|---|
| Model rate limit | Bounded retry within the remaining time and cost budget | Retries aren’t multiplied across layers |
| Model outage | An evaluated fallback configuration, or a safe delay or handover | Tool behavior and wording on the fallback are acceptable |
| Knowledge unavailable | State the limit and use the approved support route | No invented policy, and no stale source presented as current |
| Write result unknown | Hold the conversation and reconcile the original operation | No duplicate business action, such as a second refund |
| Helpdesk send fails | Keep the delivery pending and reconcile | No claim of delivery without a downstream record |
| Worker restart | Recover durable jobs with ownership and expiry checks | No accepted event is lost and no stale action runs |
The unknown write deserves extra care. If the refund API times out, the refund may or may not have gone through. Retrying blindly can pay the customer twice, and telling them it failed may be untrue. The safe pattern is to hold, look up the original operation by its ID, and only then decide what to say and do.
Use one end-to-end budget
Start with how long the customer can reasonably wait on that channel. A live chat needs a much faster reply than an email. From that total, reserve time for retrieval, tools, model work, delivery and the fallback path, then pass the remaining deadline down to each operation so it knows when to stop. A timeout on each individual request doesn’t cap the total. If the SDK, the gateway, the worker and your application each retry three times, a single slow dependency can turn one call into dozens and keep the customer waiting long after the answer stopped being useful.
Classify errors before you retry. Invalid input and denied access won’t succeed on a second attempt and need a different response from a temporary rate limit. Where the provider sends a Retry-After header, respect it. For safe operations, use bounded retries with backoff (a growing wait between attempts) and jitter (a small random offset so many workers don’t retry at the same moment). Keep the maximum attempts, the backoff and the total spend explicit, and decide which layer owns retries so the others don’t add their own.
Monitor queue age as well as request duration. A worker can process each job quickly and still be answering a customer message that arrived twenty minutes ago.
Evaluate fallback behavior
Portkey and Cloudflare AI Gateway are examples of request-routing and observability services that sit between your application and model providers. They can help you route around a provider failure and see what happened. They can’t make two different models behave the same way with the same prompt and tools, so a successful HTTP response from the backup isn’t the same as acceptable support behavior.
Treat a fallback as an approved configuration in its own right, with tested instructions, tool schemas and conversation handling. It might be the same model reached through another route, which is the closest match, or a different model that has passed your evaluation with settings tuned for it. Retest it whenever the primary model, prompt or tools change. If no route meets your quality and safety requirements, a short delay or a transfer to your team is a valid service response. Log the route each request actually took, so evaluations and metrics don’t silently credit the fallback’s results to the primary model.
Cap load, abuse and spend
A traffic spike, a looping conversation or a deliberate attack can exhaust your model budget and starve normal support. Set limits on requests, tokens, tool steps per turn and concurrent work, and bound attachment sizes and how much retrieval a single message can trigger. Keep expensive background jobs, such as bulk re-indexing or evaluation runs, away from the capacity that serves live customers. Pair cost alerts with a pause control, so someone can stop automated replies quickly if spend climbs for a reason nobody understands yet.
Practice recovery before an incident
Keep development and production credentials separate, so a test can never act on real customers. Using harmless fixtures, rehearse the failures you planned for: a worker restart in the middle of a reply, a revoked connector, a failed deployment and a rollback. After each one, check the version that’s actually running and what really happened downstream, rather than trusting a status page or a success message.
Give each urgent alert an owner, a threshold and a next action. A queue-age alert should link straight to the affected jobs and the recovery procedure. Your monthly review is for routine work. It isn’t the response path for an active data leak or a run of refunds with unknown outcomes, which need someone acting within minutes.
Ask AI to explain this guide
Build decisions and checks
Use these decisions to turn the worked case into your own implementation. A check is complete only when you have evidence for the stated result.
Treat external content as untrusted
Required. An email, document, image or tool response can contain instructions aimed at the agent.
Build. Keep business instructions separate from external text. Enforce tool permissions and output handling in code. Validate links and rendered HTML. Test direct and indirect prompt injection (instructions hidden in content the agent reads). A stronger prompt on its own isn't enough.
Check. An injected document can't extract a secret, widen access or trigger a forbidden action.
Protect data across every service
Required. Customer data can spread into logs, caches, backups and evaluation exports.
Build. Map where data flows and who can access it. Set retention, deletion, encryption, region and vendor requirements. Redact sensitive fields before logging, and restrict console and network access. Test cache keys and document filters across accounts. Include copies in gateway logs, tracing tools, derived learnings, evaluation datasets and archives. Allow only approved trace fields, and test with a synthetic secret under an unexpected key. Define retention and how deletion reaches every copy, including documented limits for backups.
Check. Follow a synthetic record through storage, logs and deletion, and confirm another account can't retrieve it. Check every mapped store. Show that sensitive data is absent from model and log payloads, and that deletion or restricted access reaches derived copies as the policy states.
Set one end-to-end time budget
Required. Retries at each layer can multiply cost and keep a customer waiting after the answer stops being useful.
Build. Budget time for the whole turn and for each external call. Honor Retry-After headers where supported. Use limited retries with backoff and jitter for operations that are safe to repeat. Coordinate the retry policies of the SDK, gateway, worker and application so they don't stack. Stop at the deadline.
Check. Slow down several dependencies at once. The run stops within its budget without repeating any writes.
Plan outages and tested fallbacks
Required. A gateway can restore HTTP success while the replacement model handles tasks or handovers differently.
Build. Define how the service degrades safely when the model, search, tools, helpdesk or logging go down. Prefer a proven equivalent deployment where you can. A different model needs its own tested prompt, settings and tool compatibility. If none is ready, delay safely or hand over to a human.
Check. Run an outage drill. Check response quality, permissions, preserved state and the messages actually delivered.
Recover jobs, queues and deployments
Required. A deploy or worker crash can lose accepted work or replay an action that already completed.
Build. Use durable job state, bounded queues and a dead-letter queue for jobs that keep failing. Define how workers drain and restart. Keep configuration and data migrations backward compatible. Use health checks, deployment verification and rollback.
Check. Restart in the middle of a run. Work resumes or stops in a known state. Read back the deployed version and a synthetic end-to-end result.
Control load, abuse and spend
Required. An attack or traffic spike can burn through the model budget or crowd out normal support.
Build. Set limits on requests, tokens, tool steps and concurrency. Cap attachment size and retrieval work. Keep expensive jobs away from live conversations. Protect private cache entries. Set cost alerts and a control to pause the service.
Check. Run burst and abuse tests. Normal requests keep their capacity, and limits stop runaway loops.
Ask AI to apply these decisions
Templates and examples
Every file uses synthetic data. Adapt it to your own policies and test it somewhere safe before it touches customers.
- incident-runbook.mdTemplate (Markdown or text)
- data-copy-register.csvSpreadsheet worksheet (CSV)
- failure-test-plan.mdTemplate (Markdown or text)
Keep it working
Launch is the start of the work, not the end. These are the recurring jobs this part of the agent needs, and who usually owns them.
| Cadence | Owner | Work |
|---|---|---|
| Monthly | Security | Add new attack cases after incidents and relevant security research. |
| Monthly | Security | Review access, retention jobs, dependencies and vendor changes. Each month, check third-party retention settings against the data policy. |
| Weekly | Engineering | Review tail latency, retry volume and runs that used up their budget. |
| On change | Engineering | Test fallbacks again after model, prompt or tool changes. |
| Daily | Engineering | Check queue age, failed jobs, backups and recovery drills. |
| Daily | Engineering | Review unusual spend, capacity and cache correctness. |
Test your decision
The primary model fails, and a gateway can immediately route requests to another provider. Is availability restored?
Show the expected decision
Infrastructure availability may be restored, but acceptable agent behavior isn’t proven until that configuration has passed the relevant tests. Use only an approved fallback, or follow the defined delay or handover path until one is available.
