How a digital-goods marketplace achieves 58% AI resolution, saving 1,768 hrs a month
A digital-goods marketplace case study: 58% of 36,413 Intercom customer support conversations resolved by AI in 30 days, 92% AI CSAT, 1,768 hours saved.
Mike is an experienced Product Manager who focuses on all the “non-development” areas of My AskAI, from finance and customer success to product design, copywriting, testing and more.
A digital-goods marketplace resolves 58% of 36,413 Intercom conversations with AI over the last 30 days, holding 92% AI CSAT and saving 1,768 hours of agent time.
A digital-goods marketplace runs its customer support on Intercom, and our AI agent answers there directly, resolving 58% of conversations without a human and saving 1,768 hours of agent time. Almost every ticket is somebody's money or somebody's item in transit, so the answer turns on which trade and which payout the customer means, and the agent's first job is to find out.
A payout has not landed, or a trade canceled after the items were already sent. The customer usually arrives convinced something has gone wrong, and often they are right.
I have read plenty of AI resolution rates that were earned on password resets and where-is-my-order. Tickets like that barely exist in this inbox.
What follows is what they connected, what they wrote, and where a human still steps in.
A digital-goods marketplace's AI customer service results on Intercom at a glance
Fact
A digital-goods marketplace
Industry
Digital-goods marketplace, consumer, global
Helpdesk
Intercom (direct replies)
Support volume
36,413 conversations (30 days)
AI resolution rate
58%
AI CSAT
92%
Time saved
1,768 hours
Escalation rate
10%
Key features used
Tasks, Guidance, Custom Answers, human handover
Knowledge sources
Intercom Help Center, uploaded internal documents, Custom Answers
Go-live mode
Direct replies inside Intercom
What does the marketplace do?
Players buy and sell digital goods for real money, mostly in-game cosmetic items known as skins. Every completed sale moves two things at once, an item to a buyer and a payout to a seller.
They keep their Intercom setup, their agents, their tags and their routing, so nothing about the way they work had to move.
A screenshot showing the My AskAI customer service agent app in the Intercom app store.
Every conversation the agent handles comes through Intercom. If this team ever moves helpdesk, the trained agent moves with them (the Custom Answers, the Guidance rules and the Tasks all travel too).
Pricing models pull apart at this kind of volume. Paying for every conversation and paying only for the ones the AI resolves are two different bills at 20,000 a month, and both sit on top of the Intercom seats this marketplace keeps paying for either way.
How did the marketplace train their AI customer service agent?
Three kinds of knowledge, added in this order.
First, the Intercom Help Center connector. Their existing help center syncs straight in, and it covers the questions a help center is good at, such as how selling works and what the fees are.
A screenshot of the Knowledge page in the My AskAI dashboard, a generic example showing the three source types on offer (Help Center connectors, website content and knowledge apps).
Second, their internal operational documents, uploaded directly as files. These are the procedures that never make it into a public help center, and in an inbox like this one they carry more weight than the public articles do.
Third, Custom Answers, which are exact question-and-answer pairs our AI returns word for word. On a marketplace where money and fraud are involved, plenty of answers are policy, and a paraphrase of a policy is a support ticket of its own.
If you are the person who would have to build this, how much of it is work you do yourself?
Less than it sounds. Connecting a help center is a switch (ours syncs the articles in and keeps them current), and the documents are an upload.
The exact answers and the rules our AI follows are the real work. Somebody on the team wrote them, and worded them well enough to cover the payout policy.
A public help center on its own is thin cover for a support load this varied, which is why the written answers carry most of the weight.
When did the marketplace decide to turn on 'direct replies' to customers?
From the start. They skipped the internal-note stage, where our AI writes its answers as internal notes and a human decides what the customer sees, and had it answer customers directly inside Intercom.
A Support Agent conversation inside Intercom where 'How does My AskAI work?' is answered in a highlighted internal note carrying a terms-and-conditions link. This is a generic library shot of the agent working natively inside Intercom, not this marketplace's own conversation.
When a conversation needs a person, the agent hands it over and steps back.
The AI Copilot Chrome Extension is not switched on for this account, so the 58% is a number our AI earned on its own conversations.
What was the biggest thing the marketplace did to improve their AI agent's resolution?
They built a named Task for each of the ticket types that arrive most.
A Task is a multi-step workflow written in plain English. Our AI picks the right one for the conversation in front of it, asks the questions that Task needs answered, and works through it with the customer.
A screenshot of the Tasks and Tools page in the My AskAI dashboard, showing one generic example Task and one example API Tool. It is a stock shot, not this marketplace's own task list.
Twelve live Tasks cover at least 68% of the tickets that arrived in the last 30 days. Anyone who has run a marketplace will recognize the types: a payout that has not landed, a withdrawal that will not go through, a trade that shows as canceled after the items were sent, a verification check that blocked an account.
Each of those needs its own questions answered before a reply is possible (which trade, at what time, in what state).
The trade-swap case is the highest-stakes ticket the team gets. A customer has sent their items and the trade then shows as canceled, and on a ticket where somebody's items are already gone, our agent asks those questions before it answers.
These Tasks scope a conversation, work out which case it is, and give the customer the correct answer. Where the resolution needs a human action, the Task collects everything needed and hands over.
Process flow of what one named Task does: pick the Task that matches the conversation, ask the questions that Task needs answered, answer the customer for that case, then hand over to a human where the resolution needs an action.
At least 68% is the share of tickets these twelve Tasks cover. The 58% is the share our AI resolves.
How does the marketplace customize their AI agent setup to work for their business?
Shaping how the agent talks and when it asks
Guidance is our set of plain-English rules for how the agent behaves. This team's rules come in three kinds: how it talks, what it asks before it answers, and when it hands over.
A screenshot of the Guidance page in the My AskAI dashboard, a generic example headed 'Communication style' with three rule cards visible (Thank you, Free trial, Use cases). It is not this marketplace's own Guidance set.
The clarification rules do the most work here. The correct answer depends on which trade, which payout and which account, and those rules are what get the customer to name it.
Handing over the moment a customer is upset
Anyone in clear distress (yes, including anyone writing in all caps) goes to a human straight away, and gets an apology first.
What is AI-to-human handoff? The bit everyone gets wrong
Escalating early is deliberate here. When the subject is somebody's money, the team wants a human on it sooner, and it shows in the numbers: escalations run at 10% and AI CSAT sits at 92%.
What impact is the marketplace's AI customer service agent having now?
58% AI resolution rate over the last 30 days.
36,413 conversations, 58% of them resolved by the AI without a human.
Stat callout with four figures: 58% AI resolution rate, 92% AI CSAT, 1,768 hours saved, and 36,413 conversations in the last 30 days.
Take one deployment's number over one window with a grain of salt (this one included). Our AI resolution rate benchmark study puts 58% against the wider set.
Where does the marketplace go from here?
The next step is live data. Connecting their account data to our agent lets a Task check a payout's actual status rather than explain what usually happens to it. We would expect the next lift in resolution to come from there.
As handovers build up, Self-Learning drafts new articles from what the human agent said after taking over. The knowledge grows out of the tickets the AI could not close.
The tickets that do reach a human are the hardest ones. Our AI Copilot Chrome Extension would draft those replies for the agent inside Intercom, which is the natural next switch to flip.
Some share of money movement and fraud claims will always need a person to decide. The ceiling they are working toward is our AI handling the conversation and a human deciding the outcome.
If your inbox looks like this one, high volume, real money, and a customer who suspects they have been wronged, nothing here is exotic. A help center, the answers your agents already give, and a Task for each ticket type you see most is the whole recipe. Our pricing is per ticket, so the bill does not climb as the resolution rate does.
Mike is an experienced Product Manager who focuses on all the “non-development” areas of My AskAI, from finance and customer success to product design, copywriting, testing and more.