Mike HeapMike is an experienced Product Manager who focuses on all the “non-development” areas of My AskAI, from finance and customer success to product design, copywriting, testing and more.
.png?table=block&id=139cbb2d-d767-81fe-b695-ed482813c4a1&cache=v2)
Created time
Nov 9, 2024 07:22 PM
Author
Tags
Publish date
Nov 9, 2024
Slug
evaluating-ai-support-key-metrics-for-success
Featured
Featured
Type
Article
Ready to Publish
Ready to Publish
If you are currently testing out AI support solutions and have no idea how to compare their answer quality, here are a few pointers:
You should be scoring AI providers on:
- Answer accuracy: Did it answer the question correctly?
- Incorrect answers: Did it answer any questions incorrectly (a big red flag)?
- Conciseness: AI tends to ramble, but customers don’t tend to like reading long responses, so make sure the ones you get are of an appropriate length.
- Source accuracy: Did it provide what you would deem to be the most relevant sources (links or documents) back in the answer for further reading?
- Answer tone: Does it fit with your company and brand?
- Clarifications: Did it ask follow-up questions if the question was vague?
- Focus: Did it answer anything it shouldn’t, did it go off-topic or talk about competitors?
- Conversation: How did it handle small talk or informal conversation?
Scoring in these areas will give you a very good idea of the quality of the AI agent.
.png?table=block&id=139cbb2d-d767-810e-8fec-d8207d52abb3&cache=v2)
.png?table=block&id=139cbb2d-d767-8181-b47c-c2c7b3d10583&cache=v2)
.png?table=block&id=139cbb2d-d767-81d5-b0cc-c5419d8d0fe3&cache=v2)
.png?table=block&id=139cbb2d-d767-815e-8559-c9ae9617e2b7&cache=v2)
.png?table=block&id=139cbb2d-d767-8192-8ed1-e644240aaada&cache=v2)