Mike HeapMike is an experienced Product Manager who focuses on all the “non-development” areas of My AskAI, from finance and customer success to product design, copywriting, testing and more.
Created time
Apr 22, 2024 12:24 PM
Author
Tags
Image
Publish date
Apr 22, 2024
Slug
ai-knowledge-base-optimization
Featured
Featured
Type
Article
Ready to Publish
Ready to Publish
AI customer support chatbots have landed, and it’s only a matter of time before you start using them for your business.
Impressive as they can be, expecting an AI chatbot to be a “magic bullet” to bring down your number of support tickets instantly may be a little naive.
Like most technology, chatbots suffer from the classic problem of “garbage in, garbage out”. This means the performance of your chatbot is dependent on the quality and ‘AI-readiness’ of your help documentation.
In this guide we’ll show you how you can improve your existing help documentation to ensure when you do start using AI chatbots, they can answer anything about your business.
How do AI chatbots answer questions?
I’ll get to the practical tips in a second, but first, it can help to understand how AI chatbots answer questions so we can explain why certain changes might be necessary.
(Plus, it will give you a better mental model to work from when writing new help content).
Skip ahead to “So, what do I need to do?” if you just want the practical tips.
There are 3 key steps to AI-question answering:
- Extracting data from your help content
- Converting the help content into something the AI can understand and “search”
- Answering the question
Here are all the ways each stage can result in an incorrect answer:
Extracting data from your help content
Your AI chatbot will start by ingesting, or reading, all your help content. But, it doesn’t do this like you or I.
Often it will have to “scrape” text from each page on your help docs site, but, without knowing exactly which bits of the page it needs to take (as well as the help content itself, pages often contain menus, related articles, page footers).
So it will just take all of it, as plain text.
Rather than taking it all in one-go, it will break each page into “chunks”, often of similar (but usually arbitrary) sizes e.g. 500 characters. Each of these chunks will be an individual piece of knowledge that your AI will learn from and use to answer a question.
This results in a few issues:
- A chunk can have varying information about multiple topics, making it harder for the AI to understand what the chunk is trying to say.
- If the knowledge article is particularly long the information may span multiple chunks and context may be lost.
- Depending on the page layout, as the AI will often read “top to bottom”, rather than by ‘section’, different parts of the page may get mixed up within a chunk.
- AI chatbots often only ‘understand’ text, they won’t “read” images, embedded videos, embedded files or be able to keep the structure of most tables within the content.
Converting the help content into something the AI can understand and “search”
Once the AI has extracted the text, it converts the chunks of knowledge into something the AI understands, so it can group and index them for searching over later.
This process is “embedding”, whereby each chunk gets converted into a set of numbers that, when looked at overall, represent the “semantic meaning” of the chunk of text.
The AI is working out what each chunk of knowledge is trying to say, then grouping it with similar chunks of information.
When one of your customers then asks a question, the “embedding” process happens again over the question and its “semantic meaning” is used as a search term to locate the most similar knowledge chunks to the question itself (more on why this is later).
But this too can cause a few issues:
- AI models used by AI chatbots haven’t been trained on your specific company or industry jargon or acronyms. So they won’t assign much semantic “meaning” to content that includes them, making the content harder for the AI to understand.
- The more ambiguous or less specific your content is, the harder it will be for the AI to differentiate between and accurately ‘group’ the content into distinct buckets of meaning.
- Simple responses to questions, without context, like “yes” or “no” won’t be understood or as tightly grouped by the AI as richer responses like “yes, you can create a…” So it will be harder to “find” when it needs to answer a question with them.
- If a “chunk” of content answers several, different questions then the signal given of what that content is about will be harder to place/noisier. This makes it harder for the AI to retrieve information from.
- Where your help docs cover similar (but different) products, or even different versions of your product, it will be very difficult for the AI to differentiate between similar knowledge chunks as the meaning of the chunk will be very similar, but for one small change.
Answering the question
The final stage of the AI answering process is where the AI will look at the question, along with the most relevant knowledge chunks that have been found related to that question, to try and determine an answer to the question they have asked.
Nowadays, AI reasoning and logic capabilities are very high. If the AI has the information in front of it it will likely generate a comprehensive and accurate answer.
But, if issues with the previous 2 steps have meant that the relevant information required to answer the customer’s question is not presented to the AI then, just like a human not given enough information, it will be unable to answer.
So, what do I need to do?
The good news is that if you make any of the changes we are proposing, it also makes your help docs more usable and accessible for people too, so it’s win-win!
So here are the 7 things you need to do to optimize your help content to make it AI-ready (with examples):
1. Be clear, concise and unambiguous
Each article or question you answer should be short, to the point, and use language specific to the issue being addressed.
Think overly specific rather than general. Spell things out as if you are writing to someone who has never seen or heard of your product or business or understands what it does. Don’t worry too much about repetition across articles.
It is worth pointing out here too that having less “personality” or “brand” to your help content will likely be a good thing for the AI’s understanding (you have lots of other opportunities across your product to get this across!).
2. Think formatting
You can’t go overboard when formatting for AI.
Think of it as ‘sign-posting’. Use lots of headings and subheadings to identify each section of the content or guide and use bullets or numbering to show the steps of a process.
This type of formatting helps the AI to ‘read’ and interpret what each piece of help content is about without it having to ‘read’ more information around it.
Also, make sure you use full stops at the end of sentences and bullet points, as otherwise when the text is extracted it will be harder for the AI to determine where sentences and phrases begin and end.
3. Drop the jargon, but think about keywords
AI won’t understand your company jargon or acronyms, but a lot of the time, your users won’t either.
So limit the use of it to improve the likelihood that your AI will understand what you are talking about.
Use a plain English explanation with jargon in brackets afterwards (if you want to educate your users), rather than the other way around.
Think about the words you can use to make it clear what the help article is about for the AI. Think about what a user might search for if they were asking the question.
If you had to summarize the help article in 2-3 words, what would they be and are they mentioned several times throughout the article?
4. Take care when using tables and images
Most of the time, the AI won’t read a table as a table, it will just take all the text from it, without knowing which text comes from which column.
So the fewer tables you use, the better, or, if you have to use them, keep them smaller, with 2-3 rows maximum.
The same goes for images, most AI chatbots won’t be able to interpret them, so don’t use them to explain core concepts or show diagrams or screenshots without explaining them in text format in the body of the article.
(Added bonus here - it will only help make your articles more accessible for those with visual impairments).
5. One help article, one topic
The less ‘overlap’ between help articles, the easier it will be for the AI to identify which article it should use to answer the user’s question.
Try not to put articles that cover different parts of the product or business in one article, but do make sure that anything about a specific topic is in one place as it gives the AI more contextual understanding for answering questions.
6. Answer in full, using context
When writing out help articles and FAQs, they are often framed as questions with a “yes” or “no” response. AI needs more context than this, so instead of just saying “yes” update answers to be “yes, it can <whatever your question was>”.
This will make it far easier for the AI to find these answers when searching.
In addition, if an answer or help article is a little longer, it helps if you can provide some more context or background to the problem the article solves or question it is answering. This will help the AI discover it, but also give it more background when it comes to actually answering the question.
7. Avoid conflicting or contradictory information
You shouldn’t have outdated information in your help content. This is especially important if you use AI as the chatbot finds it nearly impossible to distinguish between the old and the new content and therefore can give outdated responses.
The same principle applies if your business has multiple versions or products.
Unless each article specifically calls out which version or product it relates to, the AI will have no broader knowledge to draw from to determine the correct answer.
So make sure, when giving context, if there are product or version-specific differences, that you call these out explicitly in your help content.
How to measure how well your help documentation is set up for AI
The best (and quickest) way to measure how well your help docs are set up for AI is to upload them to a chatbot and start asking questions.
Start off with a list of the most common questions you receive from customers and see how it performs. Compare the answers you get to your ‘best-practice’ answers.
Updating your help documentation for AI will be an ongoing process.
My AskAI has several features that will help you improve yours over time:
Unanswered question monitoring
Within AskAI you can see which user questions are going unanswered.
Questions go unanswered for a couple of reasons:
- The knowledge or help content to answer the question doesn’t exist in the first place for the AI to find, and so you need to write an article.
- The content to answer the question exists, but the AI is unable to find it, so it may need to be rewritten or edited to take into consideration the 7 tips points above.
Insights
In addition to seeing which questions are unanswered, you use the Insights feature to see where common questions are coming up from your users.
This will help you to focus on where you may want to spend more time optimising your help content for AI answers.
Conclusion
To wrap up, remember that integrating AI into support tools is a gradual process that requires attention to detail.
By clarifying and refining your content, you set the stage for your AI chatbot to provide nuanced and accurate support, enhancing customer satisfaction and operational efficiency.
If there is one thing you take away from this piece it should be that the performance of your AI chatbot is directly linked to the quality of input it receives, so continuous improvement of your documentation is essential for maintaining its effectiveness, good luck and do share with us any additional tips and tricks you find!
