How do customer service AI chatbots actually work (in 2024)

AI chatbots used for customer service are naturally getting extremely popular, but not many people know how they actually work…

How do customer service AI chatbots actually work (in 2024)
Created time
Mar 1, 2024 11:27 AM
Author
Image
Publish date
Mar 1, 2024
Slug
how-do-ai-chatbots-actually-work-in-2024
Featured
Featured
Type
Article
Ready to Publish
Ready to Publish
AI chatbots used for customer service are naturally getting extremely popular, but not many people know how theyactuallywork… Understanding this can really help a business adopt one for tasks like customer support or an internal employee assistant. Companies offering AI chatbots will describe their secret sauce in lots of different ways: fine-tuned models, custom LLMs, personalised AI models — but the reality is, under the hood, they all work in a very similar way.
 

1. Indexing all your company’s knowledge

Step 1 is always scraping, extracting, or accessing all your company’s knowledge. This could be on your website, Notion workspace or within Google Drive. This content is basically copied, separated into individual paragraphs (called ‘chunks’ in the industry), and converted into a format that can be retrieved very quickly using natural language search. Which basically means, when a question is asked like “How do I connect to the API”, the paragraphs that are brought back are very similar in meaning (intent) to the question. The most relevant paragraph is not the paragraph with the most keyword matches (traditional search). Once all a company’s knowledge has been indexed, we can now look at how a question can be answered.
 

2. Question from a customer “How do I add a new team member”

This could be a typical question seen in customer support for a SaaS company. Below I’ll describe how an AI model is used to answer this.
 

3. Retrieve the most relevant information for the question

The user’s question is converted into that same format as the company’s knowledge base and used to do a natural language search. The result of this search will be ~5 relevant paragraphs, from across all of your company’s content, to the user’s question. In this example, the paragraphs will likely include instructions on adding and removing team members
 

4. Ask an AI model to answer the question from the relevant information

Now that we have ~5 paragraphs that likely contain the answer somewhere within it (along with some irrelevant info, like removing team members) we can pass this to an AI model. Specially we’re talking about a large language model, like ChatGPT. To visualise this, imagine you’re using ChatGPT on the web, and you type in a request like:
You’re an intelligent customer support assistant, please answer the customers questions with the help documentation provided. If you can’t find an answer, say so.
<5 paragraphs of relevant info pasted>
Customer’s question: How do I add a new team member?
 
This is just a simplified example of the request an AI chatbot will make, but the principles are the same. It’s providing the AI model with some context so that it can answer a really specific question. And importantly, we’re also telling the AI model to not make an answer up if it can’t find a solution in the paragraphs provided. This helps avoid “hallucinations”, also known as lying, by the AI model. Because if you don’t have instructions on how to add team members, you don’t want it making up those instructions.
 
That’s it.
 
From this, you can start to understand the things that will affect the output quality of an AI chatbot. This can then help you decide on which company you choose to create your chatbot with
 
  • Help documentation / knowledge available. In an answer is not documented somewhere, an AI model will never provide a good answer
  • AI model. Whether at chatbot is using the latest AI models available, will have a big impact on answer quality.
  • Chatbot ‘prompt’ or instructions given to the AI model. Small changes in these instructions can have big consequences on answer quality.
  • User’s question. AI models are very literal and precise, and for that reason, small changes in how a question is phrased can result in quite different answers. This can be helpful and unhelpful.

Start using AI customer support in your business today

Create free AI agent

Written by

Mike Heap
Mike Heap

Mike is an experienced Product Manager who focuses on all the “non-development” areas of My AskAI, from finance and customer success to product design, copywriting, testing and more.

Related posts

9 Benefits of Using AI in Customer Support

9 Benefits of Using AI in Customer Support

Read on to discover how AI in customer support can streamline operations, enhance customer satisfaction, and boost business growth

How To Use Chatbots for Customer Support

How To Use Chatbots for Customer Support

Read on and discover how to use chatbots for customer service and revamp your business’s assistance service quality in no time.

8 Best Customer Support Chatbots

8 Best Customer Support Chatbots

Maximize your business performance and customer satisfaction with our best customer support chatbots in 2024.

Klarna’s new AI chatbot handles two-thirds of their customer support requests

Klarna’s new AI chatbot handles two-thirds of their customer support requests

The AI chatbot already does the work of 700 full-time agents, handles 65%+ of their support requests and is on track to drive an extra $40 million in profit this year alone.

Embracing AI in the Workplace. A Practical Guide for the Skeptical

Embracing AI in the Workplace. A Practical Guide for the Skeptical

A short hands-on guide on how you can showcase the value of AI to your boss or client, who may be worried, skeptical, or hesitant about using AI within their company.

2024 Chatbot Cost Analysis: What to Expect?

2024 Chatbot Cost Analysis: What to Expect?

Read on to learn how much a chatbot costs in 2024 before integrating them into your business’s customer support.

6 Practical Tips For Awesome Customer Support Replies

6 Practical Tips For Awesome Customer Support Replies

In this era of AI-content, and AI-customer support, a human touch to your customer’s questions and issues can help set you apart.