Advice: 1,000+ hours building AI chatbots, don't build your own

Created time

Oct 4, 2023 08:03 AM

Author

Are chatbots coming back to life with AI?

Let’s start from the beginning though (as you may not be as hyped about AI chatbots as me…)

When chatbots first arrived on the scene 5 years ago, we thought we were going to be able to say goodbye to human customer support, what went wrong?

Chatbot interest over time, Google search trends

Maybe you were one of those who tried to build a chatbot on Facebook Messenger, WhatsApp or Intercom? But then you quickly realised the functionality was clunky and the logic was basic at best, forcing you to predict every question a user might ask in advance.

So customers quickly got fed up with having rigid, dead-end conversations, and, like most companies, you probably gave up on building your chatbot, consigning it to another over-hyped tech trend.

Fast forward to 2023, to the launch of ChatGPT and the rise of Large Language AI Models (LLMs).

Overnight, the capability available to you to build AI chatbots has changed radically. The technology is readily (and cheaply) available for you to create a sophisticated and intelligent AI chatbot for your business.

But the age-old software question still stands - do you build or do you buy?

Why you should build your own AI chatbot

“It’s never been easier to build an AI chatbot” — no doubt you’ll have heard this countless times in all those daily AI newsletters.

You’ve probably been eyeing up Langchain or Llama Index, read their docs and a few tutorials, and fancy deploying your own AI bot in a weekend hack.

In fact, here’s exactly how you’d do it:

Fetch, transform, and ingest your data into a vector database

Spin up a chat interface from a template

Connect to a super-intelligent AI model, like OpenAI’s ChatGPT, that uses your content and can have a conversation with someone — answering their questions like an expert

Simple right?

A standard AI chatbot architecture with access to private data or content

After your weekend hackathon (and after you’ve shared your MVP with your team), you realize you want to take your AI chatbot to the next stage, after all, now you now have complete control over the:

Underlying data used by the chatbot - you can add all your private forecast reports or internal process docs and learning materials.

User interface and branding - you can make it feel like it’s really a part of your existing stack.

Data privacy and security - you can decide what happens to a user's chat history, or how ingested data is stored and processed. If your data is particularly sensitive, this may even be table stakes.

Access to your secure or private systems - systems that you wouldn’t want to expose to a 3rd Party.

All sounds good, right? But there are some pretty significant drawbacks, and there are some things your team will need in order to make a success of this.

Why you shouldn’t build your own chatbot

Nothing is ever as easy as it seems.

As useful as the frameworks and libraries are to get you started quickly, they hide a lot of detail and a long list of small decisions that all add up to have a significant impact on the quality of your chatbot (and the risk to your business).

Here are all the things you’ll need to consider and stay on top of if you are going to build yourself:

Initial business case: A custom AI chatbot for a business is estimated to cost upwards of $50,000 and these costs can quickly spiral upwards for use cases that require a high degree of accuracy or where a company has large volumes of content for the chatbot to access. You will need to justify this to management, likely before you can demonstrate the actual value.

AI expertise: You’ll need people in the business who understand how LLMs work at a high level. Especially what they can’t do. This expertise will be essential if you want your chatbot to be more than just a novel toy for customers or employees (better start actually reading that AI book you bought yourself to “get up to speed”)

Product continuity: What happens when you leave? Looking forward to having to document how your AI chatbot works for the next person or team?

AI provider and model selection: You’ll need to decide on which AI provider you partner with and which AI model you want to use. This will involve weighing up the Pros and Cons and trade-offs you’ll need to make e.g. Speed vs. Cost vs. Quality

Keeping up with AI developments: You’ll need a team to constantly stay on top of the latest AI developments to make sure your chatbot is leveraging the newest tech. But importantly, to make sure you’re not reliant on methods that are being phased out. We’ve seen changes to AI models that change answer quality overnight, the solutions for which, never seem to be straightforward!

Preparing your data and content: The data processing stage is crucial, this is where your content is manipulated into an optimal format for AI models to use. There are 20+ small, technical, decisions packed into just this topic and hundreds of edge cases to consider (did you know Arabic takes up 4x as many tokens as English for example?). There are already large groups of companies dedicated to solving just this problem, like carbon.ai and psychic.dev

Automated chat testing: You’ll have to define a testing approach to understand how changes you make affect the quality of your chatbot. The biggest challenge here is the sheer number of permutations that are available to you. We worked out there are over 1 million combinations today, and this grows with every new model and parameter. With each permutation, you have to test your AI chatbot like real humans would — but in an automated way, at scale, to be meaningful.

Integrations: Do you want to integrate your chatbot with Slack or Teams? Or embed it on your website? These are all additional features that need to be built and approvals for them takes months with A LOT of back and forth on minor details.

We live and breathe all these topics and challenges every day (and even for us some days they can feel overwhelming!). But we specialise, deeply, in all these areas with a single goal: creating the best AI chatbot to provide answers based on your content.

Why reinvent the wheel?

Now, back to our enterprise search analogy. You wouldn’t consider building it yourself, even though it is fundamental in helping your teams locate the right content or data. You know how intricate and specialized the art of good search is. AI chat is much the same but with much more uncertainty.

You also have to ask yourself if there aren’t more important challenges to be working on in your business? For instance, considering where AI fits more strategically, instead of how you are going to pre-process 5,000 PDF reports.

So, after 1,000+ hours building AI chatbots, I’m confident when I say that for 90%+ of businesses, buying in your AI chatbot solution is the way to go, but if you are still not convinced, here is the breakdown to help:

ㅤ	If you want to BUILD…	If you want to BUY…
AI capability and knowledge	Existing internal AI capability or expertise	Limited or no existing AI knowledge in the business
Data sensitivity	Highly sensitive data is required for the AI chatbot to access	Private (not highly sensitive) or public data and content
Content type	Business content is static and limited to <500 items/reports/documents	Business content can be dynamic, growing, and, available in large volumes of >1,000 items/reports/documents
Dedicated team	Ability to dedicate a team to building and managing an AI chatbot	Objective to integrate AI chatbot capability into an existing team to manage (like enterprise search)
Chatbot as core differentiator	You expect an AI chatbot to be a significant differentiator and part of your business’s offering	An AI chatbot will leverage or enrich your business’s existing core offering e.g. your editorial trend reports

And while you may think that Slack or Teams integrations, personalization, data security, and enterprise controls are other reasons to build your own. With My AskAI, and some other providers, these features come out of the box.

Of course, your ‘buy instead of build’ strategy may change over time. For example, if your chatbot becomes a central part of your business offering or how it works, then it might justify the investment to build your own at some stage. But initially, while you dip your toes to see where generative AI solutions, like a chatbot, fit within your business. It’s probably best to just optimize for learning quickly and getting something live.

So, what next?

The best thing you can do for your business is to start. Today.

The average My AskAI customer has their first AI chatbot live in under 10 mins. We’ve become known for being the easiest place to launch an AI chatbot.

We’ve helped over 30,000 businesses build an AI chatbot, trained on their own data and content, branded and personalized, and launched to where their customers or employees most need it.