An AI chatbot for customer support: what it takes to make one that works

Start with what customers actually ask

Before any technology, read a few hundred real support conversations and sort them. In most businesses a small number of question types make up most of the volume: order status, returns, opening hours, how to do a common task, a handful of known problems.

This tells you two things: which questions a chatbot could realistically take, and which need a person because they involve judgment, a complaint or an exception.

The answers have to exist in writing

A chatbot does not know your policies. It reads them. If the correct answer to a common question lives only in the heads of your support team, the chatbot cannot give it.

• Help articles and policies need to be written, current and consistent with each other.
• Where two documents disagree, the chatbot will sometimes pick the wrong one.
• Someone has to own the content and update it when things change.

Teams often find that preparing this material improves their human support as well, because the answers are finally written down in one place.

Connecting it to your systems

Answering from documents handles general questions. Many support requests are about one customer's own situation: where is my order, when does my plan renew.

For those, the chatbot needs controlled access to your systems, limited to reading what that customer is entitled to see, and only after they have been identified. This is ordinary integration work, and it is where much of the value is.

Handing over to a person

• Offer a way to reach a person at any point, and say so clearly.
• Hand over when the customer asks, when the chatbot is unsure, and for complaints and anything involving money.
• Pass the whole conversation to the agent, so the customer does not have to repeat it.
• Outside working hours, say when a person will reply.

Customers are tolerant of a chatbot that knows its limits. They are not tolerant of one that keeps them from a person.

Guardrails

Decide what the chatbot must never do before you decide what it can do. It should not invent policies, promise refunds, give legal or medical advice, or discuss other customers. It should say it does not know when it does not know.

These rules are tested like any other feature, with deliberately difficult questions, before the chatbot meets a customer.

Measuring whether it works

• How many conversations ended with the problem solved, without a person.
• How many were handed over, and why.
• What customers said afterwards.
• Which questions it could not answer, as a list of content to write.
• What each conversation costs.

The number of conversations handled is the wrong measure on its own. A chatbot that closes many chats and solves few problems is hiding work, not doing it.