🚀
GUIDE GRATUIT
RFP Chatbot : Ne partez pas d'une page blanche
Un mode d'emploi pensé pour les métiers avec un guide pas à pas pour rédiger chaque section de votre appel d'offres.
Méthodologie guidée : Quoi écrire dans chaque chapitre (Contexte, Fonctionnel, Tech)
Concepts clés décryptés : RAG, LLM, Base de connaissance
Modèles inclus : Structures prêtes à l'emploi pour vos cas d'usage
👉 Télécharger le guide
The “chatbot resolution rate” shown in most dashboards actually measures automation: the share of conversations that ended without a handover to an agent. It does not tell you whether the customer’s request was dealt with.
Depending on how it is calculated, the same bot, on the same conversations, shows 33% or 50%. Intercom has just demonstrated this with its own figures. Here are the three metrics you should not confuse, and how to get a resolution rate you can defend in front of your leadership team.
Imagine your chatbot shows an 85% resolution rate, while your agents still hear “the bot didn’t help me”. Nobody on the executive committee really knows what that 85% measures. The problem is not the bot, but the definition of the metric.
Key takeaways
Deflection, automation and resolution are three different metrics. Only the resolution rate tells you whether the customer’s request was dealt with.
The market is converging on AI-verified resolution. Intercom changed its denominator in July 2026 (33% → 50% in its example), and Zendesk has counted only “Verified Resolutions” since May 2026.
A real resolution rate is measured by reading every conversation. An AI separate from the bot reviews 100% of completed conversations and gives a reasoned verdict.
Your first figure will be around 40 to 50%. It is a starting point, not a grade. Bots monitored with this measurement sit between 50 and 85% verified resolution.
A reliable resolution rate is auditable: you can open the conversation behind the percentage.
Deflection, automation and resolution measure three different things
Three metrics present themselves as a chatbot’s “resolution rate”, but only one tells you whether the customer’s request was dealt with. Deflection and automation count conversations without a handover, even when the customer left without an answer. The gap between them runs to tens of points.
Deflection rate
What it counts: conversations that never reached a human.
What it misses: a customer who gave up in anger is counted as “deflected”.
Automation rate (or containment)
What it counts: conversations that stayed in the bot’s channel, without a handover.
What it misses: it does not tell you whether the bot answered the question.
Resolution rate
What it counts: conversations where the customer’s request was dealt with.
What it misses: nothing, provided it is actually measured.
Definition. A chatbot’s resolution rate is the share of conversations in which the user’s stated request was actually dealt with. That can happen through a direct answer, a valid redirection or a properly managed escalation, meaning a clean handover to an agent.
Most dashboards display the automation rate under the name of resolution rate. Any conversation without a handover to an agent is counted as “resolved”.
So the user who asked their question three times, got three generic answers, then closed the window is counted as resolved. So is the one who wrote “thanks for nothing”.
The gap is not theoretical. According to a Gartner study cited by Maven AGI, AI deflects more than 45% of customer service requests, but only around 14% end in full self-service resolution. That is a thirty-point gap between “the bot took the conversation” and “the bot solved the problem”.
This is also why the 90 or 95% figures you read in brochures almost always measure automation. They are not wrong: they answer a different question.
Market leaders now count verified resolution
Two recent changes show that the market is converging on verified resolution, rather than a count of conversations without handover. A resolution is no longer counted because there was no handover, but because a reading of the conversation confirmed it. The denominator only includes conversations where the bot genuinely had a request to deal with.
Intercom, July 2026. Intercom changed the definition of Fin’s metrics. Conversations where Fin was active but “did not have the opportunity to answer” are removed from the denominator. In the official example: 1,000 conversations, 250 resolved. Old rate: 33%. New rate: 50%. Same bot, same conversations, 17 points more. Intercom was honest enough to point out in the same article that the automation rate does not move.
Zendesk, May 2026. Since 18 May 2026, Zendesk has billed and counted only “Verified Resolutions”: the AI agent answered, a verification step by a language model confirmed that the problem was solved, and the customer did not dispute it.
If you are comparing vendors right now, ask them this question first: how do you count a resolution, and who verifies it? Add this criterion to your specifications, alongside those in our guide to writing an AI chatbot RFP.
An AI separate from the bot reviews every completed conversation and gives a verdict
The only reliable way to know whether a conversation was resolved is to read it. With 50,000 conversations a year, nobody does: so the job is given to an AI. A language model, separate from the one that answers customers, reviews every completed conversation against a fixed set of criteria and gives a verdict.
This approach is called LLM as a judge, meaning a language model used as a judge. At ViaSay, a French provider of conversational assistants, this is the role of AI Quality Monitoring. It evaluates 100% of completed conversations, not a sample, including scripted flows without generative AI.
For each conversation, the evaluator answers four questions:
What was it about? It picks a topic from a closed list that you control: payment, booking changes, baggage, and so on.
Was the request dealt with? The verdict is “resolved” or “unresolved”: this is the resolution rate.
Did the bot have the information? If not, it is a knowledge gap, meaning information missing from the knowledge base: the signal you can act on most directly.
How did the user experience it? Sentiment and effort are combined with resolution into a CX Score out of 10.
The evaluator sets aside a third case: the customer never made a request, or left during a pending step. That could be a question asked, an identifier requested, a choice to make or a handover to accept. These conversations say nothing about the bot’s quality, either way.
The evaluation takes place after the conversation ends, never during it.
The criteria that decide whether a conversation is resolved
This is where most definitions fall apart. Here are the evaluator’s criteria.
What counts as resolved:
A direct and relevant answer to the request.
A redirection to a valid resource that matches the request, for example an order tracking page when the customer wants to track their order.
A correct escalation to a human who deals with the request.
A bot that identifies that the action requires support, says so clearly and gives a concrete way to reach them.
A suggested link that the customer followed, or a fully button-based flow that leads to what they were looking for.
What counts as unresolved:
A generic or off-topic answer.
A request that was never understood.
A refusal to escalate with no alternative.
A redirection when the bot could have answered.
A conversation that ends with neither resolution nor clean escalation.
An answer that boils down to “no agent is available at the moment”, even with a callback offered. The customer leaves with nothing they can do on their side.
The customer’s silence decides nothing: the evaluator judges what they were given before they left. A customer who goes silent after usable guidance has been served; after an empty answer, they have not.
A customer who leaves while being asked for their booking number is not judged at all. That is the difference with an “assumed resolution”, where silence counts as agreement by default.
A well-managed escalation is a resolution. A bot that recognises its limits and hands over cleanly has done its job.
Customer sentiment does not affect the resolution verdict. A cancellation policy correctly stated to a furious customer is a resolved conversation. Dissatisfaction is measured elsewhere, in the CX Score, and this choice is deliberate: mixing the two would make both unreadable.
In the dashboard, every figure can be checked conversation by conversation
The resolution rate is not a monthly report: it is a live metric, displayed on the same screen as volume and handover rate. Every AI-judged metric is flagged, and every figure links to the conversations behind it.
This screenshot, taken with demo data, shows three things.
The resolution rate is 52%, next to an automation rate of 98%. These are the two metrics described above, side by side. The second did not move when we started measuring the first, and it is the first that you can improve.
Every AI-derived metric carries an “AI” badge. You can always tell what is counted, such as volume, handovers or CSAT votes (the satisfaction ratings given by customers), from what is judged.
The AI Impact section compares conversations handled by generative AI with those handled by scripted flows, on the same criteria. It answers the question: “does AI do better, for us, on our topics?”
Behind every figure are the conversations. One click on any metric opens their list, already filtered, with each one’s verdict and score.
Every verdict comes with its reasoning
On each conversation, hover over the Resolved, Unresolved or Knowledge gap tag: the evaluator gives its reasoning in one sentence.
So you do not have to take a percentage on trust. You can open any line, read the reasoning and then the conversation, and form your own view. A figure you can audit is a figure you can present.
The first verified resolution rate is often lower than expected
When you move from an automation rate to a verified resolution rate, the figure drops, often sharply. Take a bot that has been in production for years, with a good automation rate and satisfied customers. In the first month of measurement, it often discovers a real resolution rate of around 40%.
That does not mean the bot was bad: the measurement made visible what had been hidden. Before measuring, you assume everything is fine; on the day you measure, everything seems broken. Neither impression is accurate.
Today, ViaSay customers in production sit between 50 and 85% resolution, measured conversation by conversation by the evaluator. The gap between the bottom and top of that range has nothing to do with the technology, which is the same for everyone.
It comes down to what each team does with the measurement: see which topic is failing, understand why, then fix one thing at a time. A “technical issues” topic can go from 32% to 64% resolution in a few weeks, when its knowledge gap falls from 33% to 11%. Once you can see what is missing, you can write it.
Measure first, improve second. A low resolution rate is not a bad grade: it shows you where to act. How to use it is the subject of a second article, the 5-step method to improve your AI chatbot’s resolution rate.
See your real resolution rate, on your conversations
A 30-minute demo of AI Quality Monitoring, on your topics and your flows, not ours.
Two metrics complement the resolution rate: knowledge gap and CX Score
The resolution rate tells you that a conversation failed, but neither why nor how the customer experienced it. The knowledge gap answers the first question, by showing whether the bot had the information. The CX Score answers the second, with a score out of 10 calculated for every conversation.
Knowledge gap. A refund policy missing from the knowledge base is a knowledge gap. On the other hand, a correct redirection to the customer account area for personal data is not: the bot does not need to know your balance.
This distinction makes the signal actionable: a topic with a 25% knowledge gap is a list of articles to write.
The CX Score combines, in a score out of 10, resolution (40%), expressed sentiment (30%) and the effort required from the user (30%). It is not a predicted CSAT: CSAT remains the customers’ actual vote, and it only covers the small minority who vote. The CX Score covers everyone.
What are the limits of an AI evaluator?
An AI evaluator is not an oracle, and knowing its limits makes the rest credible. It can be wrong, only reads the conversation, does not know your business rules and assigns only one topic per conversation. Use it as a compass, not as a contractual verdict.
It can be wrong, in both directions. On a borderline conversation, a human might judge differently. That is why every verdict is reasoned and every conversation remains readable.
It only reads the conversation. It does not know that your website was down that morning, or that the customer had already called twice.
It does not know your business rules: if the bot confidently states information that is wrong but plausible, the evaluator may count the conversation as resolved. It measures whether the request was dealt with, not whether the answer was true. Faithfulness to the source is a separate matter, handled upstream by the agent’s guardrails.
It assigns a single topic per conversation. A conversation that covers three topics is filed under one.
The evaluator shows you where to look among the thousands of conversations you will never read. That is precisely what was missing.
How do you get a resolution rate you can defend?
A resolution rate can be defended when you know what it counts, who verifies it and which conversations lie behind it. If you cannot open the conversation behind the percentage, it is not a resolution rate. Three steps take you from an automation counter to a verified measurement.
Check what your dashboard counts. A conversation without a handover is not a request dealt with.
Have every completed conversation read by an AI separate from the bot, with fixed criteria, and spot-check its verdicts on a sample.
Treat your first figure as a starting point: find the topic that is failing, understand why, then fix one thing at a time.
Comparing AI chatbots right now?
Ask each vendor how they count a resolution, and who verifies it. Then come and see how ViaSay does it, on your own conversations.
Frequently asked questions
What is a good resolution rate for an AI chatbot? Measured by reading every conversation, a resolution rate between 40 and 55% is a normal starting point for a bot in production. Bots that are monitored and corrected regularly reach 70 to 85%, especially when they are connected to business systems and able to take action. A figure of 90% or more almost always measures automation, not resolution.
What is the difference between deflection rate and resolution rate? Deflection counts conversations that did not reach a human agent, including those where the customer gave up. Resolution counts those where the request was dealt with. The gap between the two often exceeds 30 points.
How can you measure a chatbot’s resolution rate without reading every conversation? By having the conversations read by an AI separate from the bot, with fixed criteria, then spot-checking its verdicts on a sample. This is the “LLM as a judge” approach. What matters is that every verdict is reasoned and the conversation remains accessible.
Does an escalation to a human agent count as a failure? Not if it is properly managed. A bot that recognises that the request is beyond its scope and hands over cleanly has dealt with the request. A bot that refuses to escalate with no alternative, or that hands over a question it could have answered, has failed.
Does the CX Score replace CSAT? No. CSAT is the customers’ actual vote, given by a minority. The CX Score is calculated on 100% of conversations from resolution, sentiment and effort. The two should be read together.
