Does it answer only from your content?
A chatbot that answers from everything it knows will eventually invent a policy. Ask how the answer is produced and whether the source is shown to the customer. Prefer tools that cite the passage behind each reply.
- Source shown under every answer
- A clear flag when the content has no answer
- No answers about topics you did not publish
What happens when it cannot answer?
Every bot reaches the edge of its content. Check what the customer sees next. The best behavior is an honest message and a handoff to a person with the question attached.
- Handoff with the full question
- No dead ends or loops
- A way to see which questions failed
How hard is it to set up and maintain?
Ask how content gets in and how long a change takes to appear in answers. Tools that need conversation trees take ongoing work. Tools that read your content let you maintain pages instead.
- Setup time for your first hundred pages
- How updates reach the answers
- Whether non-developers can manage it
How is it priced?
Compare the real monthly cost at your volume. Look at whether the price counts answers, conversations, credits, or seats, and what happens when you pass the limit. A plan you can predict is worth more than a low entry price.
- What counts toward the limit
- What happens after the limit
- Whether the API costs extra
Can your developers and your support team both use it?
Support wants a widget and analytics. Developers want an API with a predictable response. Check that one product covers both, so the answer is the same everywhere.
- Widget for the site
- API with structured output
- Seats for your team
Questions to ask every vendor
Bring this list to every demo. Run the same five customer questions through each tool, using your own content, and compare the answers and the sources.
- Where does each answer come from
- What does the customer see when the answer is missing
- Which plan includes the API
- How is usage counted
- How do I cancel