Knowledge Tester

The Knowledge Tester is a quality assurance tool that automatically tests how well your chatbot answers questions based on its knowledge base. It generates realistic test questions, runs them against your chatbot, evaluates the answers with AI, and helps you fix any knowledge gaps — all from a single dashboard.

Test Modes

Auto Test is best for a broad health check of your entire knowledge base. Custom Question is ideal for testing whether a specific topic or recently added content is being answered correctly.

Running a Test

1. Select a chatbot from the dropdown
2. Choose the test mode: Auto Test or Custom Question
3. Set the number of questions (Auto Test) or variants (Custom Question)
4. For Custom Question mode: enter your question in the text field
5. Click Start Test

The system then executes three steps automatically:

Evaluation Criteria

Every answer is classified into one of three categories:

For Partial and Not Answered results, the AI provides a specific recommendation explaining what information is missing or incorrect. Use these recommendations to guide your content improvements.

Results Dashboard

Score Overview

After a test completes, four KPI cards summarize the results:

Score color thresholds:

• Green (80% and above) — Your knowledge base covers the tested topics well
• Amber (50–79%) — Some gaps exist that should be addressed
• Red (below 50%) — Significant knowledge gaps need attention

Individual Results

Each tested question is displayed as a result card showing:

• Question — The exact question that was tested
• Evaluation badge — Color-coded label (Answered / Partial / Not Answered)
• Answer — The chatbot's full response, rendered as formatted text
• Recommendation — For Partial and Not Answered results, a specific suggestion on what information is missing
• Action buttons — Fix with AI, Try again, and Find Source (see sections below)

The results header also shows how many messages from your quota were used during the test.

Fix with AI

Fix with AI is the primary tool for resolving knowledge gaps found during testing. It analyzes all failed questions, searches your existing knowledge base for context, and generates comprehensive content to fill the gaps.

1. Click Fix with AI on any Partial or Not Answered result (or on the results summary)
2. The system analyzes all failed questions and searches your knowledge base for related content
3. AI generates a text covering all missing information, using concrete data from your existing sources where available
4. Review and edit the generated content in the text editor
5. Optionally set a document name (left empty, a name is auto-generated based on the content)
6. Click Add as Data Source to save the content as a new text data source

After the data source is added, you are prompted to verify the fix:

The generated content is added as a plain text data source, visible and editable in your chatbot's Data Sources tab.

Find Source

Find Source helps you locate which existing data sources are relevant to a specific question. This is useful when you want to edit an existing source rather than creating new content.

1. Click Find Source on any result card
2. The system searches your knowledge base using vector similarity search (with text search as fallback)
3. Up to 10 matching results are displayed

Each result shows:

From the results, you can:

• Click Edit to modify a text data source directly
• Click Open in Dashboard to navigate to the chatbot's Data Sources tab for that source

Use the search field at the top to search with different terms if the initial results are not helpful.

Retrying Questions

After making changes to your knowledge base, you can re-run individual questions to verify improvements:

• Try again — Click this button on any result card to re-run a single question. The answer and evaluation are updated in place, and the overall score is recalculated immediately. A success or failure message indicates whether the answer improved.
• Retest all failed — Available after using Fix with AI. Re-runs all Partial and Not Answered questions in one batch. A progress indicator shows the current question being tested. After completion, a summary shows how many questions are now answered.

Test History

When no active test result is displayed, the Test History section shows your previous test runs:

• Displays the last 20 test runs for the selected chatbot (or all chatbots if none is selected)
• Each entry shows:
  • Score — Percentage with color-coded badge and ratio (e.g. "4/5")
  • Test mode — Auto Test or Custom Question badge
  • Chatbot name — Which chatbot was tested
  • Date — When the test was run
  • Custom question — For Custom Question tests, the original question is shown
• Click any entry to view its full results
• Hover over an entry to reveal the delete button
• Use the chatbot dropdown to filter history by a specific chatbot

Limits & Constraints

Best Practices

• Run auto tests after adding data sources — Verify your new content is being picked up and used correctly by the chatbot
• Use custom questions for specific topics — Test edge cases, recently added content, or particular subjects in depth with multiple phrasings
• Fix gaps immediately — Use Fix with AI to generate missing content and add it as a data source right away while the context is fresh
• Always retest after fixing — Confirm your fixes actually improved the chatbot's answers before moving on to the next issue
• Combine with the Questions dashboard — Cross-reference test results with real visitor questions in the Questions Dashboard for a complete picture of knowledge gaps
• Monitor your Analytics — Track the Unanswered metric in the Analytics Dashboard to measure overall improvement over time
• Test regularly — Make knowledge testing part of your routine chatbot maintenance, especially after content updates or re-indexing
• Prioritize Not Answered over Partial — Focus on completely unanswered questions first, as they represent the biggest gaps in your knowledge base