Knowledge Optimizer

The Knowledge Optimizer is the hub for keeping your chatbot accurate. Instead of one tool, it bundles five tabs that each tackle a different part of answer quality — from generating test questions to mining gaps out of real visitor conversations, auditing your knowledge base for contradictions, rewriting the bot's role, and comparing AI models head to head.

Pick a chatbot at the top of the page; that selection is shared across every tab. Each tab runs its own analysis and, where relevant, drops improvement suggestions into the shared Optimization Inbox for you to review and apply.

The Tabs at a Glance

Optimize Tab — Mine Gaps From Real Conversations

The Optimize tab finds what your bot actually failed to answer, using real visitor history rather than guesses. It pulls in unanswered and collected questions from the selected time window, groups them by topic, and generates editable content suggestions to fill each gap.

How to run it:

1. Choose a window: Last 30 days or Last 90 days
2. Click Analyze
3. The tool collects unanswered questions, clusters them by topic, then analyzes each cluster and drafts content

Each run uses up to 10 AI analyses and consumes no chat messages. Results land in the Optimization Inbox below as Knowledge gap suggestions. A suggestion that came from several real questions shows how many occurrences it represents, so you can prioritize the most common gaps first.

If the window has no unanswered questions, the tab tells you there's nothing to optimize right now.

Audit Tab — Find Problems in Your Knowledge Base

The Audit tab checks the content you already have, not what's missing. It scans your indexed sources for three problems and proposes fixes:

How to run it:

1. Click Run audit
2. The tool scans your knowledge base, then asks an AI judge to check the suspicious pairs it found
3. Findings appear in a review list with the affected excerpts side by side

Review the findings, keep the ones worth acting on (each has a checkbox), and click Save to inbox. Where an automatic rewrite is possible, the finding is marked Auto-fix available and you can apply the corrected text directly from the inbox. For crawled or synced sources, the suggestion only flags the issue — you fix it at the origin. If nothing is wrong, the audit reports a healthy knowledge base.

Role Tab — Tune the Bot's System Prompt

The Role tab improves your chatbot's role (its system prompt). It reviews failed test questions and known knowledge gaps, then proposes a rewritten role aimed at better answers.

How to run it:

1. Click Propose improved role
2. Compare the current role and the proposed role side by side — the proposed text is editable, and a rationale explains what changed and why
3. Validate it with an A/B test: choose a question source (generate 5 fresh test questions or reuse the questions from your last test run) and click Run A/B test
4. The same questions are asked with both roles, an AI judge picks the better answer per question, and you get a win rate plus a per-question breakdown
5. If the new role wins, click Apply role to make it live

The A/B test asks each question twice (current vs. proposed), so it uses roughly 20 test chat messages from your monthly quota. Applying the role requires the chatbots permission.

Model Arena Tab — Compare AI Models

The Model Arena runs the same set of questions against 2–3 AI models in parallel and compares them on answer quality (scored by an AI judge) and median response time. Use it to decide whether a faster or cheaper model holds up on your real content.

How to run it:

1. Select 2–3 models to compare (your current model is marked)
2. Pick a question source (generate fresh questions or reuse the last run)
3. Click Start comparison
4. Review the results table — quality percentage, median latency, and a recommendation for the best balance of the two
5. Expand any question to see each model's actual answer
6. Click Use for this bot on a model to switch to it

This tab uses test chat messages (counted toward your monthly quota) and applying a model requires the chatbots permission.

Knowledge Test Tab — Auto-Test Your Knowledge Base

The Knowledge Test tab is the classic Knowledge Tester. It automatically generates realistic test questions, runs them against your chatbot, evaluates the answers with AI, and helps you fix any gaps.

Test Modes

Auto Test is best for a broad health check of your whole knowledge base. Custom Question is ideal for checking whether a specific topic or recently added content is answered correctly.

Running a Test

1. Choose the test mode: Auto Test or Custom Question
2. Set the number of questions (Auto Test) or variants (Custom Question)
3. For Custom Question mode: enter your question
4. Click Start Test

The system then runs three steps automatically:

Evaluation Criteria

For Partial and Not Answered results, the AI explains what's missing or wrong. Use those recommendations to guide your fixes.

Results Dashboard

After a test completes, four cards summarize it: Answered, Partial, Not Answered, and an overall Score (answered ÷ total) with a color-coded bar:

• Green (80%+) — your knowledge base covers the tested topics well
• Amber (50–79%) — some gaps to address
• Red (below 50%) — significant gaps need attention

Each tested question appears as a result card with the question, an evaluation badge, the bot's full answer, a recommendation (for Partial/Not Answered), and action buttons: Fix with AI, Try again, and Find Source.

Fix with AI

Fix with AI is the main way to resolve gaps found during testing. It analyzes all failed questions, searches your existing knowledge base for context, and generates content to fill the gaps.

1. Click Fix with AI on a Partial or Not Answered result (or on the results summary)
2. The system gathers all failed questions and pulls related content from your knowledge base
3. AI drafts text covering the missing information, using concrete data from your sources where available
4. Review and edit the draft; optionally set a document name (left empty, one is auto-generated)
5. Click Add as Data Source to save it as a new text data source

After saving, you're prompted to verify the fix: Retest all failed, Retest first (a quick single-question check), or Later. The generated content shows up in your chatbot's Data Sources tab, where it stays editable.

Find Source

Find Source locates which existing data sources are relevant to a question — useful when you'd rather edit an existing source than add new content.

1. Click Find Source on a result card
2. The system searches your knowledge base using vector similarity (with text search as a fallback)
3. Up to 10 matches are shown, each with source name, type, a relevance score and a 500-character content preview

From there you can Edit a text source directly or Open in Dashboard to jump to that source in the Data Sources tab.

Retrying & History

• Try again re-runs a single question and updates its answer, evaluation and the overall score in place.
• Retest all failed (after Fix with AI) re-runs every Partial and Not Answered question in one batch.
• The Test History section lists your last 20 test runs per chatbot — score, mode, chatbot and date. Click any entry to reopen its full results.

Limits

The Optimization Inbox

The Optimization Inbox sits below the Optimize and Audit tabs and collects the AI's improvement suggestions in one place. Each suggestion has a type — Knowledge gap, Contradiction, Duplicate, Thin content or Role proposal — and a status:

• Open — waiting for your decision
• Accepted — applied to your chatbot
• Dismissed — set aside (you can reopen it later)

Click a suggestion to open its detail view. Depending on the type you'll see the real questions behind a gap, the affected source excerpts, or a current-vs-proposed comparison. Where the suggestion includes editable content, you can tweak it before applying. Click Add to knowledge base (or Apply rewrite / Apply role, depending on type) to apply it, or Dismiss to skip it.

Best Practices

• Start with Optimize — real conversations show you the gaps that actually cost you answers
• Run Audit after big imports — catch contradictions before visitors hit them
• Use Knowledge Test to verify — after adding content, confirm the bot now answers correctly
• A/B test role changes — don't ship a new prompt on a hunch; the Role tab proves the lift
• Try Model Arena before switching models — make sure a faster model still answers well on your data
• Cross-check with the Questions and Analytics dashboards — watch the Unanswered metric drop as you close gaps