URL Search Tool
URL Search lets the chatbot fetch a page live, in the middle of a conversation, instead of relying on content you indexed earlier. Use it for things that change constantly — current prices, availability, opening hours, today's offers. When a visitor asks something that matches one of your URL groups, the bot pulls those pages fresh and answers from what it just read.
Everything else should stay in your normal data sources — they're faster and cheaper. URL Search is the exception for content that goes stale by the hour.
URL Search is available on Standard plan and higher.
Configuration
| Field | Description | Default |
|---|---|---|
| Enable URL Search | Activates real-time URL searching for this chatbot. | Off |
URL Groups
URLs are organized into groups. Each group defines a set of URLs and a prompt that tells the chatbot when to search them.
You can create up to 10 URL groups per chatbot.
Group Fields
| Field | Description |
|---|---|
| Group Name | A descriptive name for this URL group (e.g. "Weather", "Stock Prices", "Support Articles"). |
| Prompt | Instructions for when the chatbot should search these URLs (e.g. "Search these URLs when the user asks about current weather conditions"). |
| URLs | One URL per line. Only http:// and https:// URLs are accepted. Up to 10 URLs per group. |
Example Configuration
Group: Product Prices
- Name: Product Prices
- Prompt: Search these URLs when the visitor asks about current prices, pricing, or product costs.
- URLs:
https://shop.example.com/pricing https://shop.example.com/products
Group: Support Articles
- Name: Support Articles
- Prompt: Search these URLs when the visitor has a technical issue or asks about troubleshooting.
- URLs:
https://help.example.com/faq https://help.example.com/troubleshooting
How It Works
- The visitor asks a question
- The chatbot evaluates the question against each group's prompt
- If a group matches, the chatbot fetches the URLs in real-time
- The fetched content is used to generate the answer
- The visitor gets an answer based on the latest content
URL Search vs. Standard Data Sources
| Feature | URL Search | Standard Data Sources |
|---|---|---|
| Content freshness | Always current (fetched in real-time) | Updated on re-index |
| Response speed | Slightly slower (needs to fetch) | Fast (pre-indexed) |
| Best for | Dynamic, frequently changing content | Static or rarely changing content |
| Cost | Uses more tokens per query | Pre-processed, lower per-query cost |
Best Practices
- Use for dynamic content only — Static content is better served by standard data sources
- Write clear prompts — The prompt determines when the chatbot uses each URL group
- Limit URLs per group — Fewer URLs means faster responses
- Combine with data sources — Use URL Search for dynamic data and standard sources for static content