Content Sources
Sources are the content your chatbot uses to answer questions. You can add websites to crawl, upload documents, or both. The more relevant content you add, the better your bot will perform.
Website Crawling
Our crawler visits your website and indexes all the text content. It follows links to discover pages automatically, so you don't need to add each page manually.
Add a Website Source
- Go to Sources in your dashboard
- Click Add Source
- Select Website
- Enter your website URL (e.g.,
https://yoursite.com) - Click Start Crawl
Crawl Settings
| Setting | Description |
|---|---|
| Max Pages | Maximum number of pages to crawl (based on your plan) |
| Max Depth | How many link levels deep to crawl (1-5) |
| Include Subdomains | Also crawl blog.yoursite.com, etc. |
WooCommerce Products
If your site runs WooCommerce, we can automatically detect and index your products with structured data - names, descriptions, prices, variations, and stock status.
Enable WooCommerce Indexing
- Go to Sources in your dashboard
- Add a new website source or edit an existing one
- Enable the WooCommerce Products toggle
- Start the crawl
With WooCommerce indexing enabled, your bot can answer questions like:
"Do you have the blue hoodie in size XL?"
"Yes! The Classic Blue Hoodie is available in XL for $59.99. We currently have 12 in stock."
Document Upload
Upload PDFs, Word documents, text files, or markdown to add content that isn't on your public website. Great for FAQs, product manuals, internal documentation, or policies.
Supported Formats
- PDF - Product manuals, brochures, reports
- DOCX - Word documents
- TXT - Plain text files
- MD - Markdown documentation
Upload a Document
- Go to Sources in your dashboard
- Click Add Source
- Select Document
- Drag and drop your file or click to browse
- Wait for processing to complete
Keeping Content Fresh
When your website content changes, you'll want to re-crawl to update your bot's knowledge. Re-crawling doesn't count against your page quota - you can do it as often as you need.
Re-crawl a Source
- Go to Sources in your dashboard
- Find the source you want to update
- Click the Re-crawl button
- Wait for the crawl to complete
Managing Sources
View Indexed Pages
Click on any source to see the list of pages that were indexed. You can see the page title, URL, and when it was last crawled.
Delete a Source
To remove a source and all its indexed content, click the delete button next to the source. This immediately removes the content from your bot's knowledge base.