# Automate Your Knowledge Base with Web Scraping (BETA)

Creating a high-quality AI agent starts with high-quality knowledge. At VoiceB.ai, we’ve made that easier than ever — by introducing a **new beta feature** that automatically generates a `.txt` knowledge base file by scraping your public-facing web content.

Whether it’s your **homepage, support center, blog, or help docs**, VoiceB can now scan, structure, and format your content into a clean `.txt` file ready to power your AI assistant.

This feature is currently in **BETA**. To request early access, reach out to **<hello@voiceb.ai>**.

### 🔍 What does this feature do?

When activated, VoiceB’s Web Scraper:

* Crawls all public pages linked from your website, including:
  * Product pages
  * Pricing
  * Help center / FAQs
  * Blog posts
  * Documentation and resources
* Extracts useful content: headlines, paragraphs, lists, answers, call-to-actions
* Cleans and compiles the information into a **structured `.txt` file** formatted for optimal AI comprehension

This file is then used as the **knowledge base** to train your VoiceB agent. No need to write prompts from scratch or manually input support topics.

***

### 🧠 Why it matters

Your AI agent is only as good as the knowledge it has. With this feature, we remove the manual work and ensure your assistant:

✅ Stays **up to date**\
We refresh your `.txt` file daily to reflect website changes — no need to re-train your agent.

✅ Covers all your public info\
From product specs to FAQs, the AI has full access to your published content.

✅ Speaks your language\
It learns your tone, brand, and messaging by absorbing the way you present your content online.

✅ Saves time\
No more copying and pasting. Your knowledge base builds itself.

***

### 📂 What’s inside the .txt file?

Here’s a simplified example:

```
vbnetCopyEditSection: Plans

The Mobile Unlimited plan costs $15/month and includes unlimited calls and 50GB at 5G speeds.

The Fiber 600Mb plan includes internet and mobile for $35/month.

Section: FAQs

Q: How long does installation take?  
A: Installation usually takes 1–3 days depending on availability.

Q: Can I change my plan later?  
A: Yes, all plans are flexible and can be upgraded at any time.
```

> This format is optimized for AI processing, ensuring accurate and human-like responses from your agent.

***

### ⚠️ Beta Notice

This feature is still in **BETA**, which means:

* Scraping coverage may vary depending on your site structure
* You’ll have the option to review, edit, or expand your `.txt` file before use
* Feedback is welcome! Help us improve the feature

To enable web scraping and auto-generated knowledge base, please contact **<hello@voiceb.ai>**.

***

### 🗣️ Final Thoughts

With this new beta feature, VoiceB helps you launch faster and smarter by turning your website into a living source of truth for your AI agent.

This is especially valuable for:

* SaaS companies with complex products
* eCommerce brands with evolving catalogs
* Telcos, banks, and insurers with dynamic pricing and FAQs
* Any team that updates its site more often than it updates documentation

Ready to let your website teach your AI?

👉 **Request early access today** by emailing **<hello@voiceb.ai>**.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.voiceb.ai/product-guides/automate-your-knowledge-base-with-web-scraping-beta.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.