How you build and maintain your knowledge base through intelligent website scraping
OpsPilot’s newest feature revolutionizes how you build and maintain your knowledge base through intelligent website scraping. This powerful addition lets you automatically capture and organize web content, making critical information instantly accessible to your team. Let’s explore how to harness this capability.
Getting Started with Website Sources
Setting up your first website source is straightforward:
- Launch the Knowledge Hub: Navigate to the Knowledge section in OpsPilot Hub and select Website from the Add Knowledge dropdown menu.
- Configure Your Source: Enter your desired URL in the Base URL field. For precise control, remember that OpsPilot will only scrape subpaths under your specified URL. For example, entering example.com/docs/observability will exclude content from paths like example.com/product.
- Organize with Tags: Apply relevant tags to categorize and easily retrieve your content later.
Begin Scraping: Click Add website to start the process or Close to cancel anytime.
Fine-Tune Your Scraping Strategy
OpsPilot offers advanced configuration options to ensure you capture exactly what you need:
- Custom Metadata: While OpsPilot can automatically generate titles and descriptions, you can override these with your own to better suit your organization’s needs.
- Scraping Depth: Control how deep OpsPilot should traverse from your base URL. For instance, with a depth of 2 from example.com/docs, you’ll capture content up to paths like example.com/docs/observability/about.
Page Limits: Set a maximum number of pages to scrape, or leave this field empty for unlimited collection.
Keep Your Knowledge Fresh
Maintaining an up-to-date knowledge base is crucial. OpsPilot provides two essential maintenance tools:
- Rescrape: Use the **Rescrape** button to refresh your content, ensuring your knowledge base reflects the latest updates from source websites.
Clean Up: Remove outdated or unnecessary content using the Delete button (bin icon) to keep your knowledge base lean and relevant.
Transform Your Knowledge Management
Website scraping in OpsPilot marks a significant advancement in knowledge base management. This feature streamlines how teams access and maintain critical documentation by automating the content collection and providing granular control over your information sources. Explore these capabilities today to build a more dynamic and comprehensive knowledge base.
