Scrapes, indexes, and searches documentation websites to create a local, highly-searchable knowledge base.
Sponsored
DocShark functions as a Model Context Protocol (MCP) server, designed to automate the process of transforming public documentation websites into a powerful, local knowledge base. It efficiently scrapes, indexes, and enables full-text search across documentation, leveraging technologies like FTS5 and BM25 ranking. This capability allows AI assistants and other MCP-compatible clients to effortlessly query the most current information, ensuring they always have access to up-to-date documentation.
Key Features
01Automated crawling via sitemaps or BFS with respect for robots.txt
02Smart content extraction to clean Markdown, filtering out non-essential elements
03Semantic chunking of content based on headings for improved AI understanding
04High-performance full-text search with FTS5 and BM25 ranking
05Automatic detection and support for JavaScript-rendered Single-Page Applications (SPAs)
061 GitHub stars
Use Cases
01Creating local, searchable knowledge bases for AI assistants
02Archiving and providing fast access to public documentation
03Integrating with MCP-compatible clients like VS Code Copilot agents
