Enables open-source web scraping and multimodal data extraction for AI model context and data gathering.
Sponsored
WET (Web ExTract) is an open-source Model Context Protocol (MCP) server designed to empower AI models with comprehensive web scraping and multimodal data extraction capabilities. Replacing services like Tavily, it offers robust features for real-time web search, clean content extraction (Markdown/Text), deep crawling with depth control, and website structure mapping. Crucially, WET includes advanced anti-bot measures, allowing it to bypass common protections like Cloudflare, Medium, LinkedIn, and Twitter, ensuring reliable access to web content for diverse AI applications.
Key Features
01Clean Content & Multimodal Media Extraction (Markdown/Text, images, videos, audio)
02Deep Web Crawling with configurable depth
032 GitHub stars
04Website URL Structure Discovery (Site Map)
05Advanced Anti-bot Bypass for Cloudflare, Medium, LinkedIn, and Twitter
06Integrated Web Search via embedded SearXNG
Use Cases
01Automating content acquisition while bypassing anti-bot systems for research or analysis
02Providing real-time web context for AI agent decision-making
03Collecting structured and unstructured data for AI model training
