Web Scraping & Data Collection Agent Skills

Industry Research

Conducts comprehensive market analysis and trend forecasting across the consumer, technology, healthcare, and finance sectors.

PySpider Development Toolkit

Streamlines the creation of production-ready PySpider web crawlers using standardized templates, advanced scraping patterns, and enterprise-grade best practices.

Competitive Research with Bright Data

Conducts enterprise-grade company research, competitive analysis, and market intelligence using professional web scraping and search tools.

YouTube Info MCP Installer

Installs and configures the YouTube Info MCP server to enable automated extraction of video metadata and details within Claude Code.

Pure.md MCP Installer

Automates the installation and configuration of the Pure.md MCP server for seamless web-to-markdown conversion within Claude Code.

Google Trends ATH Detector

Detects and analyzes Google Trends search spikes and all-time highs using automated browser simulation and statistical modeling.

YouTube Video Downloader

Downloads YouTube videos and audio with customizable quality and format settings directly through the command line.

YT-DLP Video Downloader

Downloads audio and video from thousands of websites with advanced control over formats, subtitles, and metadata.

Crawl4AI

Crawls entire websites and extracts clean, structured content into markdown files with AI-enriched metadata.

Rolex Market Liquidity Proxy Analyzer

Analyzes global liquidity and risk-on sentiment by using the Rolex Market Index as a high-beta proxy for financial conditions.

Insight Extractor

Extracts valuable insights and key information from URLs or text documents to streamline information processing.

APOD Downloader

Downloads NASA's Astronomy Picture of the Day along with detailed metadata directly to your desktop.

Video Downloader

Automates video and audio downloads from various web platforms with customizable quality and format settings using yt-dlp.

YouTube Transcript Extractor

Extracts high-quality subtitles and transcripts from YouTube videos in multiple formats and languages for analysis and automation.

Competitive Ads Extractor

Extracts and analyzes competitor advertisements from major ad libraries to provide actionable messaging and creative insights.

Persistent Sector Caching

Optimizes yfinance data retrieval by implementing a permanent SQLite caching layer to bypass API rate limits and 401 errors.

RSS Feed Operations

Reads and processes RSS/Atom feeds from any blog URL with intelligent automatic feed discovery.

Competition Scraper & GitHub Persister

Scrapes Australian creative writing competitions and automatically manages them as structured GitHub issues with intelligent duplicate detection.

bioRxiv Database Search

Searches and retrieves life sciences preprints from the bioRxiv database using keywords, authors, and DOIs.

Website Crawler

Maps complex website structures and discovers URLs using a high-performance Go-powered engine with TypeScript integration.

Structured Evidence Collection

Standardizes research by creating traceable YAML evidence objects with semantic IDs and confidence scores to ensure data-driven decision making.

Gemini Web Search

Enhances Claude with advanced web search capabilities using Gemini to retrieve and synthesize real-time information from the internet.

Deep Research Diffusion

Conducts exhaustive, multi-perspective research using parallel agent workers and iterative gap detection.

Web Research Pro

Verifies technical requirements and gathers real-time information through structured web searches and documentation analysis.

Structured Web Research

Conducts systematic web investigations by deploying parallel subagents to gather, verify, and synthesize information from multiple sources.

Web Research & Synthesis

Conducts multi-threaded, deep-dive web investigations by orchestrating parallel subagents to gather and synthesize complex information.

Web Research & Synthesis

Conducts structured, multi-threaded web research by delegating sub-tasks to specialized agents and synthesizing findings into comprehensive reports.

YouTube Video Downloader

Downloads YouTube videos and audio with customizable quality, format, and output settings using yt-dlp.

Competitive Ads Extractor

Scrapes and analyzes competitor advertisements from major ad libraries to uncover successful messaging and creative patterns.

Competitive Ads Extractor

Scrapes and analyzes competitor advertisements from major ad libraries to uncover successful messaging, creative patterns, and market positioning.