Web Scraping & Data Collection Agent Skills

arXiv Search & Metadata Retrieval

Automates the retrieval and normalization of academic paper metadata from arXiv to support research pipelines and literature reviews.

Reddit Data Connector

Extracts and analyzes Reddit content including posts, comments, subreddits, and user profiles using the public JSON API.

YouTube Transcript Extractor

Extracts and formats transcripts from YouTube videos using URLs or video IDs.

MarkItDown File Converter

Converts diverse file formats including PDFs, Office documents, and media into structured, token-efficient Markdown for LLM processing.

YouTube Transcript Extractor

Extracts text transcripts and captions from YouTube videos for content analysis, summarization, and documentation.

Tavily Web Search & Research

Empowers Claude with real-time web search, content extraction, and deep research capabilities using the Tavily API.

Tavily Web Search & Research

Enables high-quality web search, content extraction, and deep research capabilities using the Tavily API.

Apify Actorization

Converts existing applications into serverless Apify Actors for scalable web scraping and data processing workflows.

Apify Brand Reputation Monitoring

Monitor brand reputation and sentiment by scraping reviews and mentions across major social and review platforms.

Apify Actor Development

Develops, debugs, and deploys high-performance serverless actors for web scraping, data processing, and automation on the Apify platform.

Apify Market Research

Analyzes market trends, competitive pricing, and consumer behavior by extracting real-time data from major social and location-based platforms.

Competitor Intelligence by Apify

Extracts and analyzes competitor data across social media, maps, and travel platforms using specialized Apify Actors.

Apify Ultimate Web Scraper

Automates data extraction from over 55 major platforms including Instagram, TikTok, and Google Maps using AI-driven Apify Actors.

Apify Trend Analysis

Monitors and extracts emerging trends from Google, Instagram, Facebook, YouTube, and TikTok to power data-driven content strategies.

Tavily Web Search

Integrates the Tavily Search API to provide real-time, clean, and LLM-optimized web search results for RAG pipelines.

PDF Processing

Parses and extracts structured content from complex PDF documents using LlamaParse and agentic OCR capabilities.

Structured Data Extraction

Transforms unstructured files like PDFs, Word documents, and presentations into structured Pydantic models using LlamaExtract services.

Firecrawl Web Scraper

Converts websites into LLM-ready markdown and structured data using the Firecrawl API.

Exa Semantic Search

Performs semantic and neural web searches to find content based on context and meaning rather than simple keywords.

Deep Research Workflow

Orchestrates multi-agent parallel workflows to conduct comprehensive research and generate structured, insight-driven reports.

YouTube Downloader

Downloads high-quality videos and HLS streams from platforms like YouTube, Vimeo, and Mux using optimized workflows for yt-dlp and ffmpeg.

Website Clone & Replicate

Replicates existing websites into production-ready Next.js 16 and Tailwind CSS v4 codebases using Firecrawl MCP.

Firecrawl Web Scraping & Search

Scrapes, crawls, and extracts LLM-optimized content from any website using the Firecrawl CLI.

Bright Data Web Scraper

Automates web content extraction using a progressive four-tier strategy to bypass bot detection and CAPTCHAs.

Bright Data Web Scraper

Implements a four-tier progressive fallback strategy to reliably extract web content from any URL, regardless of bot detection or JavaScript requirements.

YouTube Downloader

Downloads high-quality video and audio content from YouTube and HLS-based streaming platforms while resolving common authentication and formatting issues.

Data Deduplication & Canonicalization

Implements sophisticated ID and content-based deduplication with reputation-aware canonical selection for multi-source data aggregation.

Web Research Workflow

Automates web content extraction and competitive intelligence by intelligently selecting between WebFetch, Tavily, and agent-driven browsers.

Web Research & Monitoring Workflow

Automates intelligent web content extraction and competitive intelligence gathering through a multi-tiered tool selection framework.

Web Research & Monitoring Workflow

Automates web content extraction and competitive monitoring by intelligently selecting the optimal tool for any target URL or research task.