How does the skill decide which research tool to use?

It follows a prioritized decision tree: starting with WebFetch for speed, escalating to Tavily for rich extraction, and falling back to a full agent-browser for JS-heavy or authenticated sites.

Can it track changes on competitor websites?

Yes, it includes specialized logic for snapshot diffing and change classification, allowing it to alert you to price changes, feature updates, or positioning shifts.

Is a Tavily API key required to use this skill?

A Tavily key is recommended for high-quality semantic extraction, but if missing, the workflow automatically defaults to a 2-tier system using WebFetch and the agent-browser.

Does it support JavaScript-heavy Single Page Applications (SPAs)?

Yes, when static extraction fails, the workflow triggers a browser-based agent to wait for network idle or specific text before capturing content.

Web Research & Monitoring Workflow

Name: Web Research & Monitoring Workflow
Author: yonatangross

byyonatangross

•

Web Scraping & Data Collection

Automates web content extraction and competitive monitoring by intelligently selecting the optimal tool for any target URL or research task.

This skill provides a unified decision-making engine for Claude Code to perform deep web research and competitive analysis. It intelligently navigates between WebFetch for quick data, Tavily for semantic extraction and batch processing, and a full agent-driven browser for complex, JavaScript-rendered sites. Beyond simple scraping, it features specialized logic for tracking competitor changes, classifying the severity of updates, and extracting clean markdown from documentation or marketing pages, ensuring developers and analysts get the highest quality web data with minimal manual overhead.

Key Features

01Automated competitive monitoring with snapshot diffing and alerting

02Intelligent 3-tier tool selection (WebFetch, Tavily, Agent-Browser)

03High-quality markdown extraction from documentation and articles

04Structured data extraction and change classification patterns

05Advanced handling for SPAs and JavaScript-heavy websites

0692 GitHub stars

Use Cases

01Monitoring competitor pricing, features, and positioning shifts

02Extracting clean documentation and API references for software development

03Automated data collection from complex, authenticated web portals

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yonatangross/orchestkit web-research-workflow

For use in Claude.ai and ChatGPT

Download Skill