How does it handle duplicate records?

It includes a specialized deduplication tool that can filter records based on specific unique fields, such as URLs or product IDs.

What input file formats does this skill support?

The skill is designed to handle raw data in both JSON and CSV formats commonly used by web crawlers.

Can I customize the data cleaning logic?

Absolutely. The skill provides a Python-based framework using pandas, allowing you to easily adjust formatting rules or statistical metrics.

Does the Excel report include visual charts?

Yes, it uses openpyxl to automatically generate statistical charts and summaries within the exported Excel workbook.

Web Scraper Data Processor

Name: Web Scraper Data Processor
Author: zsutxz

byzsutxz

0•

Web Scraping & Data Collection

Processes raw web scraping data into clean, formatted Excel reports with automated statistical analysis and deduplication.

This skill automates the post-collection workflow for web scraping projects by cleaning raw JSON or CSV data. It handles essential data engineering tasks such as deduplication, missing value imputation, and format standardization. By leveraging pandas and openpyxl, it transforms unstructured crawl results into professional Excel workbooks complete with statistical summaries and visual charts, making it an essential tool for developers and analysts who need to turn raw data into actionable insights.

Key Features

01High-performance deduplication for JSON and CSV datasets

020 GitHub stars

03Built-in data visualization using automated BarCharts

04Generation of multi-sheet Excel reports with statistical summaries

05Format standardization for prices and numerical fields

06Automated data cleaning and missing value filling

Use Cases

01Deduplicating and formatting lead lists extracted from web directories

02Transforming raw e-commerce scrapes into clean price-comparison spreadsheets

03Generating automated weekly market research reports from recurring crawls

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add zsutxz/claudelearning data-processor

For use in Claude.ai and ChatGPT

Download Skill