Does this skill support modern Python versions?

Yes, it is specifically optimized for Python 3.12+ and modern tooling like uv, Ruff, and Pydantic v2.

How does this help with processing large datasets?

It implements generator-based patterns and chunked reading strategies to ensure data is processed efficiently without causing MemoryErrors.

What libraries does this skill prefer for data manipulation?

While it supports standard libraries, it emphasizes high-performance alternatives like Polars for DataFrame operations and httpx/aiohttp for networking.

Can it help with API data ingestion?

Absolutely. It includes advanced async/await patterns with built-in concurrency limits and exponential backoff retry logic for reliable data fetching.

Python Programming for Data Engineering

Name: Python Programming for Data Engineering
Author: pluginagentmarketplace

bypluginagentmarketplace

•

Data Science & ML

Develops production-grade Python scripts for scalable ETL pipelines and high-performance data processing systems.

This skill equips Claude with the specialized patterns and best practices required to build robust data infrastructure using modern Python 3.12+. It focuses on high-performance execution, emphasizing memory-efficient processing via generators, type-safe data structures with Pydantic, and asynchronous I/O for concurrent data ingestion. Designed for data engineers, it ensures that generated code adheres to 2025 industry standards, including modern dependency management with uv and linting with Ruff, while providing specific solutions for common failure modes like GIL bottlenecks and memory overflows.

Key Features

01Type-safe data modeling with dataclasses and Pydantic v2

02Modern toolchain support including uv, Ruff, and Polars

03High-concurrency I/O patterns with async/await and backoff retries

042 GitHub stars

05Memory-efficient chunked processing using Python generators

06Integrated performance profiling and structured logging

Use Cases

01Optimizing memory-intensive scripts for constrained environments

02Developing high-performance data processing tools with Polars

03Building scalable ETL pipelines for large-scale data ingestion

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add pluginagentmarketplace/custom-plugin-data-engineer python-programming

For use in Claude.ai and ChatGPT

Download Skill