How does it handle data quality?

It integrates data quality tests and validation patterns using dbt testing frameworks and tools like Great Expectations to ensure reliable analytical outputs.

Does this skill support dbt incremental models?

Yes, it provides patterns for implementing incremental data loads and merge strategies to handle large datasets efficiently while minimizing compute costs.

Which orchestration tools are covered?

The skill includes implementation patterns for industry-standard tools like Apache Airflow, Dagster, and Prefect, helping you choose the best fit for your architecture.

Can I use this for performance optimization?

Absolutely. It includes specific migration guides for moving from pandas to polars or PySpark to handle larger data volumes and significantly increase processing speed.

Data Transformation & Pipelines

Name: Data Transformation & Pipelines
Author: ancoleman

byancoleman

•

158

•

Database Management

Transforms raw data into clean, analytical assets using modern ETL/ELT patterns, SQL (dbt), and high-performance Python frameworks.

This skill enables Claude to architect and implement robust data transformation workflows across the modern data stack. It provides comprehensive guidance for choosing between ETL and ELT patterns, building modular SQL models with dbt, and performing high-performance data processing with Python libraries like pandas, polars, and PySpark. Whether you are migrating legacy pipelines, implementing incremental data loads, or orchestrating complex multi-step workflows with Airflow, this skill ensures your data pipelines are production-ready, testable, and optimized for scale.

Key Features

01Orchestrates complex workflows with Airflow, Dagster, or Prefect

02Develops modular dbt models with staging, intermediate, and marts layers

03Implements ETL and ELT transformation patterns for modern cloud warehouses

04Ensures data integrity with integrated testing, validation, and quality checks

05Optimizes Python data processing using polars and PySpark for high performance

06158 GitHub stars

Use Cases

01Building incremental dbt models for large-scale cloud data warehouses

02Migrating performance-bottlenecked pandas scripts to polars or PySpark

03Designing production-ready Airflow DAGs with retries and dependency logic

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add ancoleman/ai-design-components transforming-data

For use in Claude.ai and ChatGPT

Download Skill