Does this skill support Airflow 2.0 features?

Yes, it prioritizes modern Airflow 2.x patterns, specifically emphasizing the TaskFlow API (@dag and @task decorators) for simplified data passing and cleaner code.

Does it provide assistance with Airflow testing?

Yes, the skill provides standardized patterns for testing DAG integrity and loading using Pytest and the Airflow DagBag utility.

Can I implement custom alerting and error handling?

Yes, it includes templates for failure callbacks and trigger rules like ALL_DONE or NONE_FAILED to ensure your pipelines handle errors gracefully and notify teams via Slack or email.

Can I use this for complex task dependency management?

Absolutely. It includes implementation patterns for fan-in/fan-out structures, branching logic, and cross-DAG dependencies using ExternalTaskSensors.

Airflow DAG Patterns

Name: Airflow DAG Patterns
Author: wshobson

bywshobson

•

23,194

•

Productivity & Workflow

Builds production-grade Apache Airflow workflows using best-practice design patterns for task orchestration and data pipelines.

This skill equips Claude with the patterns and best practices needed to engineer robust Apache Airflow DAGs. It covers essential orchestration concepts such as idempotency, TaskFlow API implementation, dynamic DAG generation, and complex branching logic. Whether you are building an ETL pipeline, setting up S3 sensors, or implementing custom error-handling callbacks, this skill provides the implementation templates and testing strategies required for scalable, production-ready data engineering and workflow automation.

Key Features

01Modern TaskFlow API implementation for cleaner, decorator-based workflows

02Advanced orchestration logic including branching, conditional execution, and trigger rules

0323,194 GitHub stars

04Dynamic DAG factory patterns for programmatically generating multiple pipelines

05Robust sensor patterns for S3, APIs, and external task dependencies

06Standardized error handling, retry policies, and notification callback templates

Use Cases

01Architecting scalable ETL/ELT data pipelines with automated retries and idempotency

02Migrating legacy PythonOperator workflows to the modern Airflow 2.x TaskFlow API

03Implementing cross-DAG dependencies and automated data quality checks

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add wshobson/agents airflow-dag-patterns

For use in Claude.ai and ChatGPT

Download Skill