How does it handle complex error management?

The skill provides templates for task-level and DAG-level failure callbacks, Slack/PagerDuty notifications, and advanced trigger rules for cleanup operations.

Can I use this for dynamic DAG generation?

Absolutely. It includes specific patterns for creating multiple DAG instances from external configurations, JSON files, or database metadata.

Does this skill support Airflow 2.0+ features?

Yes, it prioritizes modern patterns such as the TaskFlow API, decorators, and updated provider packages introduced in Airflow 2.0 and later.

Are there built-in testing strategies for DAGs?

Yes, it includes standardized patterns for using pytest to ensure DAGs load without cycle errors and that logic remains consistent during deployments.

Airflow DAG Patterns

Name: Airflow DAG Patterns
Author: gwickman

bygwickman

0•

Deployment & DevOps

Builds production-grade Apache Airflow DAGs using industry best practices for orchestration, testing, and error handling.

This skill provides Claude with specialized knowledge for architecting and implementing robust Apache Airflow pipelines. It covers modern patterns including the TaskFlow API, dynamic DAG generation, complex branching logic, and custom sensors. Whether you are building idempotent ETL processes, setting up cross-DAG dependencies, or implementing sophisticated retry policies and failure callbacks, this skill ensures your data orchestration follows production-ready standards and remains maintainable through standardized testing patterns.

Key Features

01Dynamic DAG generation for scalable pipeline management

02Standardized unit testing patterns for DAG integrity validation

030 GitHub stars

04Advanced sensor patterns for S3, external tasks, and custom APIs

05Comprehensive error handling with failure callbacks and trigger rules

06Implementation of Airflow 2.0+ TaskFlow API and decorators

Use Cases

01Automating the creation of multiple similar DAGs from configuration files

02Implementing cross-pipeline dependencies and external task synchronization

03Designing idempotent and atomic ETL pipelines for large-scale data processing

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add gwickman/auto-dev-test-target-1 airflow-dag-patterns

For use in Claude.ai and ChatGPT

Download Skill