How do I maintain consistent naming across different platforms?

This skill provides guidance on using OpenLineage naming helpers for Snowflake, BigQuery, S3, and PostgreSQL to ensure metadata remains consistent across your ecosystem.

When should I use manual inlets and outlets in Airflow?

Use them when an operator lacks built-in OpenLineage extractors, or when you need a quick, table-level lineage setup without writing custom extraction logic.

What dataset types are supported for lineage annotation?

The skill supports OpenLineage Dataset objects, Airflow 3+ Assets, and Airflow 2.4+ Datasets.

Does this skill support column-level lineage?

No, the inlets/outlets approach is limited to table-level lineage. For column-level details, you should implement OpenLineage methods or custom extractors.

Airflow Data Lineage Annotator

Name: Airflow Data Lineage Annotator
Author: astronomer

byastronomer

•

155

•

Analytics & Monitoring

Simplifies the process of adding manual data lineage tracking to Apache Airflow tasks using inlets and outlets.

This skill provides expert guidance for annotating Airflow tasks with lineage metadata, allowing users to track data flow through operators that lack built-in OpenLineage extraction. It helps developers define input and output datasets using OpenLineage and Airflow-native formats, implements best practices through naming helpers, and clarifies precedence rules to ensure robust data observability within complex data engineering workflows. By utilizing this skill, engineers can bridge lineage gaps in their DAGs, enabling better impact analysis and data governance across platforms like Snowflake, BigQuery, and S3.

Key Features

01Support for OpenLineage Datasets and Airflow Asset types

02Manual lineage annotation using inlets and outlets

03Comprehensive decision matrix for lineage extraction strategies

04Platform-specific naming helpers for Snowflake, BigQuery, and S3

05Dynamic lineage implementation for custom operators

06155 GitHub stars

Use Cases

01Enabling lineage for operators without built-in OpenLineage extractors

02Standardizing dataset metadata across diverse cloud data warehouses

03Implementing data observability in complex ETL/ELT pipelines

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add astronomer/agents annotating-task-lineage

For use in Claude.ai and ChatGPT

Download Skill