Is it compatible with orchestration tools?

Yes, the patterns are designed to be used within orchestration frameworks like Airflow, Dagster, or dbt for post-load transformations.

Does this skill support SQLAlchemy 2.0?

Yes, it specifically focuses on SQLAlchemy 2.0+ patterns including Mapped types, TypeDecorators, and modern repository patterns.

How does it handle API schema changes?

The skill provides patterns for schema resilience using PostgreSQL JSONB columns to capture evolving API fields without breaking existing database migrations.

Can I use this for dimensional modeling?

Absolutely. It includes best practices for Kimball star schemas, including fact/dimension naming conventions and bridge table patterns.

Python Data Engineering

Name: Python Data Engineering
Author: AeyeOps

byAeyeOps

0•

Database Management

Builds production-grade data pipelines and dimensional models using SQLAlchemy 2.0 and modern warehouse architectures.

This skill provides comprehensive patterns for architecting robust Python-based data pipelines, focusing on SQLAlchemy 2.0 implementations and modern data warehousing principles like Kimball star schemas and medallion layers. It guides developers through implementing complex ETL/ELT workflows, managing schema evolution via JSONB, and ensuring data integrity with Pydantic validation. By establishing a clear three-layer separation between source systems, database schemas, and application logic, it enables the creation of maintainable, scalable, and high-performance data infrastructure on PostgreSQL.

Key Features

01SQLAlchemy 2.0 advanced patterns including TypeDecorators and factory methods

02Schema resilience techniques using PostgreSQL JSONB for evolving SaaS API data

03Implementation of medallion architecture layers (bronze, silver, and gold)

040 GitHub stars

05Dimensional modeling for Kimball star schemas using dim_ and fact_ conventions

06Incremental sync strategies using high-water mark patterns for optimized performance

Use Cases

01Building custom Python ETL pipelines for sources lacking standard connectors

02Converting raw SaaS API data into structured PostgreSQL dimensional models

03Implementing historical data tracking using Slowly Changing Dimensions (SCD Type 2)

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add aeyeops/aeo-skill-marketplace python-data-engineering

For use in Claude.ai and ChatGPT

Download Skill