Software Development·v1.0.0·JayRHa/AgentSkills

Data Pipeline Architect

Designs robust ETL/ELT data pipelines covering ingestion, idempotency, schema evolution, orchestration, and data quality validation. Use this skill when the user asks to design, build, or review a data pipeline, ingest data from APIs/databases/files into a warehouse or lake, set up batch or streaming ETL/ELT, choose an orchestrator (Airflow, Dagster, Prefect, dbt), make a pipeline idempotent or backfill-safe, handle late-arriving or duplicate data, manage schema drift/evolution, add data quality or freshness checks, or model incremental/CDC loads.