Data Engineer Agent Skill

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms. Use PROACTIVELY for data pipeline design, analytics infrastructure, or modern data stack implementation.

3k tokens

context cost

the whole folder, loaded on every use

files

instructions only

copies elsewhere

how many repositories repackaged it

505

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/Dokhacgiakhoa/Agent-Skills-4-Vibe-Coding-CLI --skill data-engineer

What comes with it

9 871 bytes besides the instruction

sub-skills/batch-processing-etlelt.md

sub-skills/cloud-data-platforms-services.md

sub-skills/data-modeling-warehousing.md

sub-skills/data-quality-governance.md

sub-skills/data-security-compliance.md

sub-skills/database-technologies-integration.md

sub-skills/infrastructure-devops-for-data.md

sub-skills/integration-api-development.md

sub-skills/modern-data-stack-architecture.md

sub-skills/performance-optimization-scaling.md

sub-skills/real-time-streaming-event-processing.md

sub-skills/workflow-orchestration-pipeline-management.md

The instruction itself

19 sections, as written by the author

You are a data engineer specializing in scalable data pipelines, modern data architecture, and analytics infrastructure.

Use this skill when

Designing batch or streaming data pipelines
Building data warehouses or lakehouse architectures
Implementing data quality, lineage, or governance

Do not use this skill when

You only need exploratory data analysis
You are doing ML model development without pipelines
You cannot access data sources or storage systems

Instructions

Define sources, SLAs, and data contracts.
Choose architecture, storage, and orchestration tools.
Implement ingestion, transformation, and validation.
Monitor quality, costs, and operational reliability.

Safety

Protect PII and enforce least-privilege access.
Validate data before writing to production sinks.

Purpose

Expert data engineer specializing in building robust, scalable data pipelines and modern data platforms. Masters the complete modern data stack including batch and streaming processing, data warehousing, lakehouse architectures, and cloud-native data services. Focuses on reliable, performant, and cost-effective data solutions.

Capabilities

🧠 Knowledge Modules (Fractal Skills)

1. Modern Data Stack & Architecture

2. Batch Processing & ETL/ELT

3. Real-Time Streaming & Event Processing

4. Workflow Orchestration & Pipeline Management

5. Data Modeling & Warehousing

6. Cloud Data Platforms & Services

7. Data Quality & Governance

8. Performance Optimization & Scaling

9. Database Technologies & Integration

10. Infrastructure & DevOps for Data

11. Data Security & Compliance

12. Integration & API Development

Other skills for the same job

different authors, same section of the catalogue

Hedgefundmonitor

by christophacham

×2

Query the OFR (Office of Financial Research) Hedge Fund Monitor API for hedge fund data including SEC Form PF aggregated statistics, CFTC Traders in Financial Futures, FICC Sponsored Repo volumes, and FRB SCOOS dealer financing terms. Access time series data on hedge fund size, leverage, counterparties, liquidity, complexity, and risk management. No API key or registration required. Use when working with hedge fund data, systemic risk monitoring, financial stability research, hedge fund leverage or leverage ratios, counterparty concentration, Form PF statistics, repo market data, or OFR financial research data.

10k tokens

ETL Pipeline

by claude-office-skills

×1

Design and automate Extract, Transform, Load data pipelines for data integration and analytics

3k tokens

Us Gov Shutdown Tracker

by fleurytian

×1

Track and analyze US government shutdown liquidity impacts by monitoring TGA (Treasury General Account), bank reserves, EFFR, and SOFR data from FRED API. Use when user wants to (1) analyze current or past government shutdown effects on financial markets, (2) track liquidity conditions during fiscal policy disruptions, (3) assess "stealth tightening" effects, (4) compare shutdown episodes across different monetary policy regimes (QE vs QT), or (5) generate liquidity stress reports with historical context. Recommended usage frequency is weekly on Wednesdays after TGA/reserve data releases.

8k tokens scripts

Azure Monitor Opentelemetry Ts

by lingxling

×1

Auto-instrument Node.js applications with distributed tracing, metrics, and logs.

2k tokens

Azure Monitor Query Java

by lingxling

×1

Azure Monitor Query SDK for Java. Execute Kusto queries against Log Analytics workspaces and query metrics from Azure resources.

3k tokens

Azure Monitor Query Py

by lingxling

×1

Azure Monitor Query SDK for Python. Use for querying Log Analytics workspaces and Azure Monitor metrics.

Use this skill when you need to search Datadog logs, query metrics, tail logs in real-time, trace distributed requests, investigate errors, compare time periods, find log patterns, check service health, or export observability data.

Write comprehensive clinical reports including case reports (CARE guidelines), diagnostic reports (radiology/pathology/lab), clinical trial reports (ICH-E3, SAE, CSR), and patient documentation (SOAP, H&P, discharge summaries). Full support with templates, regulatory compliance (HIPAA, FDA, ICH-GCP), and validation tools.

87k tokens scripts

How to use it

Copy the folder

Take dokhacgiakhoa/data-engineer from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.