Architecture Paradigm Pipeline Skill for Claude
Applies pipes-and-filters for sequential data transformations. Use when data flows through discrete stages like ETL, streaming analytics, or CI/CD pipelines.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/athola/claude-night-market --skill architecture-paradigm-pipelineThe instruction itself
8 sections, as written by the authorThe Pipeline (Pipes and Filters) Paradigm
When to Employ This Paradigm
- When data must flow through a fixed sequence of discrete transformations, such as in ETL jobs, streaming analytics, or CI/CD pipelines.
- When reusing individual processing stages is needed, either independently or to scale bottleneck stages separately from others.
- When failure isolation between stages is a critical requirement.
When NOT To Use
- Interactive request/response systems (use
archetypes:architecture-paradigm-client-server)
- Stages that must share mutable state, which the pattern cannot express
Adoption Steps
- Define Filters: Design each stage (filter) to perform a single, well-defined transformation. Each filter must have a clear input and output data schema.
- Connect via Pipes: Connect the filters using "pipes," which can be implemented as streams, message queues, or in-memory channels. validate these pipes support back-pressure and buffering.
- Maintain Stateless Filters: Where possible, design filters to be stateless. Any required state should be persisted externally or managed at the boundaries of the pipeline.
- Instrument Each Stage: Implement monitoring for each filter to track key metrics such as latency, throughput, and error rates.
- Orchestrate Deployments: Design the deployment strategy to allow each stage to be scaled horizontally and upgraded independently.
Key Deliverables
- An Architecture Decision Record (ADR) documenting the filters, the chosen pipe technology, the error-handling strategy, and the tools for replaying data.
- A suite of contract tests for each filter, plus integration tests that cover representative end-to-end pipeline executions.
- Observability dashboards that visualize stage-level Key Performance Indicators (KPIs).
Risks & Mitigations
- Single-Stage Bottlenecks:
- Mitigation: Implement auto-scaling for individual filters. If a single filter remains a bottleneck, consider refactoring it into a more granular sub-pipeline.
- Schema Drift Between Stages:
- Mitigation: Centralize schema definitions in a shared repository and enforce compatibility tests as part of the CI/CD process to prevent breaking changes.
- Back-Pressure Failures:
- Mitigation: Conduct rigorous load testing to simulate high-volume scenarios. Validate that buffering, retry logic, and back-pressure mechanisms behave as expected under stress.
Concrete Components
These vocabulary items name the concrete tools and abstractions
that show up when the paradigm is implemented. They are not
required dependencies and they are not part of the skill's
tools: frontmatter (which is reserved for Claude Code tool
restrictions). Use this list to disambiguate during architecture
discussions.
stream-processor: the runtime that executes a filter (e.g. Flink, Apache Beam, Faust)message-queue: the durable pipe between filters (e.g. Kafka, RabbitMQ, in-memory channel)data-validator: schema-checks every record at filter input and output
Exit Criteria
- [ ] An ADR documents every filter in the pipeline, the chosen pipe technology, the
error-handling strategy (DLQ, retry count, dead-letter routing), and the data replay
mechanism.
- [ ] Each filter has a contract test covering its input and output schema; schema drift between
adjacent filters is caught by a CI compatibility check.
- [ ] Observability dashboards are configured showing per-stage latency, throughput, and error
rate before the pipeline is promoted to production.
- [ ] Load testing validates that back-pressure and buffering mechanisms prevent data loss at
2x the expected peak throughput.
Other skills for the same job
different authors, same section of the catalogue
Assess Kubernetes workloads and cluster configuration for AKS Automatic compatibility. Identifies incompatibilities, generates fixes, and guides migration from AKS Standard to AKS Automatic. WHEN: migrate to AKS Automatic, check AKS Automatic readiness, validate manifests for Automatic, assess cluster for Automatic compatibility, fix deployment for Automatic compatibility, identify AKS Automatic migration blockers, is my cluster ready for AKS Automatic.
Discovers available Azure OpenAI model capacity across regions and projects. Analyzes quota limits, compares availability, and recommends optimal deployment locations based on capacity requirements. USE FOR: find capacity, check quota, where can I deploy, capacity discovery, best region for capacity, multi-project capacity search, quota analysis, model availability, region comparison, check TPM availability. DO NOT USE FOR: actual deployment (hand off to preset or customize after discovery), quota increase requests (direct user to Azure Portal), listing existing deployments.
Interactive guided deployment flow for Azure OpenAI models with full customization control. Step-by-step selection of model version, SKU (GlobalStandard/Standard/ProvisionedManaged), capacity, RAI policy (content filter), and advanced options (dynamic quota, priority processing, spillover). USE FOR: custom deployment, customize model deployment, choose version, select SKU, set capacity, configure content filter, RAI policy, deployment options, detailed deployment, advanced deployment, PTU deployment, provisioned throughput. DO NOT USE FOR: quick deployment to optimal region (use preset).
Unified Azure OpenAI model deployment skill with intelligent intent-based routing. Handles quick preset deployments, fully customized deployments (version/SKU/capacity/RAI policy), and capacity discovery across regions and projects. USE FOR: deploy model, deploy gpt, create deployment, model deployment, deploy openai model, set up model, provision model, find capacity, check model availability, where can I deploy, best region for model, capacity analysis. DO NOT USE FOR: listing existing deployments (use foundry_models_deployments_list MCP tool), deleting deployments, agent creation (use agent/create), project creation (use project/create).
Intelligently deploys Azure OpenAI models to optimal regions by analyzing capacity across all available regions. Automatically checks current region first and shows alternatives if needed. USE FOR: quick deployment, optimal region, best region, automatic region selection, fast setup, multi-region capacity check, high availability deployment, deploy to best location. DO NOT USE FOR: custom SKU selection (use customize), specific version selection (use customize), custom capacity configuration (use customize), PTU deployments (use customize).
This skill should be used when working with LaminDB, an open-source data framework for biology that makes data queryable, traceable, reproducible, and FAIR. Use when managing biological datasets (scRNA-seq, spatial, flow cytometry, etc.), tracking computational workflows, curating and validating data with biological ontologies, building data lakehouses, or ensuring data lineage and reproducibility in biological research. Covers data management, annotation, ontologies (genes, cell types, diseases, tissues), schema validation, integrations with workflow managers (Nextflow, Snakemake) and MLOps platforms (W&B, MLflow), and deployment strategies.
Latch platform for bioinformatics workflows. Build pipelines with Latch SDK, @workflow/@task decorators, deploy serverless workflows, LatchFile/LatchDir, Nextflow/Snakemake integration.
Run Python code in the cloud with serverless containers, GPUs, and autoscaling. Use when deploying ML models, running batch processing jobs, scheduling compute-intensive tasks, or serving APIs that require GPU acceleration or dynamic scaling.
How to use it
Take athola/architecture-paradigm-pipeline from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.