pytorch Skills

Debug AOTInductor (AOTI) errors and crashes. Use when encountering AOTI segfaults, device mismatch errors, constant loading failures, or runtime errors from aot_compile, aot_load, aoti_compile_and_package, or aoti_load_package.

Query PyTorch CI, GitHub Actions, HUD, Grafana, and infrastructure metrics. Use when users ask about CI duration, job failures, queue times, workflow trends, runner health, dashboard data, or PyTorch infrastructure metrics.

Choose 32-bit vs 64-bit index math in PyTorch CUDA kernels. Use when fixing large-tensor indexing overflows, deciding whether to use int64_t, canUse32BitIndexMath, CUDA_KERNEL_LOOP_TYPE, or AT_DISPATCH_INDEX_TYPES, and when considering binary-size or performance impact of index-type templating.

Sub-triages issues in the oncall:distributed queue by assigning distributed module labels, routing to sub-oncalls, and marking triaged. Use when an issue has been routed to oncall:distributed and needs second-level triage.

Document undocumented public APIs in PyTorch by removing functions from coverage_ignore_functions and coverage_ignore_classes in docs/source/conf.py, running Sphinx coverage, and adding the appropriate autodoc directives to the correct .md or .rst doc files. Use when a user asks to remove functions from conf.py ignore lists.

Fix bugs reported in PyTorch GitHub issues by reproducing, root-causing, and implementing a fix in the local working tree. Use when the user asks to fix a PyTorch GitHub issue.

Write Metal/MPS kernels for PyTorch operators. Use when adding MPS device support to operators, implementing Metal shaders, or porting CUDA kernels to Apple Silicon. Covers native_functions.yaml dispatch, host-side operators, and Metal kernel implementation.

Debug PyTorch 2 compiler stack failures including Dynamo graph breaks, Inductor codegen errors, AOTAutograd crashes, and accuracy mismatches. Use when encountering torch.compile errors, BackendCompilerFailed exceptions, recompilation issues, Triton kernel failures, FX graph problems, or when the user mentions debugging PT2, Dynamo, Inductor, or compiled model issues.

Migrate a file to use stricter Pyrefly type checking with annotations required for all functions, classes, and attributes.

Fetch, analyze, reproduce, and minimize GitHub issue reproductions. Use only when the user asks to check whether a GitHub issue reproduces, minimize a GitHub issue's repro, analyze a GitHub bug report, or scrub/triage a GitHub issue for reproducibility. Do not use for standalone reproduction requests without a GitHub issue.

Triages GitHub issues by routing to oncall teams, applying labels, and closing questions. Use when processing new PyTorch issues or when asked to triage an issue.

Review PyTorch tutorials pull requests for content quality, code correctness, build compatibility, and style. Use when reviewing PRs, when asked to review code changes, or when the user mentions "review PR", "code review", or "check this PR".

Add a new HF MoE model to the transformers_modeling_backend experiment. Use when the user provides a HuggingFace model ID and wants to integrate it with Titan's MoE, or invokes /add_moe_model.

Profiler-driven hill-climbing to close the inference throughput gap between TorchTitan's unified model (running inside vLLM) and vLLM's native model. Benchmark with generate.py --benchmark, climb optimization rungs (compile / cudagraph / fused kernels), profile torchtitan vs the native target, then patch the single biggest gap at a time and re-measure. Use when the user wants to benchmark or optimize RL inference generation speed, reproduce previous hill climbing study, or invokes /inference_perf_hillclimb.

Capture and compare per-op activations between two TorchTitan runs to spot numerics divergence (eager vs aot_fx_trace, FSDP vs no-FSDP, before vs after a refactor). Use when the user wants to debug bitwise / numeric drift in training, or invokes /numerics_debugging.

Bisect PyTorch commits to find the regression that breaks TorchTitan. Use when the user wants to bisect PyTorch or invokes /torch_bisect.

Analyze and reduce ExecuTorch binary size. Use when investigating binary size, running size tests, or optimizing the runtime for size-constrained deployments.

Build ExecuTorch from source — Python package, C++ runtime, runners, cross-compilation, and backend-specific builds. Use when compiling anything in the ExecuTorch repo, diagnosing build failures, or setting up platform-specific builds.

Build, test, or develop the Cortex-M (CMSIS-NN) backend. Use when working on backends/cortex_m/, running Cortex-M tests, or exporting models for Cortex-M targets.

Search the ExecuTorch tribal knowledge base covering QNN, XNNPACK, Vulkan, CoreML, Arm, and Cadence backends, quantization recipes, export pitfalls, runtime errors, and SoC compatibility. Use when debugging ExecuTorch errors, choosing quantization configs, checking backend op support, or answering questions about Qualcomm HTP / Snapdragon / Apple Neural Engine behavior.

Export a PyTorch model to .pte format for ExecuTorch. Use when converting models, lowering to edge, or generating .pte files.

Profile ExecuTorch model execution. Use when measuring performance, analyzing operator timing, or debugging slow models.

Build, test, or develop the QNN (Qualcomm AI Engine Direct) backend. Use when working on backends/qualcomm/, building QNN (use backends/qualcomm/scripts/build.sh), adding new ops or passes, running QNN delegate tests, or exporting models for Qualcomm HTP/GPU targets. Also exposes a Buck-vs-CMake parity workflow — invoke as `/qualcomm buck-fix`, `/qualcomm buck-cmake fix`, `/qualcomm buck-parity`, or any user request to fix `test-qnn-buck-build-linux` CI failures or check buck/cmake drift in backends/qualcomm/. Also covers QNN intermediate-output / per-layer accuracy debugging — trigger on phrases like "QNN accuracy issue", "QNN output doesn't match CPU", "debug per-layer for QNN", "find which QNN layer is wrong".

Set up ExecuTorch development environment. Use when installing dependencies, setting up conda environments, or preparing to develop with ExecuTorch.

Build and configure ExecuTorch as a Zephyr RTOS module for embedded boards. Use when setting up a Zephyr workspace with ET, adding board support (overlays, confs, memory layout), building with west, or debugging linker memory overflow.

Analyze torch-tensorrt local test results and drive failures to a fix. Use when the user pastes a test report / summary, asks why tests failed, asks to triage or fix failing tests, or mentions the JUnit/test-summary output from `just tests-report` / `just test-summary`. Covers where the JUnit XMLs live, how to read the consolidated report, how to reproduce a single failure, and how to categorize (real bug vs torch-API change vs OOM/skip vs flake).

Build torch-tensorrt locally — install/pin the matching PyTorch nightly, drive Bazel through setup.py, do a clean rebuild after libtorch ABI changes, and recover from common build failures (undefined symbol, stale _C.so, libtorchtrt.so missing). Invoke whenever the user asks to build, rebuild, install editable, or upgrade the torch nightly; or when an import fails with an undefined-symbol error tying torch_tensorrt to libtorch.

Update the third_party/kineto submodule in PyTorch to the latest commit from this kineto repo and commit the change. Use when updating the kineto submodule hash for a release.

Investigate a Helion benchmark dashboard regression (helionlang.com/dashboard) — find the cause and classify it. Auto-activate when the user reports a perf drop/spike on a dashboard platform (e.g. b200 cute) around a given date.

Verify changes to the Helion CuTe backend by running both test suites, lint, and a code review subagent — then fix any issues found.

Address CI failures and unresolved review comments on a Helion pull request. Auto-activate when the user mentions a URL like https://github.com/pytorch/helion/pull/<number>.

Auto-activate when the user mentions runpod or scripts/runpod.py

Context for working on Helion's TPU/Pallas backend. Auto-activate when the user mentions TPU or Pallas in the context of Helion development.

Review a pytorch/pytorch pull request's changes and decide whether they are safe to land. Emits a single machine-readable verdict (LAND or NO_LAND) for the greenlight auto-land gate.

Step-by-step playbook for migrating a pytorch/pytorch .github/workflows/*.yml from EC2 to OSDC (ARC) runners — covers both dial-up and 100% opt-in patterns, with the inputs that must be plumbed through _linux-build.yml / _linux-test.yml. Use when migrating a workflow off EC2 onto on-site data center / EKS-hosted self-hosted runners (OSDC / ARC), enabling the ARC experiment, or wiring up `use-arc` / `runner_prefix` / `ci-docker-hash` / `python-version` / `compiler` / `cuda-version` inputs.

Disable or re-enable Cloudflare R2 (download-r2.pytorch.org) usage in manage_v2.py during R2 outages. Can toggle R2 off/on for nightly builds, prod/stable builds, or both. The affected packages are PT_FOUNDATION_PACKAGES (torch, torchvision, torchaudio, fbgemm_gpu, fbgemm_gpu_genai, triton, pytorch_triton, pytorch_triton_rocm, pytorch_triton_xpu).

Find reverts that landed on pytorch/pytorch main but are missing from a release branch (release/X.Y) because the reverted commit was already shipped in a release candidate, and open one cherry-pick PR per missing revert against the release branch (on a fork branch, never pushing to release/X.Y directly), optionally posting a cherry-pick nomination comment for each on the release tracker issue. Driven by tools/analytics/github_analyze.py --analyze-missing-reverts-from-branch (the "GitHub Analytics Daily" workflow). Triggered by "missing reverts", "analyze reverts for release", "cherry-pick missing reverts", "reverts not in release/X.Y", or pointing at a GitHub Analytics Daily run and asking to cherry-pick the flagged reverts.

Generate (and optionally open) a PyTorch release tracker / cherry-pick tracking issue from a release announcement, like https://github.com/pytorch/pytorch/issues/180506. Triggered by mentions of "release tracker", "release tracking issue", "cherry-pick tracker", or "cut a release branch issue".

Generate a PyTorch release validation checklist issue by pulling open/closed issues from a GitHub milestone and cherry-picks from a release tracker issue. Triggered by mentions of "validation issue", "validation checklist", "release validation", or "release checklist".

Update tools/scripts/generate_binary_build_matrix.py when a PyTorch release goes live. Advances CURRENT_STABLE_VERSION to the new stable, promotes the release-channel CUDA/ROCm arches to match the candidate/test channel, and regenerates the JSON test fixtures. Triggered by mentions of "release go live", "advance stable version", "promote release matrix", or "update binary build matrix" for a new PyTorch X.Y release.

Pin (or re-pin) the Linux manywheel builder docker images used by the nightly/release binary build workflows in pytorch/pytorch to a fixed .ci/docker build. Takes the release version (e.g. 2.13) as input. Updates DOCKER_IMAGE_PIN in .github/scripts/generate_binary_build_matrix.py and regenerates the workflows. Triggered by mentions of "pin docker image", "update docker pin", "docker image pin", "pin builder images", or "bump the manywheel image pin" for a release branch.

Triage a failing vLLM Buildkite CI build for a PyTorch version-bump PR, isolate new regressions vs. pre-existing failures on main by comparing against recent "Full CI run - nightly/daily" builds, classify by root cause, and file one grouped issue per root cause in pytorch/pytorch (linked under an umbrella issue). Use when the user points at a Buildkite build for a torch/triton upgrade and asks "what's broken that isn't broken on main" or "file issues for each failure".

> Includes command references, safety boundaries, and the deploy-audit ConfigMaps in osdc-system (what version is running where, who deployed it, when, did it succeed, is a deploy stuck). Applies to ~/meta/ci-infra/osdc. Load when investigating cluster state, debugging pods, inspecting infrastructure, comparing deployed versions across clusters, or diagnosing a failed or stuck deploy.

> OSDC deployment workflow, just recipes, base deploy order, module deploy order, clusters.yaml configuration, Terraform architecture, smoke tests, justfile conventions. Applies to ~/meta/ci-infra/osdc. Load when deploying, adding modules, or modifying deploy scripts.

> Harbor pull-through cache configuration, Helm chart gotchas (inconsistent value paths, per-component image overrides, taint tolerations), and image mirroring strategy. Applies to ~/meta/ci-infra/osdc. Load when working on Harbor, image mirroring, or container registry configuration.

> OSDC NodeLocal DNSCache (NLD) base component — per-node CoreDNS DaemonSet under base/kubernetes/nodelocaldns/. Covers iptables-mode rationale (vs IPVS-mode), dynamic kube-dns ClusterIP substitution via deploy.sh (validates the kube-dns ClusterIP is IPv6 and fails fast on IPv4 — OSDC EKS is IPv6-only by design and NLD is single-family by deploy-script enforcement; the IPv6-only cluster recreate has already shipped in commit a6b4c8c), Service-before-DaemonSet apply ordering (KUBE_DNS_UPSTREAM env-var injection), kube-dns-upstream auxiliary Service, two metrics ports (9253 CoreDNS plugin / 9353 binary setup_errors_total), the coredns_nodecache_* vs nodelocaldns_* metric-name confusion, no preStop / no memory limit / no startup taint design decisions, Harbor ≥3-day soak gate (applies to any FUTURE cluster-wide NLD cutover). Applies to ~/meta/ci-infra/osdc. Load when developing, debugging, or maintaining the nodelocaldns base component, investigating DNS issues on runner nodes, modifying its Corefile / DaemonSet manifest, or preparing a future cutover that touches NLD.

> Grafana Cloud Loki + Mimir queries, label strategy, credential setup, and troubleshooting. Applies to ~/meta/ci-infra/osdc. Load when working on monitoring, logging, Alloy, or querying logs.

> OSDC platform architecture, project organization, directory tree, key design decisions, external knowledge base, key files, and docs index. Applies to ~/meta/ci-infra/osdc. Load when working on OSDC to understand project layout and architecture.

> OSDC PyPI wheel cache module — per-CUDA-slug nginx+pypiserver fanout backed by shared EFS wheelhouse, fed by an external wheel-build pipeline via S3. Covers architecture (4 components), slug naming, EFS PVC, NVMe nginx cache, S3 bucket layout, needbuild override, prebuilt-cache.txt matrix invalidation, njs merge handler, /whl/ rewrites, NetworkPolicy, IRSA roles, log rotation, pod resource computation, cache-enforcer SNI matching. Applies to ~/meta/ci-infra/osdc. Load when developing, debugging, or maintaining the pypi-cache module, investigating pip install failures on runners, adding CUDA versions, or working with the wants/wheel-syncer pipeline.

> OSDC runners, NodePools, BuildKit, GitHub Actions constraints, EKS node taints, image mirroring, and the runner/nodepool change checklist. Applies to ~/meta/ci-infra/osdc. Load when modifying runners, nodepools, BuildKit, or node configurations.

> OSDC tools (tofu, just, mise, crane, uv), automation hierarchy, unit tests, code style and linting rules (13 linters, indentation), "Don't Do" list, and quality gates. Applies to the OSDC project (`osdc/`). Load when writing code, running linters, adding scripts/tests, or debugging and trying to understand issues.

> How PyTorch CI jobs in .github/workflows/ resolve to a specific runner, AND which vs current OSDC ARC-on-EKS) and the dot-vs-dash prefix split (linux./lf./c. = old EC2; mt-/lf-/c-mt- = OSDC ARC). Covers the runner determinator (fleet prefixes mt- default / lf- / c-mt- plus the separate amd-do-), the test-infra #5132 experiment config (lf, amd-do), the now-unconditional EC2->ARC label translation via .github/arc.yaml and map_ec2_to_arc.py (the meta_only_runners H100/B200 override and the onnx exclusion), build-vs-test runner decoupling, and the _runner-determinator.yml / _linux-build.yml / _linux-test.yml flow. Old-infra ownership spans test-infra/terraform-aws-github-runner, test-infra scale-config.yml / lf-scale-config.yml, and pytorch-gha-infra/runners; OSDC ownership lives in the osdc repo (clusters.yaml + modules/arc-runners*). Applies to ~/meta/pytorch. LOAD THIS SKILL for ANY question about PyTorch CI runners, runner labels, test matrices, runs-on values, runner prefixes (mt-, lf-, c-mt-, amd-do-, or legacy linux./lf./c.), WHICH infra a label runs on ("does lf.linux.2xlarge run on ARC or EC2?", "mt- vs lf- vs lf.?", "old ALI vs OSDC runners", "where is this runner defined?"), experiments (lf / amd-do), the determinator, ARC fleet routing, H100/B200/ A100 placement, or how a job gets assigned to a runner pool.

Add unsigned integer (uint) type support to PyTorch operators by updating AT_DISPATCH macros. Use when adding support for uint16, uint32, uint64 types to operators, kernels, or when user mentions enabling unsigned types, barebones unsigned types, or uint support.

Convert PyTorch AT_DISPATCH macros to AT_DISPATCH_V2 format in ATen C++ code. Use when porting AT_DISPATCH_ALL_TYPES_AND*, AT_DISPATCH_FLOATING_TYPES*, or other dispatch macros to the new v2 API. For ATen kernel files, CUDA kernels, and native operator implementations.

Write docstrings for PyTorch functions and methods following PyTorch conventions. Use when writing or updating docstrings in PyTorch code.

Guide users through creating Agent Skills for Claude Code. Use when the user wants to create, write, author, or design a new Skill, or needs help with SKILL.md files, frontmatter, or skill structure.