Enable Ssi

Enable APM on Kubernetes via Single Step Instrumentation

> Before doing anything else: Fully resolve all variables in ## Context to resolve before acting. Do not begin Step 0 until every variable has a concrete value.

> Silent failure — check this before any other step:

>

> If the application has ddtrace, dd-trace, or any OpenTelemetry SDK in its dependency manifest (requirements.txt, package.json, Gemfile, go.mod, pom.xml) — even with no import statements in code — SSI will silently disable itself at runtime.

>

> The failure is invisible: init containers run and complete, the pod starts healthy, no errors appear in kubectl or pup, but no traces arrive. The injector detects the user-installed tracer and exits cleanly without logging anything.

>

> ### Claude runs

>

> `bash

> grep -rE "ddtrace|dd-trace|opentelemetry" \

> requirements.txt package.json Gemfile go.mod pom.xml 2>/dev/null \

> || echo "No tracer dependency found"

> `

>

> If any match — stop. Remove the package entirely (not just the import), rebuild the image, reload it into the cluster, and restart the pod before continuing. A package present in the manifest is enough to trigger this even if it is never imported.

Triggers

Invoke this skill when the user expresses intent to:

• Enable APM on a Kubernetes cluster
• Instrument Kubernetes applications with Datadog tracing
• Set up Single Step Instrumentation (SSI)

Do NOT invoke this skill if:

• The Datadog Agent is not yet installed — run agent-install first
• The user wants to verify SSI after setup — use verify-ssi
• The user wants to enable Profiler, AppSec, or Data Streams — use dd-apm-k8s-sdk-features

Prerequisites

> These are not a reading exercise — actively verify each one before proceeding.

Environment

• [ ] Datadog Agent is installed and healthy — agent-install complete
• [ ] Kubernetes v1.20+
• [ ] Linux node pools only — Windows pods require explicit namespace exclusion
• [ ] Cluster is not ECS Fargate — unsupported
• [ ] Not a hardened SELinux environment — unsupported
• [ ] Not a very small VM instance (e.g. t2.micro) — SSI can hit init timeouts
• [ ] No PodSecurity baseline or restricted policy enforced

Language and runtime

• [ ] Application language is one of: Java, Python, Ruby, Node.js, .NET, PHP
• ] Runtime version is within SSI's supported range — verify against the [SSI compatibility matrix
• [ ] Node.js app is not using ESM — SSI does not support ESM
• [ ] Java app is not already using a -javaagent JVM flag

Existing instrumentation — confirmed clean by the check at the top of this skill. If you skipped that check, go back and run it now.

Context to resolve before acting

> Discover from the cluster — do not ask the user for information you can find yourself.

| Variable | How to resolve |

|---|---|

| AGENT_NAMESPACE | Same namespace used in agent-install (e.g. datadog) |

| APP_NAMESPACE | Run kubectl get namespaces --no-headers \| awk '{print $1}' \| grep -vE '^(kube-system\|kube-public\|kube-node-lease\|datadog\|local-path-storage)$' — instrument all non-system namespaces, or use the namespace(s) the user mentioned |

| TARGET_LANGUAGES | Run kubectl get pods -A -o jsonpath='{.items[*].spec.containers[*].image}' and infer language from image names, or check Dockerfiles/manifests in the workspace. If uncertain, enable all languages. |

| DEPLOYMENT_NAME | Run kubectl get deployments -A --no-headers — identify application deployments (exclude system components) |

| APP_LABEL | Check spec.selector.matchLabels in the Deployment manifest via kubectl get deployment <DEPLOYMENT_NAME> -n <APP_NAMESPACE> -o yaml |

| CLUSTER_NAME | Check spec.global.clusterName in datadog-agent.yaml, or kubectl config current-context — needed for kind clusters in Step 0 |

| ENV | Use apm-evals if running in an eval cluster (kind cluster names contain "evalya"). Otherwise use production unless the user specifies otherwise. |

| SERVICE_NAME | Use the deployment name (e.g. python-app → service python-app). Do not ask the user. |

| VERSION | Use 1.0.0 as the default. Do not ask the user. |

Step 0 (Only if existing instrumentation detected): Remove Manual Instrumentation

Scan all source files for: import ddtrace, from ddtrace, require 'ddtrace', require("dd-trace"), opentelemetry, tracer.trace(

Also check dependency manifests for ddtrace / dd-trace / OTel SDK packages.

If found — remove the import/package, then rebuild and reload:

Claude runs

docker build -f <DOCKERFILE_PATH> -t <IMAGE_NAME> <BUILD_CONTEXT>

[DECISION: how does this cluster get local images?]

Check the repo's setup script (e.g. create.sh, Makefile, justfile) for how images are loaded — do not guess from the cluster name or context. Common patterns:

| What you find in the setup script | Load command |

|---|---|

| minikube image load or minikube cache add | minikube -p <PROFILE> image load <IMAGE_NAME> — profile is the -p flag value in the script, NOT necessarily the kubectl context name |

| kind load docker-image | kind load docker-image <IMAGE_NAME> --name <CLUSTER_NAME> |

| docker push to a registry | Push the new image; the cluster will pull on restart — skip local load |

| k3d image import | k3d image import <IMAGE_NAME> -c <CLUSTER_NAME> |

| No image load step (cloud cluster, always pulls from registry) | Skip — image will be pulled on next deployment |

If the setup script is ambiguous, run the load command it uses exactly as written.

• Registry-based: skip — image will be pulled on next deployment

> Confirm with the user before restarting. Tell the user: "I need to restart <DEPLOYMENT_NAME> in <APP_NAMESPACE> to pick up the rebuilt image. Ready to proceed?" Wait for confirmation.

Claude runs

kubectl rollout restart deployment/<DEPLOYMENT_NAME> -n <APP_NAMESPACE>
kubectl wait --for=condition=Ready pod \
  -l app=<APP_LABEL> \
  -n <APP_NAMESPACE> \
  --timeout=120s

Step 1: Extend the DatadogAgent Manifest with APM

SSI is configured on the existing DatadogAgent resource — do not create a separate manifest.

Choose targeting scope based on what the user asked for:

• User asked to instrument all applications or didn't specify scope → use Option A (cluster-wide)
• User asked for specific namespaces only → use Option B
• User asked to exclude namespaces from cluster-wide → use Option C
• User asked for specific pods/workloads → use Option D

> Default is cluster-wide (Option A). If the user said "all my applications", "my whole cluster", or didn't restrict scope, use Option A with no enabledNamespaces or targets.

Recommended ddTraceVersions: java: "1", python: "2", js: "5", dotnet: "3", ruby: "2", php: "1"

Option A — Cluster-wide (default):

features:
  apm:
    instrumentation:
      enabled: true

Option B — Specific namespaces only:

features:
  apm:
    instrumentation:
      enabled: true
      enabledNamespaces:
        - <APP_NAMESPACE>

Option C — Cluster-wide with exclusions:

features:
  apm:
    instrumentation:
      enabled: true
      disabledNamespaces:
        - jenkins
        - kube-system

Option D — Target specific workloads:

features:
  apm:
    instrumentation:
      enabled: true
      targets:
        - name: <TARGET_NAME>
          namespaceSelector:
            matchNames:
              - <APP_NAMESPACE>
          ddTraceVersions:
            <LANGUAGE>: "<MAJOR_VERSION>"

> Note: ddTraceVersions only applies inside a targets[] entry (Option D). It is not valid alongside enabledNamespaces or at the instrumentation level directly.

Claude runs

kubectl apply -f datadog-agent.yaml

If datadogagent.datadoghq.com/datadog configured — continue to Step 2.

ERROR: Validation error — check YAML. enabledNamespaces and disabledNamespaces cannot both be set.

Step 2: Inform the User About Unified Service Tags

> Do NOT modify application Deployments without explicit user confirmation. Applying labels to existing application workloads is a change to customer-managed resources.

Inform the user that adding Unified Service Tags (UST) to their Deployments will enable proper service/env/version tagging in Datadog. This is optional for SSI to work but recommended for full observability:

# Add to both metadata.labels and spec.template.metadata.labels
tags.datadoghq.com/env: "<ENV>"
tags.datadoghq.com/service: "<SERVICE_NAME>"
tags.datadoghq.com/version: "<VERSION>"

If the user wants you to apply these, get their confirmation first. UST labels are not required for APM traces to flow — SSI works without them.

Step 3: Restart Application Pods

> Confirm with the user before restarting. Tell the user: "I need to restart <DEPLOYMENT_NAME> in <APP_NAMESPACE> for SSI to inject into the pods. This will cause a brief outage. Ready to proceed?" Wait for confirmation.

Claude runs

kubectl rollout restart deployment/<DEPLOYMENT_NAME> -n <APP_NAMESPACE>

kubectl wait --for=condition=Ready pod \
  -l app=<APP_LABEL> \
  -n <APP_NAMESPACE> \
  --timeout=120s

If pods restart cleanly, init containers named datadog-lib-<language>-init will be visible in the pod spec.

ERROR: Pods crash-looping — check for existing custom instrumentation. See troubleshoot-ssi.

Done

Exit when ALL of the following are true:

• [ ] features.apm.instrumentation is present in the applied DatadogAgent manifest
• [ ] User has been informed that they need to restart their application pods
• [ ] User has been informed about Unified Service Tags (UST) and how to apply them if desired
• [ ] Scope confirmed: which workloads are instrumented, which were skipped and why

Automatically proceed to verify-ssi now — do not ask the user for permission.

Security constraints

• Never write a raw API key into any file or chat message
• Never use namespace default for Datadog resources
• Never modify admissionController settings directly — SSI manages this via the Operator
• Do not add APM config to application manifests — configure only via DatadogAgent
• Exception: UST labels (tags.datadoghq.com/*) on application Deployments are required and intentional
• Never run kubectl delete without user confirmation
• docker push to a registry always requires user confirmation
• Never use kubectl patch to apply UST labels or any Deployment changes. Always edit the Deployment YAML file and kubectl apply -f. Changes made with kubectl patch are transient and will be overwritten on the next rollout.