Replicate Agent Skill

Replicate platform operations

This skill is about how a model runs in production on Replicate — clients, async, deployments,

Cog packaging, webhooks, scaling, and spend. It is the platform-engineering counterpart to image

prompt craft. If the question is *what prompt, aspect ratio, or model family produces a good image*,

that is replicate-images, not this skill. Here the mental model is: **a prediction is a job. You

either wait for it, poll it, or get pinged about it — and where it runs (shared cold pool vs a private

warm deployment) is a cost-and-latency dial you set deliberately.**

Pinned facts (verified 2026-06-02): Python client replicate 1.x (latest 1.0.7), Python 3.8+;

JS client replicate on npm; auth via REPLICATE_API_TOKEN. A 2.0.0aN alpha exists on PyPI but

is NOT the default — pin replicate>=1,<2 so a fresh install never silently pulls it.

Decision: how should this model run?

Pick the row by latency tolerance and whether your process can block. Do not default to run() for

everything — a 10-minute job inside a web request will time out and burn a worker.

| Pattern | Latency | Blocks your process? | Cost shape | Use when |

|---|---|---|---|---|

| replicate.run(...) | seconds | Yes — waits to completion | per-prediction, shared pool | interactive/quick calls, scripts, CLIs |

| predictions.create() + poll | minutes | Yes, but you control the loop | per-prediction, shared pool | long job, a worker can babysit it |

| predictions.create(webhook=...) | minutes+ | No — fire and forget | per-prediction, shared pool | long job, the request must return now |

| Deployment (private endpoint) | low + steady | depends on call style above | warm floor + per-prediction | sustained traffic, need warm/private/autoscale cap |

Rule: if a human or HTTP request is waiting longer than a few seconds, do not block on run() —

switch to predictions + webhook. Why: synchronous timeouts kill the request but the GPU job keeps

running and billing.

Auth & install

export REPLICATE_API_TOKEN=r8_...           # clients read this env var automatically
pip install 'replicate>=1,<2'               # pin: 2.0.0aN is alpha; unpinned can pull it
npm install replicate                        # Node client

Python 3.8+ is required. Never pip install replicate unpinned in a Dockerfile or requirements

file — a rebuild months later can resolve to the 2.0 alpha and break your imports. Why: the alpha is

a full Stainless/httpx rewrite with a different surface.

Run a model

import replicate

output = replicate.run(
    "black-forest-labs/flux-schnell",
    input={"prompt": "a red bicycle", "num_outputs": 1},
)

Since client 1.0.0, file outputs come back as FileOutput objects, not URL strings. Treating one

as a string is the single most common bug.

# Bad — output[0] is a FileOutput, not a str; this writes the repr, not the bytes
open("out.png", "w").write(output[0])

# Good — read the bytes, or take the URL explicitly
with open("out.png", "wb") as f:
    f.write(output[0].read())          # bytes
print(output[0].url)                    # hosted URL if you'd rather link

Rule: call .read() for bytes or .url for the link. Why: silently coercing a FileOutput to a

string corrupts the file and the error surfaces far from the cause.

Long-running & async

For jobs over a few seconds, create a prediction instead of blocking:

client = replicate.Client()
prediction = client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
)
prediction.reload()                      # refresh status from the API
while prediction.status not in ("succeeded", "failed", "canceled"):
    time.sleep(2)
    prediction.reload()

Set a deadline so a stuck job auto-cancels instead of billing forever, and cancel() on cleanup

paths. Why: a hung prediction with no deadline is silent, open-ended GPU spend. Poll with a small

backoff, not a tight loop — you are charged for the prediction, not the polling, but a tight loop

wastes your own process and rate budget. Full polling loop with backoff and 5xx handling is in

references/webhooks-and-async.md.

Webhooks

For fire-and-forget, hand Replicate a URL and filter to the events you care about:

client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
    webhook="https://your.app/hooks/replicate",
    webhook_events_filter=["completed"],   # not every intermediate "logs" event
)

Rule: ALWAYS verify the signature before trusting a webhook body. Why: the URL is public — anyone can

POST forged completions to it. Replicate signs each delivery; the secret has a whsec_ prefix.

Reconstruct signed content as {webhook-id}.{webhook-timestamp}.{body}, HMAC-SHA256 with the

base64-decoded secret, base64-encode, and constant-time compare against the webhook-signature

header (a space-separated v1,<sig> list). The clients expose a verification helper; the full

Python and Node recipe (including idempotency via webhook-id) is in

references/webhooks-and-async.md.

Deployments

A deployment is a private, dedicated API endpoint for one model version that autoscales from zero

to hundreds of instances. Reach for it when you need warm instances, a private endpoint, or a hard

spend cap — not for one-off runs.

• min_instances — the warm floor. Set >0 to kill cold starts for latency-sensitive traffic;

every warm instance bills whether or not it serves a request.

• max_instances — the spend cap. The ceiling on concurrent instances; protects you from a

traffic spike turning into a surprise bill.

• Hardware (NVIDIA T4, A100, H100, ...) is switchable without touching code — change the

deployment config, not predict.py.

• Rolling updates ship a new version with no downtime; built-in monitoring covers latency,

throughput, error rate, and GPU memory.

You still call a deployment with run() / predictions.create() — it just routes to your private

instances. Create/update via HTTP API, the clients, or CLI; fields and the rolling-update flow are in

references/deployments-api.md.

Package a custom model with Cog

Cog packages a model into a production container. You need two files; Replicate builds the API server

for you on push.

# cog.yaml
build:
  gpu: true
  python_version: "3.11"
  python_packages:
    - "torch==2.4.0"
predict: "predict.py:Predictor"

# predict.py
from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self):
        # load weights ONCE here, not per request
        self.model = load_model("weights.pth")

    def predict(self, prompt: str = Input(description="text prompt")) -> Path:
        result = self.model(prompt)
        return Path(result)            # Cog uploads the file

cog predict -i prompt="hello"          # run locally (needs Docker)
cog push r8.im/owner/model             # build + push; Replicate hosts it

Rule: load weights in setup(), never in predict(). Why: setup() runs once per instance;

predict() runs every request — loading weights per request makes every call pay the model-load cost.

Full cog.yaml (system packages, run steps), typed Input(...), GPU config, version pinning, and

common build failures are in references/cog-packaging.md. Building requires Docker.

Cost & scaling levers

• Scale to zero is the default — idle deployments stop billing. Only raise min_instances above

zero when cold-start latency actually hurts users, and treat the warm floor as a line item.

• Deadlines on predictions cap the worst case so a stuck job cannot bill open-ended.
• Right-size the GPU: do not run a 1B model on an H100. Hardware is a deployment-config change.
• Cache weights in setup() so per-request work is just inference.

This skill covers Replicate-specific levers only. Tracking total AI spend across many providers as a

discipline is cost-tracking.

Anti-patterns

| Anti-pattern | Why it bites | Do instead |

|---|---|---|

| Treating a FileOutput as a URL string | Corrupts files / writes a repr; error surfaces far away | .read() for bytes, .url for the link |

| Blocking run() for a 10-min job in a web request | Request times out; the GPU job keeps running and billing | predictions.create(webhook=...), return now |

| Webhook handler with no signature check | The URL is public; anyone can forge completions | Verify HMAC-SHA256 against whsec_ secret |

| High min_instances "just in case" | Every warm instance bills 24/7 idle | Scale to zero; raise floor only when cold starts hurt |

| Loading weights inside predict() | Every request pays the model-load cost | Load once in setup() |

| pip install replicate unpinned | A rebuild can pull the 2.0 alpha and break imports | Pin replicate>=1,<2 |

| A deployment for a one-off run | Pays for a private endpoint you call once | Use the shared pool via run() |

| No deadline on a long prediction | A hung job bills open-ended, silently | Set a deadline; cancel() on cleanup |

References

• references/cog-packaging.md — full cog.yaml + predict.py, GPU config, build/push, version pinning, build failures.
• references/webhooks-and-async.md — signature verification (Python + Node), event filters, idempotency, polling with backoff, 5xx retries.
• references/deployments-api.md — create/update/get deployment via HTTP + clients, autoscaling fields, rolling updates, monitoring metrics, CLI.

scripts/verify.sh statically checks an emitted artifact dir: a cog.yaml declaring build: and

predict:, a predict.py Predictor with setup + predict, and warns on an unpinned replicate

dependency or a FileOutput written as a string. Presence + key checks only — it does not run Docker.