calesthio/video-generation-gateways
>- Select, integrate, and operate multi-model video-generation gateways — hosted inference aggregators such as fal.ai, Replicate, and WaveSpeed that expose many third-party video (and video-adjacent audio) models behind one account, one API surface, and one bill. Use when deciding whether to route video generation through a gateway versus a direct model-provider API; when comparing gateways on catalog, async/queue job design, webhooks, pricing, schema discovery, and version pinning; and when operating gateway workloads in production — spend controls, retries, cross-gateway failover, content- safety differences, data retention, and commercial-rights passthrough of the underlying model. Not for direct model-provider APIs, local/self-hosted inference, model training/fine-tuning, or image-only gateway use.
npx skills add https://github.com/calesthio/generative-media-skills --skill video-generation-gateways
A video-generation gateway is a hosted inference aggregator that resells
many third-party video models (Veo, Kling, Seedance, Hailuo/MiniMax, Luma,
Wan, LTX-Video, Pika, and others) behind a single account, a single API
convention, and a single bill. You send a prompt and parameters to a
gateway-owned endpoint; the gateway runs the underlying model on its own
infrastructure and returns a hosted result URL. This skill covers fal.ai,
Replicate, and WaveSpeed as the representative gateways, and the
production decisions common to the category.
This skill is about *routing and operating*, not about art direction. Prompt
craft, shot design, and motion direction for a specific model belong to
model- or craft-specific skills. Here the questions are: which gateway, going
through a gateway at all, how the async job contract works, how to discover a
model's schema, how to pin a version, how to cap spend, how to fail over, and
whose license governs the output.
Use a gateway when (production heuristics):
premium shots, a Kling/Seedance tier for volume, and an audio or upscale
model in the same pipeline — without signing and maintaining a separate
contract, key, billing relationship, and SDK per vendor.
of learning each vendor's bespoke long-running-job protocol.
re-plumbing auth, storage, and result handling.
you are not building that yourself.
has no first-party public API and is only reachable through resellers.
Go direct (or reconsider) when:
parameter, resolution, region, private-model hosting, or SLA the reseller
has not surfaced. Gateways expose a curated subset of each model's surface.
signed DPA** that the gateway cannot pass through from the underlying vendor.
beats per-call reseller margin.
added moderation layer (see Content safety below).
tune, and the direct API offers provisioned throughput.
Out of scope for this skill: direct model-provider APIs (Google Vertex/Gemini
Veo, Kling's own API, Runway, etc.), local/self-hosted inference, training or
fine-tuning, and image-only gateway use.
*Volatile facts below verified 2026-07-10; re-verify model IDs, prices, and
limits before shipping — the catalogs and rate cards move frequently.*
| Dimension | fal.ai | Replicate | WaveSpeed |
|---|---|---|---|
| Positioning | Generative-media-first (image/video/audio/3D), latency-focused | General model marketplace (community + "official" curated models), broadest catalog | Speed/latency-optimized inference for media |
| Catalog size (self-reported) | 1,000+ models | 100+ official models plus a large community catalog | 700–1,000+ models |
| Submit endpoint | https://queue.fal.run/{model-id} | https://api.replicate.com/v1/predictions (community) or .../v1/models/{owner}/{name}/predictions (official) | https://api.wavespeed.ai/api/v3/{model-id} |
| Async job states | IN_QUEUE → IN_PROGRESS → COMPLETED | starting → processing → succeeded/failed/canceled | created → processing → completed/failed |
| Sync option | Queue is the primary path; short models also have a run endpoint | Prefer: wait header (default 60s) for sync | Poll or webhook; result endpoint |
| Billing basis | Per-output (per-second or per-video for video); GPU-per-second fallback | Per-output for official models; per-second GPU time for community models | Per-model per-run, scales with resolution/duration/count |
| Balance model | Prepaid credits | Prepaid credit (hard cap at $0 for new accounts) | Prepaid credits (non-expiring) |
| Charged on failure? | No — pay only for successful output | Community models bill for GPU time even on model errors; official models bill per output | Pay per use; check per-model behavior |
| Schema discovery | Model-endpoints API with expand=openapi-3.0 | openapi_schema on the model version object | Per-model playground / API examples |
Sources: fal queue and pricing docs; Replicate HTTP API and pricing docs;
WaveSpeed predictions and pricing docs (see Sources).
Reading the table: the three overlap heavily on *what* you can call — the
same headline models (Veo, Kling, Seedance) appear on all three — so
differentiation is mostly in the *operational contract*: billing granularity,
whether failures cost money, how you discover a schema, and how versions are
pinned. Treat the catalog as necessary but not sufficient; the operational
contract is where integrations succeed or fail.
Video generation is slow (seconds to minutes), so every gateway is
fundamentally an asynchronous job queue. Do not build on a synchronous
request that blocks until the video is done — you will hit timeouts and lose
work. The universal shape:
it expires.
POST https://queue.fal.run/{model-id} with headerAuthorization: Key $FAL_KEY. Returns request_id, response_url,
status_url, cancel_url, queue_position.
GET https://queue.fal.run/{model-id}/requests/{request_id}/status(append ?logs=1 for runner logs). States: IN_QUEUE, IN_PROGRESS,
COMPLETED.
GET https://queue.fal.run/{model-id}/requests/{request_id}.PUT .../requests/{request_id}/cancel — removes queued jobsimmediately; in-progress jobs get a best-effort cancel signal.
POST https://api.replicate.com/v1/predictionswith a version field. Submit (official):
POST https://api.replicate.com/v1/models/{owner}/{name}/predictions
(no version needed).
Prefer: wait (or Prefer: wait=5) to hold the HTTPconnection up to 60s and return output inline for fast models — not suitable
for multi-minute video.
Cancel-After header (5s–24h) auto-cancels the predictionitself (distinct from Prefer: wait, which only bounds the HTTP wait).
urls.get until succeeded or failed;output is null until done. Web debug view at urls.web.
POST https://api.wavespeed.ai/api/v3/{model-id} → returns atask/prediction ID.
GET https://api.wavespeed.ai/api/v3/predictions/{id}/result.States: created, processing, completed, failed. outputs holds
result URLs; timings.inference gives duration in ms.
persist outputs promptly.
Prefer webhooks for video: a minutes-long poll loop wastes requests and
risks rate limits, and holding a connection open is fragile. Fall back to
polling only when you cannot expose a public HTTPS endpoint (local dev,
locked-down networks). If you poll, back off — image tasks ~2s, video tasks
~5s, then widen the interval (WaveSpeed's documented guidance, and a sound
default everywhere). Never poll a single job multiple times per second.
Design every webhook handler to be idempotent: gateways retry deliveries
and may deliver more than once for the same job ID.
Webhook payloads arrive from the public internet — verify them.
fal.ai signs webhooks with ED25519. To verify (verified 2026-07-10):
X-Fal-Webhook-Request-Id, X-Fal-Webhook-User-Id,X-Fal-Webhook-Timestamp, X-Fal-Webhook-Signature (hex).
https://rest.fal.ai/.well-known/jwks.json and cache itup to 24h.
SHA-256 of the raw body, newline-joined) and verify the signature against
the JWKS ED25519 public keys.
webhook_ip_ranges fromhttps://api.fal.ai/v1/meta.
fal delivery/retry (verified 2026-07-10): initial delivery timeout 15s;
on failure it retries 10 times over 2 hours. The success payload carries
status: "OK", results, and a seed; a request error carries
status: "ERROR"; a serialization failure sets payload: null with a
payload_error. The request_id is stable across retries; gateway_request_id
reflects the latest attempt.
Replicate lets you filter which events fire via webhook_events_filter
(e.g. ["completed"]); start and completed are always sent. Replicate
supports webhook signature verification via a signing secret — verify before
trusting a payload. (Verify current mechanism in Replicate's webhook docs
before shipping; verified conceptually 2026-07-10.)
WaveSpeed delivers a POST with id, model, input, outputs (on
completed), urls.get, status, and error (on failure). Confirm current
signing details in WaveSpeed's webhook docs before relying on them.
Each model exposes a different input surface (resolution enums, duration caps,
image/video reference slots, audio toggles, seed, negative prompt). Never
hardcode assumptions — discover the schema so your integration validates
inputs and adapts when a model updates.
expand=openapi-3.0 and returnsthe full OpenAPI 3.0 schema for an endpoint in the openapi field. Query
by endpoint_id (e.g. fal-ai/veo3) to get JSON Schema for every input
parameter. Use list/search modes to enumerate the catalog. (Verified
2026-07-10.)
/v1/models/{owner}/{name}/versions/{version_id} and read
openapi_schema.components.schemas for the Input and Output schemas.
(Verified 2026-07-10.)
auto-generates Python/JS/cURL examples; there is no single documented
cross-catalog OpenAPI export as of 2026-07-10 — treat per-model docs as the
source of truth.
Heuristic: cache the discovered schema per (model-id, version) and
re-fetch on a version change or on the first 422/validation error. Building a
small schema-validation layer in front of the gateway catches
parameter-shape breakage before you spend money on a failed job.
Reproducibility is the reason to care about versions: the same prompt on a
silently-updated model can produce different motion, framing, or quality.
{owner}/{model}:{64-char-version-hash}. Pinning the hash guarantees
identical behavior over time; omitting it (or using an official model
addressed as {owner}/{model} with no hash) means Replicate keeps you on the
latest maintained version. Trade-off: pin the hash for reproducibility
and to avoid surprise behavior changes; use the unpinned official-model form
when you want automatic upgrades and less maintenance. Replicate documents
pinning as a reproducibility best practice. (Verified 2026-07-10.)
fal-ai/veo3, or versioned IDs like fal-ai/kling-video/v2/...). Version
granularity is encoded in the ID the vendor publishes; watch changelogs for
new IDs and deprecations of old ones.
Deprecation is a real operational risk on gateways, not a hypothetical.
Underlying models are retired or superseded (a new Kling/Veo/Seedance version
lands and the old endpoint is sunset). Mitigations (heuristics):
offered.
deprecated ID is a one-line change, not a code hunt.
and alerts when one stops resolving or its schema changes.
degrades to a substitute instead of failing the pipeline.
Video is the most expensive generative modality per call, and gateway spend can
run away fast under retries or a bad loop. Controls differ sharply:
your balance. You pay only for successful outputs; you are not charged for
server errors (HTTP 500+) or time spent queued. fal's own docs do not
advertise per-project budget alerts as of 2026-07-10 — build your own spend
accounting from the billing/platform APIs.
prepaid credit is now the hard cap — work stops at a $0 balance (prepaid
credit for new accounts since July 16, 2025). There are **no native budget
alerts** post-deprecation; low-balance alerting is left to you or third-party
tools. Note the failure-billing asymmetry: **community models bill for GPU
seconds even when the model errors**, so a crash-looping community model
still costs money; official models bill per output. (Verified 2026-07-10.)
pricing where cost scales with resolution, duration, and output count. The
cost estimate is shown before you run, and can be queried via the pricing
API. Monthly credit lines exist for approved/enterprise accounts. (Verified
2026-07-10.)
Production heuristics for cost safety:
is your real circuit breaker on all three.
resolution × output count (× audio, on some models)**. A 8s 1080p Veo clip
with audio is dramatically pricier than a 5s 720p Kling clip — know the unit
price. Example (fal, verified 2026-07-10): Veo 3.1 is billed ~$0.20/s at
720p/1080p without audio and ~$0.40/s with audio; Kling 2.5 Turbo Pro
~$0.07/s. Prices change — re-verify.
a failing model is the classic runaway on any prepaid gateway.
experimentation, and prefer per-output over per-GPU-second when your job
latency is unpredictable.
attribute and alert; do not rely on the gateway to warn you.
Retries. Transient failures (429 rate limit, 5xx, capacity/cold-start
timeouts) are expected. Retry with exponential backoff and jitter, an
idempotency key you control, and a cap (both attempts and total spend).
Because a submitted-but-slow job is not the same as a failed job, distinguish
"submit failed" (safe to retry) from "job running slowly" (poll/await, do not
resubmit — resubmitting double-charges). Honor Retry-After when present.
Cross-gateway failover. Because the same headline models appear on multiple
gateways, a gateway is a substitutable route, which is the strongest
operational argument for the aggregator pattern. To fail over:
constraints)`), not a gateway-specific call, so the router picks a
provider+model.
"premium cinematic" → Veo on fal *or* Veo on Replicate; "volume" → Seedance on
WaveSpeed *or* Kling on fal).
status vocabulary, parameter names, and result JSON. Failover only works if
you translate inputs/outputs to a canonical form.
exposed parameters, default resolution, moderation, or even minor pipeline
details. Failover preserves *availability*, not necessarily *identical
output* — treat a failover result as a near-substitute, not a bit-identical
one, and re-verify quality-critical shots.
is not blocked by a zero balance on the backup.
Two moderation layers stack: the underlying model's own safety behavior and
the gateway's added moderation. They are not identical across gateways, so
the *same prompt on the same model* can be accepted on one gateway and blocked
on another.
filtering, maintains an NSFW content policy, and partners with NCMEC/Thorn
(CSAM) and StopNCII (NCII). Some image models expose an
enable_safety_checker toggle (default on); video-model moderation posture
is model-dependent. Users remain bound by fal's Acceptable Use Policy.
(Verified 2026-07-10.)
moderation strictness is largely inherited from the specific model.
Heuristics: (1) test moderation behavior per gateway, not once — a
prompt cleared on your primary may be refused on your failover route, breaking
availability assumptions. (2) Handle a moderation rejection distinctly from
a transient error — it is deterministic, so retrying or failing over the *same*
prompt often just fails or double-charges; instead surface it, adjust the
prompt, or route to a model with a different policy. (3) Do not assume a gateway
strips or adds a watermark, C2PA/provenance signal, or safety filter the same
way the direct API does — verify.
Two separate questions: where does your data live, and who may use the
output commercially.
Data retention / residency:
(v3.fal.media/...). Retention is controllable via the
X-Fal-Object-Lifecycle-Preference header (expiration_duration_seconds, or
null for no expiry). Request/response JSON is stored 30 days by default.
Download outputs before they expire — expired files are unrecoverable.
(Verified 2026-07-10.)
are auto-removed after ~1 hour by default — you must persist outputs
yourself. (Verified 2026-07-10.)
Because outputs are transient, always download and re-host anything you
need; never link end users directly to a gateway CDN URL for durable content.
For regulated data, confirm whether the gateway can offer a DPA or regional
processing — most reseller flows send your inputs to third-party model infra.
Commercial-rights passthrough (the critical governance point): a gateway
granting you rights to *its service output* does not override the
underlying model's license. Rights flow through the model, not the gateway.
including commercial use — **but explicitly subject to Third-Party Terms
determined by the model you used.** Some models (e.g. permissive ones like
Stable Diffusion) impose no ownership claim; others restrict commercial use.
(Verified 2026-07-10.)
vendor license** governs whether you may use a clip commercially, whether
attribution is required, and whether the vendor asserts training or usage
rights over your inputs.
Heuristic: before using a gateway-generated video commercially, resolve the
underlying model's license and the model provider's terms, not just the
gateway's ToS. Premium models (Veo, Kling) frequently carry provider-specific
commercial terms and per-clip restrictions; a gateway reselling them does not
relax those terms. Record, per model you ship on, the license basis for
commercial use so a later dispute or model swap is auditable.
| Symptom | Likely cause | Action |
|---|---|---|
| Submit returns fast but no video | Working as designed — it is async; you got a job ID, not a result | Poll status/result or await the webhook; do not treat the submit response as the output |
| Webhook never arrives | Endpoint not public HTTPS, handler slow (>15s on fal), or signature check rejecting valid payloads | Verify reachability, respond quickly then process async, confirm signature logic; fall back to polling |
| Duplicate results processed | Gateway retried delivery | Make the handler idempotent keyed on job/request ID |
| 422/validation error | Wrong parameter shape for this model/version | Re-fetch the model schema (fal openapi-3.0, Replicate openapi_schema); validate inputs before submit |
| Charged but no usable output | Community-model GPU-time billing on error (Replicate), or output expired before download | Prefer pay-per-output models for flaky work; download outputs immediately; check retention window |
| Model ID stopped resolving | Version deprecated/sunset | Update the config-driven model map to the new versioned ID or fail over to the tier fallback |
| Prompt blocked on one gateway, fine on another | Different stacked moderation layers | Treat as deterministic; adjust prompt or route to a different-policy model — do not blind-retry |
| Runaway spend | Unbounded retries or a bad loop against a prepaid balance | Cap retries/concurrency, keep balance small, instrument per-job cost |
| Output looks different after a "no-op" change | Unpinned version silently updated | Pin explicit version hashes/IDs; add a schema/behavior probe in CI |
| Result URL 404s later | Gateway CDN retention elapsed | Re-host durable content; never serve end users a raw gateway URL |
Intent: generate 500 five-second 720p vertical clips/day, cost-sensitive,
some prompts near content-policy edges, must not exceed a fixed daily budget.
Reasoning: volume + cost sensitivity favors a per-second/per-run model on a
gateway that (a) does not charge for failures and (b) supports a hard prepaid
cap. Pick a mid-tier model (e.g. a Kling/Seedance turbo tier) rather than Veo.
Put a second gateway behind the same capability tier for failover, since
availability of a single model on a single gateway is not guaranteed at 500/day.
Plan: primary = fal queue.fal.run submit + ED25519-verified webhook,
prepaid balance sized to the daily budget as the circuit breaker; secondary =
same-tier model on WaveSpeed with its own funded balance. Router picks by
health + queue depth. Per-job cost is logged (job ID → model → params → charge).
Moderation rejections are surfaced to a review queue, not retried.
Expected result: clips delivered async, budget bounded by prepaid balances,
a sunset or outage on one route degrades to the other. Failure modes:
forgetting to fund the backup balance (failover silently fails); blind-retrying
moderation rejections (wasted spend); serving raw CDN URLs that later 404.
Intent: a hero shot must render identically across months for a marketing
pipeline; commercial use required.
Plan:
GET /v1/models/{owner}/{name}/versions and pickthe hash; store {owner}/{name}:{version_hash} in config.
POST /v1/predictions with that pinned version, a webhook, andwebhook_events_filter: ["completed"]; set a Cancel-After deadline as a
spend guard.
openapi_schema.components.schemas.Input before submitting.
retention window.
commercial use, and record that basis, before publishing.
Why structured this way: pinning the hash locks reproducibility; the
schema check prevents parameter drift from silently changing output; the
retention-aware download prevents a lost result; the rights check prevents
shipping a clip whose model license forbids commercial use. Failure modes:
using the unpinned official-model form (auto-upgrades break reproducibility);
assuming Replicate's ownership grant overrides the model's restrictive license.
Intent: integrate a video model you have not used; you do not know its
duration cap, resolution enum, or whether it takes an image reference.
Plan: call the fal model-endpoints API with endpoint_id set to the model
(e.g. fal-ai/veo3) and expand=openapi-3.0; read the returned openapi
field's JSON Schema for the input object. Generate your request typing and a
validation layer from it; cache it keyed on the model ID; re-fetch on the first
422 or a changelog-announced version bump.
Why: this makes the integration self-describing and resilient to the model
adding/renaming a parameter, instead of hardcoding a guessed body that breaks
on the next model update. Failure mode: skipping discovery and hardcoding
parameters copied from a blog post that lags the current schema.
All URLs verified 2026-07-10.
Model-specific prices (Veo 3.1 ~$0.20–0.60/s by resolution/audio; Kling 2.5
Turbo Pro ~$0.07/s) are from fal's pricing pages, verified 2026-07-10, and are
volatile — always re-verify against the live rate card.
Take calesthio/video-generation-gateways from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.