Profile Model Performance

Profile model performance

Use this skill before changing runtime behavior. The goal is to produce a

defensible bottleneck map and a short list of candidate optimizations, not to

guess from code shape alone.

Workflow

• Scope the executed path.
• Find the user-facing entry point: runner, CLI, interactive server, batch

script, notebook, or downstream adapter.

• Trace one generation step through input preparation, encode/context setup,

model or denoise loop, cache update/finalize, decode, transfer, encode, and

presentation.

• Read flashdreams-integrations before changing framework boundaries or

config contracts. Keep this skill focused on measurement and diagnosis.

• Prefer no-GPU inspection first: config resolution, --help,

--no-instantiate, static runner wiring, and small CPU tests.

• Establish a reproducible baseline.
• Use fixed input, seed, prompt/control schedule, resolution, chunk/window

settings, checkpoint source, and device.

• Make the run long enough to separate cache fill, compile/autotune, and

steady-state chunks.

• Record exact command, commit, GPU, driver, CUDA, PyTorch, cuDNN, dtype,

compile cache state, and checkpoint identifiers.

• Track startup/prewarm wall time separately from visible or steady-state

timing. Do not average cold compile or cache-fill chunks into the headline

steady-state metric.

• Add timing boundaries that respect CUDA asynchrony.
• Use CUDA events or explicit synchronization between major stages when

attributing GPU time.

• Report median and p90 after warmup for total chunk time and stage timings.
• Useful stage names: input/encode, context setup, denoise/model, cache

update submit, cache update wait, decode, GPU-to-CPU transfer,

frame/materialization, image/video encode, queue wait, present pacing, and

end-to-end chunk time.

• Print the active runtime settings in summaries so logs cannot be detached

from the flags that produced them.

• Classify the bottleneck.
• Model/denoise: attention, GEMM, normalization, scheduler loop overhead,

dynamic shapes, SDPA backend selection, torch.compile, CUDA graph

capture, or copy/layout inside the model step.

• Cache: append/slice churn, rolling-window materialization, K/V refresh

cost, cache update synchronization, reset/scene-switch rebuild behavior, or

stale state after async work.

• Decode: VAE/decoder wall time, streaming decoder cache, layout

conversions, convolution/elementwise hot blocks, lightweight decoder

quality tradeoffs, or unsafe whole-decoder compilation.

• Transfer and presentation: GPU-to-host copies, CPU image/JPEG encoding,

browser/server queue backlog, rate limiting, frame pacing, or display

latency.

• Multi-GPU/serving: context parallel shape boundaries, distributed cache

state, device-to-device transfers, per-rank persistence, and scheduler or

presenter behavior outside a single-process demo.

• Build the narrowest useful probe.
• Sweep one axis at a time when possible: window size, cache mode, compile

mode, graph mode, decoder choice, decoder layout, presentation queue, or

attention backend.

• Use fresh processes for compile/cache studies so startup behavior and

persistent compiler cache effects are visible.

• Use decoder-only same-latent probes for decoder changes so stochastic model

drift cannot explain quality differences.

• Profile only after a sweep identifies the hot stage. Treat profiler wall

time as perturbed attribution evidence, not the headline benchmark.

• End with a short diagnosis note.
• State the current bottleneck, the baseline numbers, the commands used, and

the next optimization candidates.

• Separate proven facts from hypotheses. If evidence is missing because GPU

validation was not run, say so and provide the exact command to run later.

Common pitfalls

• Do not infer the active attention or decoder backend from Python control flow;

confirm with profiler kernels or explicit runtime logging.

• Do not compare moving autoregressive rollouts as strict quality metrics when

different speeds or kernels can shift camera position or content. Use them as

smoke tests.

• Do not treat a fast lightweight decoder as a quality replacement without a

same-latent comparison against the quality decoder.

• Do not promote a startup-heavy compile path unless prewarm, persistent cache,

reset, and scene-switch behavior are acceptable for the target workflow.

• Do not optimize presentation by dropping generated frames for quality demos;

diagnose backlog separately, then tune ordered pacing and backpressure.

Deliverable

A good profiling pass leaves behind:

• a reproducible baseline command;
• trustworthy stage timings with warmup excluded;
• quality/reference artifacts when behavior may change;
• a ranked bottleneck list;
• candidate optimizations with the validation each one would require.