Compose Desktop Video No Swingpanel Agent Skill

Compose Desktop video without an embedded panel

Most native media engines expose a software render path: you hand it a pixel buffer and a size,

it decodes, scales and letterboxes a frame into that buffer, and you copy it out. The obvious way to

show the result in Compose Desktop is to blit it onto an AWT/Swing component and embed that. Do not.

Publish finished frames as immutable images on a StateFlow and draw them with a plain Image; the

engine side is identical and the whole embedding bug class disappears.

Why the embedded-panel route fails

Three separate failures, all structural rather than fixable:

• A heavyweight embedded component is an overlay: it draws above every Compose node regardless

of z-order, so nothing can be placed on top of the video.

• It repositions one frame late while the page scrolls — visible as a flicker that exposes

whatever is behind the window, and worse on a transparent window.

• The toolkit gives a component exactly one parent: two screens composing the player fight over

the single instance, and the loser renders black until next/previous — the "video randomly missing" report.

An Image participates in normal Compose rendering, and any number of screens can collect the same

frame source at once.

Traps

Pick the engine pixel format that already matches your image type. The engine this was mined

from names formats by byte order at increasing addresses (a bgr0-style name means B at offset

0, G at 1, R at 2, unused at 3) — but that is its convention, not a norm: other render APIs name

packed formats most-significant-byte-first, so check your engine's documentation before trusting

any name. The JVM's packed-integer RGB image type stores a pixel as 0x00RRGGBB, which on a

little-endian host — every platform desktop Java ships on — occupies memory as B, G, R, unused:

identical to the bgr0-style layout, so a bulk read needs no per-pixel fixup. Picking the name

with the opposite convention is exactly what produces the classic red/blue-swapped video.

Use the packed-integer type WITHOUT alpha. Engines document the fourth component as

uninitialized — often zero, but not guaranteed. In an ARGB image that byte *is* the alpha channel,

so a garbage zero makes every frame fully transparent. The non-alpha type ignores the high byte.

Honour the stride alignment the engine asks for, and make the staging buffer stride-wide. Pad

the row stride up to the requested alignment (commonly 64 bytes) to keep the engine on its fast

path, and make the staging array *stride*-wide too so the whole frame is one contiguous run, read

out of native memory in a single bulk copy. Publish only the leftmost width columns — the padding

between one row's end and the next row's start is explicitly unspecified.

Publish a new image per frame. A StateFlow conflates equal values, so re-emitting a mutated

image is dropped and the picture freezes. A fresh snapshot per frame is also immutable, which lets

any collector convert or draw it on any thread with no tearing — replacing the per-frame blit the

UI thread used to do, so total copy work is unchanged.

// adapted
val snapshot = BufferedImage(target.width, target.height, BufferedImage.TYPE_INT_RGB)
val out = (snapshot.raster.dataBuffer as DataBufferInt).data
for (y in 0 until target.height) {
    System.arraycopy(target.pixels, y * target.bufferWidth, out, y * target.width, target.width)
}
_frames.value = snapshot

Convert off the UI thread, and report the box's size back to the source:

var frame by remember(source) { mutableStateOf<ImageBitmap?>(null) }
LaunchedEffect(source) {
    withContext(Dispatchers.Default) {          // the conversion copies pixels
        source.frames.collect { image -> frame = image?.toComposeImageBitmap() }
    }
}
Box(modifier.onSizeChanged { source.setTargetSize(it.width, it.height) }) {
    frame?.let { Image(bitmap = it, contentDescription = null, contentScale = ContentScale.Fit) }
}

The engine decides the fit, not Compose. It scales *and letterboxes* into exactly the size you reported,

so the black bars are already pixels by the time Compose sees the frame — no ContentScale can remove them.

Cropping is the engine's own pan-scan property, and applying it from a LaunchedEffect keyed on the handle

and the flag re-scales the running video instead of tearing down and recreating the handle:

// adapted
LaunchedEffect(player, cropToBounds) { player?.setPanscan(if (cropToBounds) 1.0 else 0.0) }

ContentScale still matters briefly after a resize, until the next correctly-sized frame renders.

The engine's update callback fires on a foreign thread and may only signal. Render APIs

typically forbid calling any of their functions from inside it, so signal a semaphore and do all

rendering on a thread you own that calls nothing but the render functions. In that loop: use a

bounded wait so a stop flag is noticed promptly, then drain the permits — a backlog should

collapse to the newest frame, not run once per missed frame — and skip rendering entirely unless

the engine reports a new frame or a resize invalidated the buffer, since it is processor-bound.

Swap the published source unconditionally, and clear it on detach. A null-guard around "set the

frame source for the new handle" keeps the *outgoing* handle's source on screen whenever the

incoming track has no video — a dead surface belonging to a handle about to be released, the "black

video until next/previous" symptom in its second form. Nulling it on detach lets the UI fall back

to artwork instead of holding a stale frame.

Allocate and reallocate on the render thread. Size changes arrive from Compose layout; keeping

allocation on the render thread keeps it off the UI, and swapping the whole target object wholesale

— rather than mutating width/height in place — means the render loop and teardown can never

disagree about dimensions mid-copy.

Keep the native memory the render parameters point into alive. The parameter block usually

stores raw pointers into small allocations holding the size, the stride and the format string; the

engine dereferences them on every render call. Fields that look unused are load-bearing — losing

the reference lets the binding free them out from under native code.

Verifying it

Run from the repo root, where the core/media/... paths below resolve.

• SwingPanel is gone from the video path; frames publish on a StateFlow; the format pairs with the non-alpha image type:

   grep -n "val frames: StateFlow<BufferedImage?>\|bgr0\|TYPE_INT_RGB" core/media/media-jvm/src/main/java/com/simpmusic/media_jvm/mpv/MpvVideoFrameSource.kt
   grep -rn "SwingPanel(" core/media/media-jvm-ui core/media/media-jvm

Pass condition: the first prints all three; the second prints nothing.

• A fresh image is allocated per frame, inside the render loop, never the resize path:

   grep -n "fun renderLoop\|fun ensureSurface\|BufferedImage(target.width" core/media/media-jvm/src/main/java/com/simpmusic/media_jvm/mpv/MpvVideoFrameSource.kt

Pass condition: the allocation's line sits between renderLoop and the next function, never inside resize-only ensureSurface.

• setPanscan runs inside the crop-setting LaunchedEffect; ContentScale.Fit is a separate hit on the Image call:

   grep -n "LaunchedEffect(mpvPlayer, cropToBounds)\|setPanscan\|ContentScale.Fit" core/media/media-jvm-ui/src/main/java/com/maxrave/media_jvm_ui/ui/MediaPlayerView.kt

Pass condition: setPanscan is the line immediately after the LaunchedEffect match.