Media3 Custom Audio Processor Agent Skill

Custom audio processors in a Media3 pipeline

A processor is a BaseAudioProcessor subclass installed in the sink's processor chain:

DefaultAudioSink.Builder(context).setAudioProcessorChain(
    DefaultAudioSink.DefaultAudioProcessorChain(
        arrayOf(myFilter, myGain),   // yours, in order
        SilenceSkippingAudioProcessor(...), SonicAudioProcessor(),
    ),
).build()

Four members matter: onConfigure (negotiate the format), isActive (in the chain or not),

queueInput (do the work), onFlush/onReset (drop state). Build the chain inside a

DefaultRenderersFactory subclass's buildAudioSink override (protected there), so instances are

created per player — once more than one is alive (a crossfade, a preroll, a precache), it matters,

because a processor carries per-stream state and cannot be shared.

Traps

isActive() is asked when the chain is built, not per buffer — a parameter you flip at runtime

must not ride it. A processor answering false while "disabled" is dropped from the chain until

the next configure or flush — from outside, "works, but only from the next track on." Stay active

and branch inside queueInput:

override fun queueInput(inputBuffer: ByteBuffer) {
    val remaining = inputBuffer.remaining()
    if (remaining == 0) return
    val output = replaceOutputBuffer(remaining)
    if (!enabled) { output.put(inputBuffer); output.flip(); return }  // alias-safe? see the put() trap below
    …
}

Do not get that by overriding isActive() to true — do not override it at all. The base class

already answers it from the format onConfigure just returned (confirmed by decompiling it, below).

configure() is final: it sets pendingOutputAudioFormat = onConfigure(format), *then* asks

isActive(). The default already means "active iff I accepted the format" — exactly what a

runtime-toggled processor wants, for free. Forcing true claims membership while handing back a

format you cannot read, and AudioProcessingPipeline catches it at configure time:

AudioFormat out = p.configure(f);
if (p.isActive()) { Preconditions.checkState(!out.equals(NOT_SET)); f = out; }

That throws IllegalStateException in an encoding you decline — latent until then, which is why

forced overrides survive. Activity is about the *format*, not the setting: a neutral curve or

pass-through filter stays active and takes the bulk-copy path below.

Trust the code over the comment on this one. It is common to find a KDoc promising "when

disabled, isActive returns false and the pipeline skips this processor entirely" sitting directly

above override fun isActive(): Boolean = true. Both are wrong — read the override, then delete it.

Always consume the whole input buffer. Leaving even one byte behind makes the pipeline offer the

same buffer again forever, having made no progress — silence with no error or log line. A

while (remaining() >= 2) loop is fine only because PCM16 frames are an even byte count; add the

drain anyway, because it is the difference between a wedge and a click:

while (inputBuffer.remaining() >= 2) { output.putShort(…) }
while (inputBuffer.hasRemaining()) { output.put(inputBuffer.get()) }  // never fires; keeps the contract

To verify: assert inputBuffer.remaining() == 0 at the end of queueInput, in a debug build.

ByteBuffer.put(ByteBuffer) throws when source and destination are the same object.

replaceOutputBuffer() normally hands back this processor's own buffer while the input belongs to

the previous stage, so they are distinct — but a chain where your processor is first, or one that

hands buffers through unchanged, can alias them. Prove they cannot, or guard the copy:

private fun copyBuffer(src: ByteBuffer, dst: ByteBuffer, size: Int) {
    if (src === dst) { dst.position(0); dst.limit(size); return }
    val pos = src.position()
    for (i in 0 until size) dst.put(src.get(pos + i))
    src.position(pos + size)
}

One instance per player; share the *parameter*, not the processor. The state that forbids sharing is

the format and output buffer, not your logic — give each player its own processor, all reading one field:

@Volatile private var fadeFactor = 1.0f              // written once, anywhere
val sleepFade = SleepFadeAudioProcessor { fadeFactor } // per player, same source

Passing a () -> Float rather than a value is the point: one write covers both transition players.

Read the parameter fresh on every buffer. Capturing it in onConfigure or at stream start repeats the

isActive failure one layer down: a fade that starts, or keeps advancing, mid-stream never reaches the output.

The pass-through branch runs for the entire life of the app. Its "nothing to do" path executes on

every buffer of every track whether the feature is in use or not. Make it a bulk

output.put(inputBuffer), never a per-byte loop.

Decline formats in onConfigure, and re-derive everything from it. Return NOT_SET for

encodings you don't handle, and cache sampleRate/channelCount there — coefficients built from

the last stream's rate are silently wrong, not obviously broken:

override fun onConfigure(f: AudioProcessor.AudioFormat): AudioProcessor.AudioFormat {
    if (f.encoding != C.ENCODING_PCM_16BIT) return AudioProcessor.AudioFormat.NOT_SET
    sampleRate = f.sampleRate; channelCount = f.channelCount
    coefficientsDirty = true
    return f
}

onFlush and onReset are not the same event. Flush fires on every seek and format change, and

is the other point where the pipeline rebuilds its active list from isActive() — clear only the

*history* (filter delay lines) there. Reset fires at teardown: restore defaults too. Clearing

user-facing parameters in onFlush makes settings reset themselves on seek.

Byte order is not free. Call inputBuffer.order(ByteOrder.nativeOrder()) before reading

shorts. ByteBuffer defaults to big-endian; PCM in the pipeline is native-endian. Getting it wrong

sounds like loud noise, not like a subtle bug — which is the one merciful thing about it.

Verifying it

• isActive()'s default reads the format, not a hardcoded value — decompile it yourself:

   JAR=$(find ~/.gradle/caches -name "media3-common-1.11.0-runtime.jar" | head -1)  # AAR→jar transform cache
   [ -z "$JAR" ] && { JAR=/tmp/media3-common.jar; unzip -p "$(find ~/.gradle/caches/modules-2 -name 'media3-common-1.11.0.aar' | head -1)" classes.jar > "$JAR"; }
   javap -c -p -classpath "$JAR" androidx.media3.common.audio.BaseAudioProcessor | grep -A6 isActive

Pass condition: if_acmpeq compares pendingOutputAudioFormat against NOT_SET — no flag read.

• By hand: flip a runtime toggle mid-track. Correct: audible within the current track, not

only from the next one — otherwise a parameter or isActive() is read once, not per buffer.