Shortform Editing Agent Skill

Shortform Editing — Cut the Footage Into a Posted-Ready Short

*You are the edit-bay technician.* Someone hands you real footage — a talking-head take, a screen recording, a voiceover, some B-roll — and you turn it into a vertical short that is ready to post: silences gone, cuts fast, captions burned in word-by-word, motion snapped to the beat, exported to spec. You encode pixels and timing. You do not write the words or pick the hook.

The rigor is a checkable file: a 1080x1920 H.264 MP4 with burned captions that ffprobe confirms, plus the intermediate artifacts (a cut list, an .ass/.srt, a beat grid). If you cannot ffprobe the result and see the specs, you are not done.

Scope: three skills touch "a vertical video" — know your lane

| The ask | Goes to | Artifact |

|---|---|---|

| Write the script, hook, beats, on-screen caption *text*, edit decision sheet | ../video-shorts/SKILL.md | a .md script |

| Cut/caption/export real footage, tool-agnostic | here | edited .mp4 + caption/cut files |

| Build a Remotion React composition project (<TransitionSeries>, headless render) | ../remotion-video/SKILL.md | a .tsx project |

One-line router: footage in → cut, captioned, exported → this skill. Words or a plan → video-shorts. A React codebase → remotion-video. You are the only one of the three that is tool-portable.

Pick your route first

Do not start cutting before you have chosen the tool. The choice is about who runs it and whether it must reproduce.

| Situation | Route | Why |

|---|---|---|

| Reproducible / batch / CI / "do this to 40 clips" | ffmpeg + WhisperX | scriptable, deterministic, no GUI — same input, same output |

| The repo is already a Remotion project | @remotion/captions | reuse the existing composition graph; don't shell out |

| A human is polishing one video by hand | CapCut click-path | fastest for a single hands-on pass with auto-captions + auto beat markers |

The rest of this skill is the ffmpeg + WhisperX pipeline (the reproducible path), with the Remotion and CapCut routes called out where they diverge. Long command blocks live in references/ffmpeg-pipeline.md; the caption detail lives in references/captions.md.

The reproducible pipeline, in order

This is the spine. Run it top to bottom; each step feeds the next.

1. Extract a clean mono 16 kHz audio track for transcription.

ffmpeg -i in.mp4 -vn -ac 1 -ar 16000 a.wav

*Why mono 16 kHz: that is what Whisper models expect; feeding stereo 48 kHz wastes time and changes nothing about accuracy.*

2. Transcribe with word-level timestamps. Segment timestamps are not enough — word-by-word captions need per-word timing.

whisperx a.wav --model large-v3 --output_format json --highlight_words True

WhisperX (v3.8.5) runs faster-whisper plus wav2vec2 forced alignment, giving word timing accurate to under 100 ms — the floor for karaoke highlight. faster-whisper alone with word_timestamps=True works (~10% overhead) but drifts more on word boundaries. *Why forced alignment: Whisper's native word stamps are interpolated and jitter; forced alignment pins each word to the audio.* See references/captions.md for the install and the JSON shape.

3. Detect silence, then cut video AND audio together. This is the step people get wrong.

# Pass 1: find the silent ranges (speech threshold)
ffmpeg -i in.mp4 -af silencedetect=noise=-25dB:d=0.3 -f null - 2> silence.log

silenceremove alone is an audio-only filter and desyncs the video — never use it as a one-shot. Parse the silence_start / silence_end lines from silence.log, invert them into keep-ranges, and build a select + aselect filter (or a trim+concat graph) that drops the same ranges from both streams. Thresholds: speech noise=-25dB:d=0.3; music beds noise=-40dB:d=1. The filter-generation snippet is in references/ffmpeg-pipeline.md.

4. Build the beat grid (only if you are cutting to music).

python -c "import librosa; y,sr=librosa.load('music.wav'); \
_,b=librosa.beat.beat_track(y=y,sr=sr,units='time'); print(*b,sep='\n')" > beats.txt

beats.txt is a list of beat times in seconds. CapCut drops these markers on the waveform automatically; librosa (or aubio) gives you the same grid in a script.

5. Snap cut points and zoom keyframes to the nearest beat. Round each cut time to its closest value in beats.txt. A zoom-punch on the beat reads as energy; see Beat sync below for when NOT to.

6. Burn the karaoke .ass captions.

ffmpeg -i tightened.mp4 -vf "ass=captions.ass" -c:a copy captioned.mp4

The .ass carries the per-word \k timing built from step 2's word timestamps and the safe-zone margins. Generation detail and the template are in Karaoke captions below.

7. Export the platform master (see Export specs).

Karaoke captions

Word-by-word captions are the signature look. Build them from the word timestamps, not from segment timestamps.

• Timing. Each word's on-screen highlight comes from its forced-alignment start/end. In .ass, the \k<centiseconds> tag advances the karaoke fill per word. Word-by-word from *segment* timing jitters and lands the highlight on the wrong word — always use the per-word stamps from step 2.
• Legibility for sound-off. Big bold sans, heavy outline/stroke, optional drop shadow. People watch muted: if it is not readable at a glance on a phone, it failed. Keep it to ≤ 2 lines / ~6 words on screen at once.
• Safe zone — the non-obvious one. Place captions in the vertical middle band, not the bottom. Platform UI (right-rail icons, bottom caption/CTA/profile) covers the bottom ~15–20% and the right edge. Keep captions and key text inside the central ~80% of the 1080x1920 frame. Captions burned at the bottom get hidden behind the TikTok buttons — this is the #1 "my captions are broken" cause.

Minimal .ass style block (full template in references/captions.md):

[V4+ Styles]
Style: Karaoke,Montserrat,64,&H00FFFFFF,&H0000FFFF,&H00000000,&H64000000,-1,0,0,0,100,100,0,0,1,5,2,5,80,80,640,1

MarginV (the trailing 640) lifts the line into the middle band; the heavy Outline (5) keeps it readable on any background.

Remotion route: if you are inside a Remotion project, use @remotion/captions — createTikTokStyleCaptions({ captions, combineTokensWithinMilliseconds }) pages the tokens. Low (200–500 ms) = word-by-word pop; high (1200–2000 ms) = phrase pages. Page durationMs was added in v4.0.261. CapCut route: Text → Auto captions → Generate, then restyle.

Beat sync

A beat grid (step 4) is a list of seconds. Snap your cut points and zoom keyframes to the nearest beat so motion lands on the music.

• Bad → Good. Cutting mid-word to land a cut exactly on a beat reads as a mistake → nudge the cut to the nearest beat that also respects the speech boundary. Honor the words first, the beat second.
• When NOT to beat-cut. A talking-head explainer where the words carry the value: clarity beats rhythm. Beat-cutting a person mid-sentence chops their delivery. Reserve beat sync for B-roll montages, transitions, and music-led segments.
• Transitions stay cheap and fast. Hard cut is the default. Whip-pan or zoom-punch only on a beat. Avoid 1-second crossfades in a 30-second short — they kill the pace; a short lives on fast hard cuts.

Export specs

The universal vertical master — one file that plays cleanly across all three platforms:

ffmpeg -i captioned.mp4 \
  -vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2,setsar=1" \
  -c:v libx264 -profile:v high -pix_fmt yuv420p -r 30 \
  -c:a aac -b:a 192k -ar 48000 -ac 2 \
  -movflags +faststart out.mp4

yuv420p (broad playback), +faststart (web streaming starts before full download), AAC-LC 48 kHz. Use 30 fps; bump to 60 fps for high-motion Shorts.

| Platform | Video bitrate | Notes |

|---|---|---|

| TikTok | ~2,000–4,000 kbps | lightest of the three |

| Instagram Reels | ~3,500–4,500 kbps | |

| YouTube Shorts (1080p) | ~8,000–15,000 kbps | up to ~53,000 for 4K/60 |

A single 1080x1920 H.264 MP4 @30 fps under ~250 MB is safe everywhere. Verify before you ship:

ffprobe -v error -select_streams v:0 \
  -show_entries stream=width,height,codec_name,avg_frame_rate out.mp4

Expect width=1080 height=1920 codec_name=h264. The full per-platform encode flags and ffprobe assertions are in references/ffmpeg-pipeline.md.

Anti-patterns

| Bad | Why it breaks | Good |

|---|---|---|

| silenceremove as a one-shot filter | It is audio-only; the video keeps playing → audio/video desync | silencedetect → invert ranges → select+aselect cut both streams |

| Word captions from segment timestamps | Highlight jitters and lands on the wrong word | Forced-alignment per-word stamps (WhisperX / word_timestamps) |

| Captions burned in the bottom 15–20% | Platform UI (buttons, CTA, profile) covers them | Place in the vertical middle band, central ~80% |

| One export reused for every platform | Over/under the bitrate target somewhere | Per-platform bitrate, or the safe universal master |

| 1-second crossfades in a 30s short | Kills the pace; reads slow | Hard cuts; whip/zoom only on a beat |

| Beat-cutting a talking-head mid-sentence | Chops the delivery; clarity lost | Beat-sync B-roll/montage only; honor speech boundaries |

| Re-running the encode to "check the specs" | Wastes minutes; the file already exists | ffprobe the output; verify, don't re-render |

| Writing the caption copy or the hook here | That is the script, not the cut | Route to ../video-shorts/SKILL.md; you only burn it in |

| ffmpeg overwriting the source in place | One bad flag and the original is gone | Always write to a new file; keep in.mp4 untouched |

Verify & references

• bash scripts/verify.sh <output.mp4> — runs ffprobe and asserts width=1080, height=1920 (9:16), video codec h264, audio codec aac, fps in {30, 60}, and that a sibling .srt/.ass caption file exists and is non-empty. With no ffprobe it falls back to a path/extension check. Read-only; exits 0 on an empty/clean target so it never false-fails. The "does it feel on-beat" judgment is not mechanical — that lives in the capability eval.
• references/ffmpeg-pipeline.md — the full silence-detect → keep-range inversion → select/aselect filter generation that cuts video+audio together, the concat/trim approach, per-platform encode flags, ffprobe assertions, and desync/audio-drift/faststart troubleshooting.
• references/captions.md — WhisperX install + run, the word-timestamp JSON shape, the full .ass karaoke template with safe-zone margins, the Remotion createTikTokStyleCaptions() snippet with combineTokensWithinMilliseconds guidance, and the faster-whisper vs WhisperX tradeoff.