Video Understanding Agent Skill
> 把视频分析为结构化理解索引：场景检测、ASR 转写、逐场景 VLM 观察、静音窗口、融合时间线和写作 brief。 用于理解、索引或总结视频，也作为后续创作前的分析阶段。输入视频文件；输出 scenes.json、 asr_result.json、vlm_analysis.json、silence_periods.json、timeline_fusion.json、agent_narration_brief.md。 触发词：视频理解、视频分析、视频索引、video understanding、analyze video、看懂视频。
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/worldwonderer/video-recap-skills --skill video-understandingThe instruction itself
7 sections, as written by the author1. 定位
本技能把源视频转成 Agent 与下游阶段可读取的理解索引。它的创作角色是素材观察员 / 场记，不是导演：
- 先观察，再解释；事实与推断分开。
- 除了“发生了什么”，还要让下游看见知识、权力、目标、关系或情绪在哪一刻变化。
- 标出由谁的 POV 承载变化、哪个反应或表演不可替代，以及哪里存在完整台词/动作的自然剪辑边界。
- 证据不足时保留不确定性，不制造戏剧结论。
2. 处理阶段
- 场景检测：写
scenes.json，包含切点、时长和废片段过滤结果。 - 抽帧：为视觉分析提取代表帧。
- ASR：通过
mimo-v2.5-asr写时间戳对白asr_result.json。 - 静音检测：写
silence_periods.json，标注安静窗口与has_speech。 - VLM 观察：写
vlm_analysis.json，包含场景描述、深层分析和frame_facts。 - 时间线融合与创作 brief：写
timeline_fusion.json、asr_writing_chunks.json和agent_narration_brief.md。
各阶段只有在输出产物与 provenance sidecar 同时匹配当前视频及影响结果的设置时才会复用；--force 强制重算。
3. 环境要求
# ffmpeg: brew install ffmpeg | apt install ffmpeg | choco install ffmpeg
export MIMO_API_KEY=***
ASR 使用 mimo-v2.5-asr；VLM 使用 mimo-v2.5。--skip-asr 可跳过对白转写，但完整理解仍需要 MIMO_API_KEY 运行 VLM。--mimo-video-overview 可开启按场景块的视频概览。
若 work_dir/background_research.json 存在，本技能会把剧情梗概和角色名折入 VLM 上下文；--context 可补充一条简短提示。
下面的 scripts/... 均相对于本技能目录。若执行器从仓库根目录启动，请给脚本路径加上本技能的绝对目录。脚本不从其他技能目录读取文件；外部输入仅限命令显式传入的视频、参数与 work_dir 产物。
4. 运行命令
python3 scripts/understand.py <video> --work-dir <work_dir> \
[--context "节目名/角色名"] [--scene-threshold 0.1] [--skip-asr] [--mimo-video-overview] [--force]
5. 输出契约
| 文件 | 内容 |
|------|------|
| scenes.json | 场景切点、起止时间与时长 |
| asr_result.json | [{start, end, text}] 时间戳对白 |
| vlm_analysis.json | 逐场景描述、深层分析与 frame_facts |
| silence_periods.json | [{start, end, duration, has_speech}] 安静窗口 |
| timeline_fusion.json | VLM、ASR 与静音信息的统一时间线 |
| asr_writing_chunks.json | 按句界和场景切分的 ASR 写作块 |
| agent_narration_brief.md | Agent 首先阅读的创作简报 |
后续写作阶段根据创作简报与索引制定方案并写 narration.json。
6. 参考资料
- 背景调研：
references/research-guide.md，产出background_research.json。 - JSON 结构：
references/data-schema.md。
7. 能力边界
- 不写解说词，也不做解说评分；只负责生成理解索引与创作简报。
- 不剪辑、不配音、不合成视频。
- 不编造信号无法支持的剧情；当 ASR / VLM 过薄时输出素材警告。
- 不发布、不调度，只向
work_dir写产物并停止。
Other skills for the same job
different authors, same section of the catalogue
Create beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.
Creating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems. Create original algorithmic art rather than copying existing artists' work to avoid copyright violations.
Improves the quality of images, especially screenshots, by enhancing resolution, sharpness, and clarity. Perfect for preparing images for presentations, documentation, or social media posts.
Create and edit JSON Canvas files (.canvas) with nodes, edges, groups, and connections. Use when working with .canvas files, creating visual canvases, mind maps, flowcharts, or when the user mentions Canvas files in Obsidian.
Publish Markdown articles to X (Twitter) Articles editor with proper formatting. Use when user wants to publish a Markdown file/URL to X Articles, or mentions "publish to X", "post article to Twitter", "X article", or wants help with X Premium article publishing. Handles cover image upload and converts Markdown to rich text automatically.
Downloads videos from YouTube and other platforms for offline viewing, editing, or archival. Handles various formats and quality options.
Teaches the AI to design like a high-end agency. Defines the exact fonts, spacing, shadows, card structures, and animations that make a website feel expensive. Blocks all the common defaults that make AI designs look cheap or generic.
Lightweight WSI tile extraction and preprocessing. Use for basic slide processing tissue detection, tile extraction, stain normalization for H&E images. Best for simple pipelines, dataset preparation, quick tile-based analysis. For advanced spatial proteomics, multiplexed imaging, or deep learning pipelines use pathml.
How to use it
Take worldwonderer/video-understanding from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.
The instructions reference brew.
Without those the skill loads but fails at the first command.