Plotting Agent

Plotting Agent (Step 2)

Faithful implementation of the Plotting Agent from PaperOrchestra

(Song et al., 2026, arXiv:2604.05018, §4 Step 2 and App. F.1 p.45).

Cost: ~20–30 LLM calls. The paper uses PaperBanana (Zhu et al., 2026) as

the default backbone with a closed-loop VLM-critique refinement. This skill

expresses that loop in host-agent terms: you (the host agent) generate

matplotlib code with your own LLM, render via your Bash/Python tool,

optionally critique the rendered PNG with your vision model, redraw, and

finally caption.

Inputs

• workspace/outline.json — specifically the plotting_plan array
• workspace/inputs/idea.md and workspace/inputs/experimental_log.md —

the source data

• workspace/inputs/figures/ — optional pre-existing figures (PlotOn mode)

Outputs

• workspace/figures/<figure_id>.png — one PNG per plotting_plan entry

(300 DPI, sized to the requested aspect ratio)

• workspace/figures/captions.json — {figure_id: caption_text} map

Workflow

Per figure (executed independently per figure_id)

• Read the figure spec from outline.json:

   {
     "figure_id": "fig_main_results",
     "title": "Main Results on Dataset X",
     "plot_type": "plot",
     "data_source": "experimental_log.md",
     "objective": "Visual summary (Grouped Bar Chart) demonstrating ...",
     "aspect_ratio": "5:4"
   }

• Few-shot retrieval (visual planning): pick the matching pattern from

references/chart-patterns.md (for plot_type=="plot") or

references/diagram-patterns.md (for plot_type=="diagram").

• Extract data: parse idea.md and/or experimental_log.md

(data_source field tells you which) to obtain the numeric values or

conceptual entities the figure needs. For experimental_log.md, the

## 2. Raw Numeric Data section contains markdown tables.

• Render:

If PAPERBANANA_PATH is set — use the PaperBanana backbone

(Zhu et al., 2026). It runs a Retriever → Planner → Stylist → Visualizer

→ Critic loop and is especially good for plot_type == "diagram".

See references/paperbanana-cookbook.md for setup (needs a Gemini API key).

   python skills/plotting-agent/scripts/paperbanana_render.py \
       --figure-id <figure_id> \
       --caption   "<objective from figure spec>" \
       --content-file workspace/inputs/idea.md \
       --task      <diagram|plot> \
       --aspect-ratio <aspect_ratio> \
       --out       workspace/figures/<figure_id>.png

Otherwise — write a matplotlib script and run it via your Bash tool,

or use the bundled helper:

   python skills/plotting-agent/scripts/render_matplotlib.py \
       --spec spec.json \
       --out workspace/figures/<figure_id>.png

The script must apply the academic style from chart-patterns.md, use the

correct pixel size from aspect-ratios.md, save at 300 DPI, and call

plt.close() after savefig.

• VLM critique loop (optional, only if your host has vision):
• Reload the rendered PNG as a multimodal input to your LLM.
• Critique it against the figure's objective from the outline. Look for:

visual artifacts, mislabeled axes, illegible text, color clashes,

misleading scaling, missing legend, overlapping labels.

• If problems are found, regenerate the matplotlib script with corrections

and re-render. Cap at 3 critique iterations per figure.

• This is the closed-loop refinement step the paper inherits from

PaperBanana. See references/plotting-pipeline.md for the full loop

description.

• If your host has no vision input, skip this step entirely. The

figure will still render correctly, just without iterative refinement.

• Generate the caption using the verbatim Caption Generation prompt at

references/caption-prompt.md. Inputs to the caption prompt:

• task_name — the section the figure belongs to (e.g., "Methodology",

"Experiments")

• raw_content — the surrounding section text (or content_bullets from

the section_plan if the section isn't drafted yet)

• description — the objective field from the figure spec
• figure_desc — a 1-sentence description of what the rendered figure

actually shows (from your VLM critique pass, or from the script's plan

if no vision)

Write the caption to workspace/figures/captions.json keyed by

figure_id. **Captions must NOT contain Figure N: or Caption N:

prefixes** — the LaTeX template handles numbering. Plain text only, no

markdown.

Conceptual diagrams

For plot_type == "diagram", prefer PaperBanana when available — its

Retriever grounds the Planner in real published paper diagrams. If

PAPERBANANA_PATH is unset, follow references/diagram-patterns.md.

Patterns include block diagrams, system overviews, flowcharts, and

algorithm-as-graph. The bundled helper:

python skills/plotting-agent/scripts/render_diagram.py \
    --spec diagram_spec.json \
    --out workspace/figures/<figure_id>.png

handles the simple cases (boxes-and-arrows). For complex Fig-1-style

overview diagrams, write matplotlib patches code yourself.

Hard rules

• 300 DPI for every figure. Lower DPI gets rejected at the LaTeX compile

step on conference templates.

• Aspect ratio is exact. The figure spec's aspect_ratio is one of 12

enumerated strings. Use the pixel targets in references/aspect-ratios.md.

• Hide top and right spines for plots. (Diagrams: no spines at all.)
• Muted academic colors only. The palette is in chart-patterns.md.

Never use matplotlib defaults (too saturated for print).

• No 3D, no pie charts, no decorative visuals. The paper's evaluators

penalize these.

• Every figure MUST have a caption in captions.json. The Section

Writing Agent will fail-stop if a caption is missing for any figure

referenced from the outline.

• No Figure N: prefix in captions — LaTeX adds it.
• Never describe data you didn't plot. The Plotting Agent must not

hallucinate axes, baselines, or trends. Source-of-truth is

experimental_log.md or idea.md.

Pre-existing figures (PlotOn mode)

If workspace/inputs/figures/ is non-empty, check whether any pre-existing

file matches a figure_id in the outline (by filename prefix). If so,

copy it into workspace/figures/ as-is and still generate a caption

using the caption prompt. Only generate from scratch the figure_ids that

have no pre-existing counterpart.

Resources

• references/caption-prompt.md — verbatim Caption Generation prompt from App. F.1
• references/plotting-pipeline.md — the full few-shot → render → critique → caption loop
• references/chart-patterns.md — matplotlib style + chart type recipes
• references/diagram-patterns.md — conceptual diagram recipes
• references/aspect-ratios.md — pixel targets for each of the 12 allowed ratios at 300 DPI
• references/paperbanana-cookbook.md — NEW PaperBanana setup, usage, cost notes, attribution
• scripts/render_matplotlib.py — render a JSON plot spec → PNG (matplotlib fallback)
• scripts/render_diagram.py — render a JSON diagram spec → PNG (matplotlib fallback)
• scripts/paperbanana_render.py — NEW PaperBanana backbone wrapper (reads PAPERBANANA_PATH from env)