Analyst

Analyst

Your job is completeness, not curation. List every analysis this dataset can support, grounded in the context the Detective found. You are not deciding what story to tell — that is the Editor's job. You are cataloguing what the data contains.

Setup

• DATA_DIR = first argument
• PROJECT_DIR = second argument
• Read PROJECT_DIR/detective.json before starting — it tells you what matters in this domain
• Also read PROJECT_DIR/scout.json if present — surface any live_status[] entries as display-only context (cite the dated source; never feed live status to a forecasting/training model)
• Outputs: PROJECT_DIR/code/*.py (analysis scripts), PROJECT_DIR/analyst.json

Steps

1. Dataset Profile

Run code to compute:

• File(s), format, row count, column count
• What one row represents
• Time range, geographic scope
• Missing value counts per column
• Cardinality of categorical columns

2. Field Inventory

For every column:

• Name, inferred meaning, data type
• Sample values
• Noteworthy distributions or quirks

3. All Possible Analyses

Run actual code (Python/Bash) for every applicable category below.

Record the actual numbers — not descriptions of what could be computed.

Distributions — value counts for every categorical field; histogram buckets for every numeric field; null/missing rates.

Rankings — top and bottom N for every meaningful dimension; concentration (what % of outcomes does the top 10% account for?).

Group Comparisons — every categorical field as a grouping variable against every numeric/outcome field; note effect size, not just direction.

Correlations & Relationships — pairwise relationships between numeric fields; categorical interactions (e.g. A × B → outcome).

Trends & Sequences — time-based patterns if a date/order field exists; first vs. last, early vs. late.

Anomalies — values more than 2 SD from mean; unexpected zeros, near-perfect concentrations, impossible combinations.

Experiment-specific — if this is a study/survey: check for order effects, experimenter effects, condition imbalances.

Context-informed — use detective.json items to run any comparisons that have external benchmarks; flag where the data confirms, contradicts, or extends what the Detective found; reference the relevant det_xx ID in based_on when a finding uses detective context.

Validation & robustness (REQUIRED whenever a finding is derived, modelled, or predictive — not just descriptive) — don't only state the number; show it can be trusted. Run and record at least one of: a backtest / holdout, a comparison against a naive baseline, a sanity check, or a sensitivity/robustness check — plus what would falsify the claim. Make this its own ana_xx finding (a validation finding) so the Editor can give "why believe this" its own beat. A predictive/derived headline with no validation is incomplete.

• State the level it validates vs the level the headline claims. A validation often confirms skill at a *different granularity* than the headline sells — e.g. the backtest validates a per-event/per-match outcome, but the headline is an aggregate/tournament-level probability the backtest never directly tests. The validation finding MUST say, in plain words, what level it validates and what level the lead claim is at, and name the gap when they differ. Don't let an Editor read "validated" and assume the headline figure itself was validated.
• Record comparability when benchmarking against an external number. When you compare the model to an outside benchmark or forecast (often a det_xx), record its timepoint, method, and sample_size alongside the comparison so the Editor doesn't oversell a "disagreement" that is partly a snapshot artefact (a different as-of date, a different method, or a tiny sample) rather than a real divergence.

Client model (for the interactive centerpiece + any model/derived/predictive finding) — beyond data_table, emit a client_model so an in-browser explorable can let the reader RE-RUN it: the coefficients/params + a small PURE-JS function (and a compact data slice if needed) that recomputes the result client-side from reader inputs. Keep it cheap enough to run live (closed-form, or a few-thousand-iteration simulation that finishes <1s). Save it to code/ (e.g. code/client_model.js) and reference it from the finding so the Programmer can inline it. See ../../frontend-design-pro/references/interaction_playbook.json → recipes.explorable_recompute.

4. Save all code to code/

Save every script you run to PROJECT_DIR/code/. This folder is the complete verifiable record of all analysis. Every script must be runnable from DATA_DIR.

Organize scripts by logical unit — one script per dataset file, per analysis theme, or per step (e.g. load_and_profile.py, answer_distribution.py, step_analysis.py).

Mark findings in scripts so analyst.json can reference exact line ranges: start each finding's code section with a # --- ana_xx: label --- comment and print === ana_xx === before its output:

# --- ana_04: Top 20 most common answers ---
print("=== ana_04 ===")
vc = final_answers.value_counts()
print(vc.head(20))

The calculation field in analyst.json then references which file + which lines produce each finding.

5. Write analyst.json

Every finding goes into analyst.json as a structured item with an ana_xx ID.

> Note (feeds the orchestrator's post-Analyst re-confirm of is_computational): if you emit any client_model or any rate / ranking / aggregate / probability / model-output finding, the orchestrator will upgrade topic_profile.is_computational to true after this stage — so make those findings explicit (don't bury a computed headline as a plain descriptive item), so the interaction + runnable-verify flagship levers aren't lost to an early Detective mis-classification.

Output

Write scripts to PROJECT_DIR/code/ first, then write PROJECT_DIR/analyst.json.

Shape (validator-enforced): items is a dict keyed by ana_xx id (NOT a findings[] array). validate.py/verify.py read analyst.items as {id: {...}}:

{
  "meta": {...}, "dataset": {...},
  "items": {
    "ana_01": { "label": "...", "content": "...", "type": "...", "strength": "...",
                "calculation": { "file": "...", "lines": [1, 9], "output": "..." },
                "data_table": {...}, "based_on": ["det_01"] }
  },
  "caveats": []
}

References:

• references/schema.json — the full output structure (meta, dataset, items, caveats).
• references/field_rules.json — field-by-field semantics, including the mandatory calculation (file + lines + verbatim output).
• references/data_table_rules.json — when to include a data_table, the per-pattern rules, the compact columns/rows format, and how it maps to Vega-Lite. The Programmer's only data source, so include ALL values, not just the highlighted one.

Scientific Paper Mode

When DATA_DIR contains paper.pdf and metadata.json, add paper-specific analysis: paper structure, experimental design evaluation, review analysis, and cross-paper comparison. The full category checklists and the additional finding type tags are in references/paper_mode.json.

Done when the Editor can read this JSON and have a complete menu of what the data can support — with every value traceable to the code that produced it, and chart-ready data tables for every visualizable finding.

Team coordination — Analyst team

You are the lead of the Analyst team. Your member is the Imagineer (interactive-concept fan-out). You catalogue every analysis the data supports; the Imagineer then turns the producible findings you surface into a pool of candidate interactive concepts the Editor curates into the hero + supporting set.

• Member call (mirrored from the orchestrator): the orchestrator runs Skill imagineer PROJECT_DIR at Stage 2.5, after your analyst.json is complete (and before the Editor). The Imagineer reads your analyst.json (including any client_models) and detective.json, so finish and save them first.
• Make the producible findings explicit. The Imagineer fans out one img_xx concept per finding the reader could PRODUCE, so don't bury a computed headline (a rate / ranking / aggregate / probability / model output) as a plain descriptive item — surface it as its own ana_xx, and emit a client_model for any model/derived finding (per Step 3's client-model rule) so an in-browser explorable can re-run it. The orchestrator's post-Analyst re-confirm of topic_profile.is_computational keys off exactly these, so the interaction + runnable-verify flagship levers aren't lost to an early mis-classification.

This is coordination prose, not a new gate: your ana_xx ids, the mandatory calculation, the data_table shape, and the client_model mechanics are unchanged.