Experiment Code Skill for Claude
Write ML experiment code with iterative improvement. Generate training/evaluation pipelines, debug errors, and optimize results through code reflection. Use when implementing experiments for a research paper.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/lingzhi227/agent-research-skills --skill experiment-codeThe instruction itself
11 sections, as written by the authorExperiment Code
Generate and iteratively improve ML experiment code for research papers.
Input
$0— Task:generate,improve,debug,plot$1— Research plan, idea description, or error message
References
- Experiment prompts and patterns:
~/.claude/skills/experiment-code/references/experiment-prompts.md - Code patterns (error handling, repair, hill-climbing):
~/.claude/skills/experiment-code/references/code-patterns.md
Action: generate
Generate initial experiment code following this structure:
- Plan experiments first — List all runs needed (hyperparameter sweeps, ablations, baselines)
- Write self-contained code — All code in project directory, no external imports from reference repos
- Include proper logging — Save results to JSON, print intermediate metrics
- Generate figures — At minimum Figure_1.png and Figure_2.png
Mandatory Structure
project/
├── experiment.py # Main experiment script
├── plot.py # Visualization script
├── notes.txt # Experiment descriptions and results
├── run_1/ # Results from run 1
│ └── final_info.json
├── run_2/
└── ...
Constraints
- No placeholder code (
pass,...,raise NotImplementedError) - Must use actual datasets (not toy data unless explicitly requested)
- PyTorch or scikit-learn preferred (no TensorFlow/Keras)
- Each run uses:
python experiment.py --out_dir=run_i
Action: improve
Improve existing experiment code:
- Read current code and results
- Reflect on what worked and what didn't
- Apply targeted edits (prefer small edits over full rewrites)
- Re-run and compare scores
- Keep the best-performing code variant
Action: debug
Fix experiment code errors:
- Read the error message (truncate to last 1500 chars if very long)
- Identify the root cause
- Apply minimal fix
- Up to 4 retry attempts before changing approach
Action: plot
Generate publication-quality plots from experiment results:
- Read all
run_*/final_info.jsonfiles - Generate comparison plots with proper labels
- Use the figure-generation skill for styling
Rules
- Always plan experiments before writing code
- After each run, document results in notes.txt
- Include print statements explaining what results show
- Method MUST not get 0% accuracy — verify accuracy calculations
- Use seeds for reproducibility
- Before each experiment include a print statement explaining exactly what the results are meant to show
Related Skills
- Upstream: experiment-design, algorithm-design
- Downstream: data-analysis, backward-traceability
- See also: code-debugging, paper-to-code
Other skills for the same job
different authors, same section of the catalogue
Comprehensive toolkit for creating, analyzing, and visualizing complex networks and graphs in Python. Use when working with network/graph data structures, analyzing relationships between entities, computing graph algorithms (shortest paths, centrality, clustering), detecting communities, generating synthetic networks, or visualizing network topologies. Applicable to social networks, biological networks, transportation systems, citation networks, and any domain involving pairwise relationships.
Interact with Zotero reference management libraries using the pyzotero Python client. Retrieve, create, update, and delete items, collections, tags, and attachments via the Zotero Web API v3. Use this skill when working with Zotero libraries programmatically, managing bibliographic references, exporting citations, searching library contents, uploading PDF attachments, or building research automation workflows that integrate with Zotero.
Comprehensive toolkit for creating, analyzing, and visualizing complex networks and graphs in Python. Use when working with network/graph data structures, analyzing relationships between entities, computing graph algorithms (shortest paths, centrality, clustering), detecting communities, generating synthetic networks, or visualizing network topologies. Applicable to social networks, biological networks, transportation systems, citation networks, and any domain involving pairwise relationships.
Create, analyze, and visualize complex networks and graphs in Python with NetworkX. Use when working with network/graph data structures, computing graph algorithms (shortest paths, centrality, clustering), detecting communities, generating synthetic networks (random, scale-free, small-world), reading/writing graph file formats, or drawing network topologies. Common applications include social, biological, transportation, and citation networks.
Use NeuroKit2 to build or audit reproducible research workflows for physiological time-series preprocessing, event/interval analysis, multimodal alignment, variability, and complexity. Trigger when code imports neurokit2 or needs its current APIs, schemas, and method-aware validation—not for diagnosis or device validation.
Interact with Zotero reference management libraries using the pyzotero Python client. Retrieve, create, update, and delete items, collections, tags, and attachments via the Zotero Web API v3. Use this skill when working with Zotero libraries programmatically, managing bibliographic references, exporting citations, searching library contents, uploading PDF attachments, or building research automation workflows that integrate with Zotero.
Comprehensive toolkit for creating, analyzing, and visualizing complex networks and graphs in Python. Use when working with network/graph data structures, analyzing relationships between entities, computing graph algorithms (shortest paths, centrality, clustering), detecting communities, generating synthetic networks, or visualizing network topologies. Applicable to social networks, biological networks, transportation systems, citation networks, and any domain involving pairwise relationships.
A practical, jargon-free guide to fp-ts functional programming - the 80/20 approach that gets results without the academic overhead. Use when writing TypeScript with fp-ts library.
How to use it
Take lingzhi227/experiment-code from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.