Dspy Finetune Bootstrap Skill for Claude
Use for BootstrapFinetune, fine-tuning DSPy models, teacher-student distillation, weight optimization, and lower-cost deployment.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/OmidZamani/dspy-skills --skill dspy-finetune-bootstrapWhat comes with it
460 bytes besides the instructionThe instruction itself
15 sections, as written by the authorDSPy BootstrapFinetune Optimizer
Goal
Distill a DSPy program into fine-tuned model weights for efficient production deployment.
When to Use
- You have a working DSPy program with a large model
- Need to reduce inference costs
- Want faster responses (smaller model)
- Deploying to resource-constrained environments
Inputs
| Input | Type | Description |
|-------|------|-------------|
| program | dspy.Module | Teacher program to distill |
| trainset | list[dspy.Example] | Training examples |
| metric | callable | Validation metric (optional) |
| train_kwargs | dict | Training hyperparameters |
Outputs
| Output | Type | Description |
|--------|------|-------------|
| finetuned_program | dspy.Module | Program with fine-tuned weights |
| model_path | str | Path to saved model |
Workflow
Phase 1: Prepare Teacher Program
import dspy
# Configure with strong teacher model
dspy.configure(lm=dspy.LM("openai/gpt-4o"))
class TeacherQA(dspy.Module):
def __init__(self):
self.cot = dspy.ChainOfThought("question -> answer")
def forward(self, question):
return self.cot(question=question)
Phase 2: Configure Fine-Tuning
Assign the LM directly to predictors before fine-tuning:
import dspy
from dspy.teleprompt import BootstrapFinetune
optimizer = BootstrapFinetune(
metric=lambda gold, pred, trace=None: gold.answer.lower() in pred.answer.lower(),
train_kwargs={
'learning_rate': 5e-5,
'num_train_epochs': 3,
'per_device_train_batch_size': 4,
'warmup_ratio': 0.1
}
)
Phase 3: Fine-tune Student Model
teacher = TeacherQA()
teacher.set_lm(dspy.settings.lm)
finetuned = optimizer.compile(teacher, trainset=trainset)
Phase 4: Deploy
# Save the fine-tuned model (saves state-only by default)
finetuned.save("finetuned_qa_model.json")
# Load and use (must recreate architecture first)
loaded = TeacherQA()
loaded.load("finetuned_qa_model.json")
result = loaded(question="What is machine learning?")
Production Example
import dspy
from dspy.teleprompt import BootstrapFinetune
from dspy.evaluate import Evaluate
import logging
import os
logger = logging.getLogger(__name__)
class ClassificationSignature(dspy.Signature):
"""Classify text into categories."""
text: str = dspy.InputField()
label: str = dspy.OutputField(desc="Category: positive, negative, neutral")
class TextClassifier(dspy.Module):
def __init__(self):
self.classify = dspy.Predict(ClassificationSignature)
def forward(self, text):
return self.classify(text=text)
def classification_metric(gold, pred, trace=None):
"""Exact label match."""
gold_label = gold.label.lower().strip()
pred_label = pred.label.lower().strip() if pred.label else ""
return gold_label == pred_label
def finetune_classifier(trainset, devset, output_dir="./finetuned_model"):
"""Full fine-tuning pipeline."""
# Configure teacher (strong model)
dspy.configure(lm=dspy.LM("openai/gpt-4o"))
teacher = TextClassifier()
teacher.set_lm(dspy.settings.lm)
# Evaluate teacher
evaluator = Evaluate(devset=devset, metric=classification_metric, num_threads=8)
teacher_score = evaluator(teacher)
logger.info(f"Teacher score: {teacher_score:.2%}")
# Fine-tune (train_kwargs passed to constructor)
optimizer = BootstrapFinetune(
metric=classification_metric,
train_kwargs={
'learning_rate': 2e-5,
'num_train_epochs': 3,
'per_device_train_batch_size': 8,
'gradient_accumulation_steps': 2,
'warmup_ratio': 0.1,
'weight_decay': 0.01,
'logging_steps': 10,
'save_strategy': 'epoch',
'output_dir': output_dir
}
)
finetuned = optimizer.compile(
teacher,
trainset=trainset
)
# Evaluate fine-tuned model
student_score = evaluator(finetuned)
logger.info(f"Student score: {student_score:.2%}")
# Save (state-only as JSON)
finetuned.save(os.path.join(output_dir, "final_model.json"))
return {
"teacher_score": teacher_score,
"student_score": student_score,
"model_path": os.path.join(output_dir, "final_model.json")
}
# For RAG fine-tuning
class RAGClassifier(dspy.Module):
"""RAG pipeline that can be fine-tuned."""
def __init__(self, num_passages=3):
self.retrieve = dspy.Retrieve(k=num_passages)
self.classify = dspy.ChainOfThought("context, text -> label")
def forward(self, text):
context = self.retrieve(text).passages
return self.classify(context=context, text=text)
def finetune_rag_classifier(trainset, devset):
"""Fine-tune a RAG-based classifier."""
# Configure retriever and LM
colbert = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')
dspy.configure(
lm=dspy.LM("openai/gpt-4o"),
rm=colbert
)
rag = RAGClassifier()
rag.set_lm(dspy.settings.lm)
# Fine-tune (train_kwargs in constructor)
optimizer = BootstrapFinetune(
metric=classification_metric,
train_kwargs={
'learning_rate': 1e-5,
'num_train_epochs': 5
}
)
finetuned = optimizer.compile(
rag,
trainset=trainset
)
return finetuned
Training Arguments Reference
| Argument | Description | Typical Value |
|----------|-------------|---------------|
| learning_rate | Learning rate | 1e-5 to 5e-5 |
| num_train_epochs | Training epochs | 3-5 |
| per_device_train_batch_size | Batch size | 4-16 |
| gradient_accumulation_steps | Gradient accumulation | 2-8 |
| warmup_ratio | Warmup proportion | 0.1 |
| weight_decay | L2 regularization | 0.01 |
| max_grad_norm | Gradient clipping | 1.0 |
Best Practices
- Strong teacher - Use GPT-4 or Claude as teacher
- Quality data - Teacher traces are only as good as training examples
- Validate improvement - Compare student to teacher on held-out set
- Start with more epochs - Fine-tuning often needs 3-5 epochs
- Monitor overfitting - Track validation loss during training
Limitations
- Requires a provider and model that support fine-tuning
- Training requires GPU resources
- Student may not match teacher quality on all inputs
- Fine-tuning takes hours/days depending on data size
- Model size reduction may cause capability loss
Official Documentation
- DSPy Documentation: https://dspy.ai/
- DSPy GitHub: https://github.com/stanfordnlp/dspy
- BootstrapFinetune API: https://dspy.ai/api/optimizers/BootstrapFinetune/
- Fine-tuning Guide: https://dspy.ai/tutorials/classification_finetuning/
Other skills for the same job
different authors, same section of the catalogue
Production-ready patterns for building LLM applications. Covers RAG pipelines, agent architectures, prompt IDEs, and LLMOps monitoring. Use when designing AI applications, implementing RAG, building agents, or setting up LLM observability.
Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.
World-class ML engineering skill for productionizing ML models, MLOps, and building scalable ML systems. Expertise in PyTorch, TensorFlow, model deployment, feature stores, model monitoring, and ML infrastructure. Includes LLM integration, fine-tuning, RAG systems, and agentic AI. Use when deploying ML models, building ML platforms, implementing MLOps, or integrating LLMs into production systems.
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
Use this skill for reinforcement learning tasks including training RL agents (PPO, SAC, DQN, TD3, DDPG, A2C, etc.), creating custom Gym environments, implementing callbacks for monitoring and control, using vectorized environments for parallel training, and integrating with deep RL workflows. This skill should be used when users request RL algorithm implementation, agent training, environment design, or RL experimentation.
Managed vector database for production AI applications. Fully managed, auto-scaling, with hybrid search (dense + sparse), metadata filtering, and namespaces. Low latency (<100ms p95). Use for production RAG, recommendation systems, or semantic search at scale. Best for serverless, managed infrastructure.
Deploy, evaluate, fine-tune, and manage Foundry agents end-to-end with azd: hosted agent scaffold/run/deploy, prompt agent create, batch eval, continuous eval, prompt optimizer, Agent Optimizer scaffold, agent.yaml, dataset curation from traces, model fine-tuning (SFT/DPO/RFT). USE FOR: azd ai agent, azd provision/deploy, deploy agent, hosted agent, create agent, add tool to agent, invoke agent, evaluate agent, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, agent optimizer, deploy model, Foundry project, RBAC, role assignment, permissions, quota, capacity, region, troubleshoot agent, deployment failure, AI Services, create Foundry resource, provision, knowledge index, customize deployment, onboard, availability, fine-tune, SFT, DPO, RFT, training-data, grader, distillation, fine-tuned model, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).
Cost optimization patterns for LLM API usage — model routing by task complexity, budget tracking, retry logic, and prompt caching.
How to use it
Take omidzamani/dspy-finetune-bootstrap from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.