Dspy Optimizer Selection Agent Skill

DSPy Optimizer Selection

Goal

Choose the smallest DSPy optimizer that matches the data, budget, and artifact being tuned. Establish a baseline before compiling anything.

Selection Matrix

| Need | Start with | Notes |

|------|------------|-------|

| Include a few labeled examples | dspy.LabeledFewShot | Random labeled demos; useful as a baseline |

| About 10 examples | dspy.BootstrapFewShot | Teacher-generated demos with metric filtering |

| 50+ examples and stronger demo search | dspy.BootstrapFewShotWithRandomSearch | Searches multiple demo sets; alias: dspy.BootstrapRS |

| Per-input nearest demos | dspy.KNNFewShot | Retrieves nearby examples before bootstrapping |

| Instruction-only hill climbing | dspy.COPRO | Coordinate ascent over instructions |

| Instruction and demo search | dspy.MIPROv2 | Bayesian search; install dspy[optuna] |

| Mini-batch introspective rules or demos | dspy.SIMBA | Uses output variability and self-reflection |

| Rich textual feedback and trace reflection | dspy.GEPA | Metric must accept five arguments |

| Distill prompts into model weights | dspy.BootstrapFinetune | Requires a fine-tunable LM and set_lm() |

| Combine candidate programs | dspy.Ensemble | Trades inference cost for robustness |

| Sequence prompt and weight optimization | dspy.BetterTogether | Meta-optimizer for configurable optimizer chains |

Workflow

• Split data into train and validation sets.
• Evaluate the uncompiled program with dspy-evaluation-suite.
• Start with the least expensive optimizer that matches the need.
• Save the compiled program and compare it against the baseline.
• Escalate only when the measured gain justifies extra LM calls, fine-tuning, or inference cost.

Common Paths

Fast Demo Optimization

Use dspy-bootstrap-fewshot for the first optimization pass. Move to BootstrapFewShotWithRandomSearch when enough examples are available to search multiple demo sets.

Prompt Search

Use dspy-miprov2-optimizer for instruction and demonstration search. Install its optional dependency first:

pip install -U "dspy[optuna]>=3.2.1,<3.3"

Reflective Optimization

Use dspy-gepa-reflective when failures can be described with actionable text. Use dspy-simba-optimizer for a smaller mini-batch introspective loop with numeric metrics.

Prompt Plus Weight Optimization

Use dspy-better-together when a fine-tunable LM is available and prompt optimization alone has plateaued.

Best Practices

• Keep a held-out validation set.
• Track optimization cost and inference cost separately.
• Use reproducible seeds where supported.
• Avoid claiming one optimizer is universally best; compare measured results.
• Save intermediate candidates for expensive runs.

Official Documentation

• Optimizer guide: https://dspy.ai/learn/optimization/optimizers/
• Optimizer API index: https://dspy.ai/api/optimizers/
• DSPy releases: https://github.com/stanfordnlp/dspy/releases