Dspy Embedding Retrieval Agent Skill

Use for DSPy retrieval with dspy.Embedder, dspy.Embeddings, FAISS indexes, semantic search, and local or hosted embedding models.

821 tokens

context cost

the whole folder, loaded on every use

files

ships runnable scripts

copies elsewhere

how many repositories repackaged it

119

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/OmidZamani/dspy-skills --skill dspy-embedding-retrieval

What comes with it

408 bytes besides the instruction

example.py

The instruction itself

9 sections, as written by the author

DSPy Embedding Retrieval

Goal

Build semantic retrieval over an application-owned text corpus with dspy.Embedder and dspy.Embeddings.

Basic Hosted Embedder

import dspy

corpus = [
    "DSPy programs are composed from modules.",
    "MIPROv2 optimizes instructions and demonstrations.",
    "RLM explores large contexts with a sandboxed REPL.",
]

embedder = dspy.Embedder("openai/text-embedding-3-small")
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=2)

result = search("Which optimizer tunes prompts?")
print(result.passages)
print(result.indices)

Use in RAG

class LocalRAG(dspy.Module):
    def __init__(self, retriever):
        super().__init__()
        self.retriever = retriever
        self.answer = dspy.ChainOfThought("context: list[str], question -> answer")

    def forward(self, question: str):
        context = self.retriever(question).passages
        return self.answer(context=context, question=question)

Custom Local Embeddings

Wrap any callable that accepts list[str] and returns a 2D numeric array:

from sentence_transformers import SentenceTransformer
import dspy

model = SentenceTransformer("sentence-transformers/static-retrieval-mrl-en-v1")
embedder = dspy.Embedder(model.encode)
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=5)

Scores, FAISS, and Persistence

Use dspy.EmbeddingsWithScores when downstream logic needs similarity thresholds or reranking.

For corpora at or above the brute_force_threshold default of 20_000, DSPy builds a FAISS index. Install FAISS first:

pip install faiss-cpu

Persist the index when embedding the corpus is expensive:

search.save("./retrieval-index")
loaded = dspy.Embeddings.from_saved("./retrieval-index", embedder=embedder)

Build a complete pipeline: dspy-rag-pipeline
Design typed context fields: dspy-signature-designer
Harden caches: dspy-production-deployment

Best Practices

Evaluate retrieval quality separately from answer quality.
Keep corpus chunking deterministic and versioned.
Persist expensive indexes.
Use EmbeddingsWithScores when debugging relevance.
Measure memory and latency before enabling FAISS for large corpora.

Official Documentation

Embedder API: https://dspy.ai/api/models/Embedder/
Embeddings API: https://dspy.ai/api/tools/Embeddings/

Other skills for the same job

different authors, same section of the catalogue

Skill Creator

by anthropics

vendor ×10

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Access NCBI GEO for gene expression/genomics data. Search/download microarray and RNA-seq datasets (GSE, GSM, GPL), retrieve SOFT/Matrix files, for transcriptomics and expression analysis.

12k tokens

Pymc Bayesian Modeling

by christophacham

×4

Bayesian modeling with PyMC. Build hierarchical models, MCMC (NUTS), variational inference, LOO/WAIC comparison, posterior checks, for probabilistic programming and inference.

Multi-objective optimization framework. NSGA-II, NSGA-III, MOEA/D, Pareto fronts, constraint handling, benchmarks (ZDT, DTLZ), for engineering design and optimization problems.

Statistical modeling toolkit. OLS, GLM, logistic, ARIMA, time series, hypothesis tests, diagnostics, AIC/BIC, for rigorous statistical inference and econometric analysis.

Add unsigned integer (uint) type support to PyTorch operators by updating AT_DISPATCH macros. Use when adding support for uint16, uint32, uint64 types to operators, kernels, or when user mentions enabling unsigned types, barebones unsigned types, or uint support.

Convert PyTorch AT_DISPATCH macros to AT_DISPATCH_V2 format in ATen C++ code. Use when porting AT_DISPATCH_ALL_TYPES_AND*, AT_DISPATCH_FLOATING_TYPES*, or other dispatch macros to the new v2 API. For ATen kernel files, CUDA kernels, and native operator implementations.

Write docstrings for PyTorch functions and methods following PyTorch conventions. Use when writing or updating docstrings in PyTorch code.

3k tokens