mcpbeat

Dspy Embedding Retrieval

omidzamani/dspy-embedding-retrieval

Use for DSPy retrieval with dspy.Embedder, dspy.Embeddings, FAISS indexes, semantic search, and local or hosted embedding models.

821 tokens
context cost
the whole folder, loaded on every use
2
files
ships runnable scripts
0
copies elsewhere
how many repositories repackaged it
119
stars on the repo
on the repository, not the skill itself

Install

one command, takes just this skill from the repository 
npx skills add https://github.com/OmidZamani/dspy-skills --skill dspy-embedding-retrieval

What comes with it

408 bytes besides the instruction
example.py

The instruction itself

9 sections, as written by the author

DSPy Embedding Retrieval

Goal

Build semantic retrieval over an application-owned text corpus with dspy.Embedder and dspy.Embeddings.

Basic Hosted Embedder

import dspy

corpus = [
    "DSPy programs are composed from modules.",
    "MIPROv2 optimizes instructions and demonstrations.",
    "RLM explores large contexts with a sandboxed REPL.",
]

embedder = dspy.Embedder("openai/text-embedding-3-small")
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=2)

result = search("Which optimizer tunes prompts?")
print(result.passages)
print(result.indices)

Use in RAG

class LocalRAG(dspy.Module):
    def __init__(self, retriever):
        super().__init__()
        self.retriever = retriever
        self.answer = dspy.ChainOfThought("context: list[str], question -> answer")

    def forward(self, question: str):
        context = self.retriever(question).passages
        return self.answer(context=context, question=question)

Custom Local Embeddings

Wrap any callable that accepts list[str] and returns a 2D numeric array:

from sentence_transformers import SentenceTransformer
import dspy

model = SentenceTransformer("sentence-transformers/static-retrieval-mrl-en-v1")
embedder = dspy.Embedder(model.encode)
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=5)

Scores, FAISS, and Persistence

Use dspy.EmbeddingsWithScores when downstream logic needs similarity thresholds or reranking.

For corpora at or above the brute_force_threshold default of 20_000, DSPy builds a FAISS index. Install FAISS first:

pip install faiss-cpu

Persist the index when embedding the corpus is expensive:

search.save("./retrieval-index")
loaded = dspy.Embeddings.from_saved("./retrieval-index", embedder=embedder)

Related Skills

  • Build a complete pipeline: dspy-rag-pipeline
  • Design typed context fields: dspy-signature-designer
  • Harden caches: dspy-production-deployment

Best Practices

  • Evaluate retrieval quality separately from answer quality.
  • Keep corpus chunking deterministic and versioned.
  • Persist expensive indexes.
  • Use EmbeddingsWithScores when debugging relevance.
  • Measure memory and latency before enabling FAISS for large corpora.

Official Documentation

  • Embedder API: https://dspy.ai/api/models/Embedder/
  • Embeddings API: https://dspy.ai/api/tools/Embeddings/

How to use it

Copy the folder

Take omidzamani/dspy-embedding-retrieval from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.

Install what it needs

The instructions reference pip. Without those the skill loads but fails at the first command.