Agentsop Llamaindex

LlamaIndex · SOP

> Third-person analytical view of how LlamaIndex *thinks* about turning private

> documents into a grounded answering system. The skill is for an LLM agent that

> writes / reviews / debugs RAG code — not for an end user reading docs.

何时激活 (Activation Rules)

Activate this skill when any of the following holds:

• The user's request involves building, modifying, or debugging a RAG pipeline (retrieval over private/unstructured data + LLM synthesis).
• The user mentions LlamaIndex (from llama_index...), LlamaParse, LlamaCloud, or a LlamaIndex-style primitive (VectorStoreIndex, SummaryIndex, IngestionPipeline, QueryEngine, SubQuestionQueryEngine, RouterQueryEngine, Settings, Workflows).
• The user is comparing RAG frameworks (LlamaIndex vs LangChain vs Haystack vs raw vector store).
• The user is choosing between stuffing context, RAG, or an agent for a knowledge task.
• The user is debugging retrieval quality (hallucinations, wrong chunks, stale data, embedding drift) — even if the codebase predates LlamaIndex, the failure-mode taxonomy applies.
• The user is evaluating a RAG system (faithfulness, relevancy, MRR, hit-rate).

Do not activate when:

• The task is pure agent orchestration with no retrieval (use LangGraph/CrewAI skill instead).
• The corpus is tiny (<100k tokens, static) and prompt-stuffing is the correct answer.
• The data is pure SQL/tabular with no unstructured component.

核心心智模型 (Core Mental Model)

LlamaIndex's design rests on three principles that distinguish it from "vector DB SDK + custom glue":

Principle 1 — The Index is a noun, not a verb

> In LangChain, "indexing" is something you do to a vector store. In LlamaIndex, an Index is a first-class typed object with its own retrieval semantics. Picking the right Index is half the architecture decision.

The 5-layer pipeline:

Documents → Nodes → Index → Retriever → Query Engine → Response
   ↓         ↓        ↓         ↓             ↓
parsing   chunking  storage   filters    synthesis
metadata  graph     primitive  rerank    (refine/tree_sum/compact)

Each layer has a distinct failure mode and a distinct optimization knob. See references/R1-architecture.md for the layer-failure-knob mapping.

Principle 2 — A Node is a graph node, not a chunk

A Node carries: text, metadata, embedding, relationships (PREV/NEXT/PARENT/CHILD links), and lifecycle ids. The relationships field is what enables Hierarchical, Auto-Merging, and Sentence-Window retrieval. The mental flip: don't think "split into chunks", think "build a chunk-graph".

Principle 3 — Indices are not interchangeable

| Index | Pick when |

|---|---|

| VectorStoreIndex | Default; semantic Q&A over chunks; ~90% of RAG cases |

| SummaryIndex | "Summarize this whole doc" — small, fan-out synthesis |

| TreeIndex | Hierarchical content with progressive zoom-in |

| KeywordTableIndex | Keyword-heavy queries, no embeddings budget |

| PropertyGraphIndex | Multi-hop reasoning over entities |

| DocumentSummaryIndex | Mixed corpora needing document-level routing first |

A RouterQueryEngine over multiple per-task indices is often the correct top-level shape, not a single monolithic VectorStoreIndex.

The 2025 shift

LlamaIndex now positions as "the leading document agent and OCR platform" (README). LlamaParse v2 + Workflows 1.0 (June 2025) + LlamaCloud mark a strategic move from "RAG framework" to "platform between messy documents and document-grounded agents". For a coder agent: assume Workflows for any new agentic code (QueryPipeline is deprecated).

SOP 工作流 (Agentic Protocol)

The protocol every RAG implementation must walk through. Each stage gates on the next.

Stage 0 — Frame the problem

Before code, answer:

• Is the corpus unstructured + non-trivial size (>100k tokens) + growing? If not → see R4 boundaries; LlamaIndex may be the wrong tool.
• Is retrieval quality the bottleneck (not orchestration)? If orchestration dominates → LangGraph leads, LlamaIndex becomes a retrieval tool *inside* it.
• What is the query distribution? (lookup-only / summary / compare-contrast / mixed). This decides whether a single Index or a Router is needed.

Stage 1 — Baseline (cheap, fast, observable)

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.node_parser import SentenceSplitter

Settings.llm        = OpenAI(model="gpt-4o-mini")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.node_parser = SentenceSplitter(chunk_size=1024, chunk_overlap=20)

docs  = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(docs)
qe    = index.as_query_engine(similarity_top_k=4)

Pin Settings once at app boot, never inline. This eliminates the entire embedding-mismatch failure class (failure #4).

Stage 2 — Build the eval loop before optimizing anything

from llama_index.core.evaluation import (
    DatasetGenerator, FaithfulnessEvaluator,
    RelevancyEvaluator, RetrieverEvaluator,
)
qa = DatasetGenerator.from_documents(docs).generate_dataset_from_nodes(num=50)

Track {MRR, hit-rate, faithfulness, relevancy, p95 latency}. Every subsequent change must be gated on these numbers.

> Most RAG failures in production trace to weak retrieval or sloppy ingestion — not the LLM. The eval loop is what surfaces them.

Stage 3 — Optimize in LlamaIndex's recommended order

From the official basic_strategies guide:

• Prompt engineering (cheapest)
• Embedding model (pick from MTEB; full re-embed if you change it)
• Chunk size sweep ({256, 512, 1024, 2048}; default 1024 for prose, 80-160 for code)
• Hybrid search (BM25 + dense) — *only if traffic contains lexical-identity queries*
• Metadata filters — for multi-tenant / multi-collection corpora
• Document/chunk decoupling — HierarchicalNodeParser+AutoMergingRetriever *or* SentenceWindowNodeParser
• Reranking (Cohere / SentenceTransformer / ColBERT) — widen top_k to 20-50, rerank to 3-5

Note the order: prompts first, reranking last. Reranking is high-impact but expensive — exhaust cheap knobs first.

Stage 4 — Compose for query heterogeneity

| Query shape | Right primitive |

|---|---|

| "Summarize doc X" | SummaryIndex per doc, routed |

| "Find the clause about X" | VectorStoreIndex + metadata filters |

| "Compare X and Y across docs" | SubQuestionQueryEngine |

| "What entities relate to X?" | PropertyGraphIndex |

| Mixed | RouterQueryEngine over per-task engines |

Stage 5 — Production hardening

Apply the failure-mode checklist (R4). Top 5 non-negotiables:

• IngestionPipeline with docstore + UPSERTS_AND_DELETE for any live corpus.
• Settings.embed_model pinned at boot; embedding model name in index metadata.
• tree_summarize synthesizer when packing many chunks (mitigates lost-in-the-middle).
• Tracing/observability captures query + retrieved_nodes + scores + index_id + LLM prompt for every failure.
• Indices versioned as deployment artifacts; ingestion completes before traffic routing.

Stage 6 — Escalate to Workflows / Agents (only when justified)

Escalate when at least one of:

• A retrieval loop is needed ("retrieve → check → re-query").
• Tool calls beyond retrieval (calculator, web, code-exec).
• State surviving across query turns.
• Multiple specialized retrievers chosen at runtime.

Use Workflows 1.0 (event-driven), not deprecated QueryPipeline. Wrap query engines as QueryEngineTools and tune the description= carefully — it is the only signal the router/agent reads.

操作模型 (Operation Models)

Each operation: Trigger / Action / Output / Evidence.

OP-01 BaselineVectorIndex

• Trigger: First-pass RAG over a new corpus; retrieval-quality baseline unknown.
• Action: VectorStoreIndex.from_documents() with SentenceSplitter(1024, 20), top_k=4, default synthesizer. Ship to eval bench *before* tuning.
• Output: Working RAG endpoint + baseline {MRR, hit-rate, faithfulness, relevancy, p95}.
• Evidence: developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/

OP-02 TuneChunkSize

• Trigger: Faithfulness below target OR retrieved chunks visibly truncated/incomplete.
• Action: Sweep chunk_size ∈ {256, 512, 1024, 2048} with overlap at ~10-20%; re-evaluate faithfulness + relevancy + latency. Default land: 1024 for prose, 80-160 for code.
• Output: Optimal chunk_size pinned + embedding model version locked in index metadata.
• Evidence: llamaindex.ai/blog/evaluating-the-ideal-chunk-size-for-a-rag-system-using-llamaindex-6207e5d3fec5 (faithfulness peaked at 1024 in LlamaIndex's own eval on Uber 10-K).

OP-03 AddReranker

• Trigger: Top-1 wrong but relevant docs appear in top-k (failure #1 / #10).
• Action: Add CohereRerank or SentenceTransformerRerank as a NodePostprocessor; widen retrieval top_k to 20-50, narrow to top_n=3-5 after rerank.
• Output: Faithfulness lift typically 5-15pp on noisy corpora; lower context-window pressure.
• Evidence: developers.llamaindex.ai/python/framework/optimizing/rag_failure_mode_checklist/ (#1, #10).

OP-04 AddHybridBM25

• Trigger: Traffic contains exact identifiers, error codes, SKUs, code symbols, rare jargon — pure dense silently misses them.
• Action: QueryFusionRetriever([vector_retriever, BM25Retriever]) *or* vendor hybrid (Qdrant/Milvus alpha). Tune alpha per query type, not globally.
• Output: Recall lift on lexical-identity queries with no degradation on semantic queries.
• Evidence: llamaindex.ai/blog/llamaindex-enhancing-retrieval-performance-with-alpha-tuning-in-hybrid-search-in-rag-135d0c9b8a00; BM25Retriever docs.

OP-05 DecoupleChunkScope

• Trigger: Chunk-size sweep produces no single winner (small wins precision, large wins context).
• Action: HierarchicalNodeParser + AutoMergingRetriever (for structured docs) *or* SentenceWindowNodeParser + MetadataReplacementPostProcessor (for flat prose). Embed small, return large.
• Output: Precision-recall pareto improvement; LLM gets surrounding context that small chunks alone lost.
• Evidence: AutoMergingRetriever / Hierarchical / SentenceWindow docs on developers.llamaindex.ai.

OP-06 RouteByQueryType

• Trigger: Corpus serves heterogeneous tasks (summary / lookup / compare) from one entry point.
• Action: Build per-task QueryEngines (SummaryIndex for digest, VectorStoreIndex for lookup, SubQuestionQueryEngine for compare) + a RouterQueryEngine with LLM or Pydantic selector. Carefully author each QueryEngineTool.description.
• Output: Each query lands on the structurally-correct retrieval primitive; latency stays bounded.
• Evidence: DeepLearning.AI *Building Agentic RAG with LlamaIndex*; router docs.

OP-07 DecomposeMultiHop

• Trigger: Compare/contrast queries; queries needing facts from >1 document; "what changed between X and Y?".
• Action: SubQuestionQueryEngine decomposes query → dispatches sub-questions to sub-engines → synthesizes.
• Output: Multi-hop answers a single retrieval cannot assemble.
• Evidence: developers.llamaindex.ai sub-question query engine docs.

OP-08 IngestionWithDocstore

• Trigger: Documents will update/delete over time (any production system).
• Action: IngestionPipeline(transformations=..., docstore=..., vector_store=..., docstore_strategy=UPSERTS_AND_DELETE). Run on a schedule, not manually.
• Output: Idempotent re-ingestion; no duplicate vectors; deletes propagate.
• Evidence: developers.llamaindex.ai/python/framework/module_guides/loading/ingestion_pipeline/; failure #3.

OP-09 MetadataFilters

• Trigger: Multi-tenant corpus; cross-contamination between sub-collections; access control needed.
• Action: Inject structured metadata at ingestion (tenant, doc_type, date); apply MetadataFilters at query time OR enable auto-retrieval to let an LLM emit filters.
• Output: Hard isolation between tenants; targeted retrieval without expensive rerank.
• Evidence: Failure #7; basic_strategies metadata filters section.

OP-10 EvalLoop

• Trigger: Any non-trivial RAG, pre-deploy AND continuously in production.
• Action: DatasetGenerator → labeled QA pairs; run FaithfulnessEvaluator + RelevancyEvaluator + RetrieverEvaluator(["mrr","hit_rate"]). Gate every change.
• Output: Quantitative regression test for every chunking / embedding / retriever / prompt change.
• Evidence: developers.llamaindex.ai/python/framework-api-reference/evaluation/; cookbook.openai.com/examples/evaluation/evaluate_rag_with_llamaindex.

OP-11 LockGlobalSettings

• Trigger: Multiple modules each instantiate LLM/embed independently — drift risk.
• Action: Set Settings.llm and Settings.embed_model once in app bootstrap. Forbid inline overrides in PR review.
• Output: Eliminates failure #4 (config drift) and #5 (embedding mismatch).
• Evidence: docs.llamaindex.ai/en/stable/module_guides/supporting_modules/service_context_migration/.

OP-12 AgenticWorkflow

• Trigger: Need loops, tool calls beyond retrieval, multi-step reasoning, or state across turns.
• Action: Build a Workflows 1.0 event-driven workflow OR a FunctionAgent/ReActAgent with QueryEngineTools. Do NOT use the deprecated QueryPipeline.
• Output: Cycle-capable agentic system with retrieval as one tool among many.
• Evidence: llamaindex.ai/blog/announcing-workflows-1-0-a-lightweight-framework-for-agentic-systems.

困境决策案例 (Dilemma Cases)

(Full text in references/R3-dilemma-cases.md. Summarized here.)

Dilemma 1 — Chunk size: precision vs context

困境: Small chunks → precise embeddings, fragmented context for the LLM. Large chunks → rich context, embeddings become "topic averages", recall on specific queries drops. Failure modes #2 and #6 are the two poles.

约束: Embedding model has a fixed input window; metadata is propagated into payload (so very small chunks become all-metadata — GitHub #12200, #13792); token budget caps how many chunks fit downstream.

决策步骤:

• Generate ~20 eval QA pairs.
• Sweep chunk_size ∈ {128, 256, 512, 1024, 2048} with overlap = 10-20%.
• Build a VectorStoreIndex per config; record faithfulness, relevancy, latency.
• If a single winner emerges → pin it.
• If the frontier is non-flat → do not compromise; switch to small-embed/large-return via Hierarchical+AutoMerging *or* SentenceWindow.

结果: LlamaIndex's own published study (Uber 10-K) peaked at 1024 on both faithfulness and relevancy → 1024 became the framework default for prose. For code: 80-160 tokens. When the eval doesn't converge, decoupling wins; never average two bad chunk_sizes.

可提取的操作: OP-02 TuneChunkSize, OP-05 DecoupleChunkScope. Anti-pattern A1.

Dilemma 2 — Hybrid (BM25+dense) vs pure dense

困境: Adding hybrid doubles index footprint, requires per-query-type alpha tuning, complicates the pipeline. Worth it?

约束: Dense embeddings *silently fail* on identifiers, error strings, code, SKUs — they "destroy lexical identity by pooling token representations" (TianPan, 2026). BM25 scores against an inverted token index.

决策步骤:

• Build a query-type taxonomy from real traffic: semantic / lexical / mixed.
• If lexical share <5% → dense-only.
• 5-50% → add hybrid; tune alpha per query type.
• >50% (legal, code, logs) → invert: BM25-first, dense as reranker signal.
• Evaluate alpha at {0, 0.25, 0.5, 0.75, 1.0} on labeled subsets.

结果: Hybrid lifts the lexical slice without hurting the semantic slice — *if alpha is tuned per type*. A single global alpha often underperforms dense, which is why some teams wrongly conclude "hybrid didn't help".

可提取的操作: OP-04 AddHybridBM25. Decision is traffic-driven, not theoretical.

Dilemma 3 — Agent on top of RAG, RAG as tool, or just a Router?

困境: User adds compare/summary/lookup queries to a basic RAG. Three options:

• A. RouterQueryEngine over per-task engines.
• B. FunctionAgent/ReActAgent with engines as tools.
• C. SubQuestionQueryEngine to decompose.

约束: Agents add ≥1 LLM round-trip per step (latency); introduce planning errors a router cannot make; harder to debug (failure #12); most queries aren't multi-hop in practice.

决策步骤:

• Measure: what fraction of queries actually need multi-step reasoning?
• <20% multi-step + heterogeneous-but-single-step → Router (A).
• Compositional/well-shaped queries ("compare X and Y") → SubQuestion (C).
• Tool calls beyond retrieval, or cycles, or state → Agent on Workflows (B).
• Whichever you pick: invest in QueryEngineTool.description — it's the only signal the router/agent sees.

结果: DeepLearning.AI's official course ladder is Router → Agent. Production guidance consistently warns against premature agentization. Workflows 1.0 (2025) signals: when you need agency, use the agentic primitive, don't fake it with DAG pipelines.

可提取的操作: OP-06 RouteByQueryType, OP-07 DecomposeMultiHop, OP-12 AgenticWorkflow. Anti-pattern A9.

Dilemma 4 — Long-context LLM (1M tokens) vs RAG

困境: Does a 1M-token context window eliminate the need for RAG?

约束 (from llamaindex.ai/blog/towards-long-context-rag): 1M tokens ~60s latency + $0.50-$20/query; 10M tokens still doesn't cover large corpora; "lost in the middle" degrades quality by ~30%.

决策步骤:

• Corpus >1M tokens → RAG mandatory.
• p50 latency budget <5s → cannot afford full-context stuffing.
• Per-query cost ceiling <$0.05 → same.
• Apply LlamaIndex's three long-context patterns: Small-to-Big, Intelligent Routing, Retrieval-Augmented KV Caching.

结果: Long context does not replace RAG; it changes what RAG looks like. The bottleneck shifts from "fitting context" to "feeding right context in the right position" — making rerank + position-aware synthesis (tree_summarize) more important, not less.

可提取的操作: For any corpus >500k tokens or latency <5s: keep RAG. Use long-context as synthesis-stage capacity.

Dilemma 5 — Sentence-Window vs Auto-Merging

困境: Both implement "embed small, return large". Not interchangeable.

决策步骤:

• Docs have clear hierarchy (sections/headings) → Auto-Merging.
• Docs are flat prose → Sentence-Window.
• Queries are bursty multi-chunk → Auto-Merging escalates correctly.
• Queries are point-fact with surrounding context → Sentence-Window.

结果: Both beat naive top-k on faithfulness. Match parser/retriever pair to document structure, not theoretical elegance. Always pair SentenceWindowNodeParser with MetadataReplacementPostProcessor.

反模式与边界 (Anti-patterns & Boundaries)

Top 10 anti-patterns (full list in references/R4-anti-patterns.md)

| # | Anti-pattern | Correct move |

|---|---|---|

| A1 | Bump chunk_size when answers feel incomplete | Decouple embed-scope from synthesis-scope (Hierarchical / SentenceWindow) |

| A2 | Swap embedding model without re-embed | Rebuild index; tag artifact with embed model name+version |

| A3 | No eval loop; debug by anecdote | Stand up RetrieverEvaluator + FaithfulnessEvaluator + RelevancyEvaluator first |

| A4 | ServiceContext + manual config in every module | Pin Settings.llm and Settings.embed_model once at boot |

| A5 | QueryPipeline DAG for agentic logic | Use Workflows 1.0 (event-driven, supports cycles) |

| A6 | Naive top_k=N, no reranker | Widen top_k + add CohereRerank / SentenceTransformerRerank |

| A7 | Metadata not propagated to chunks; or metadata > 50% of chunk_size | Design metadata schema before ingestion; budget metadata tokens |

| A8 | Multi-modal RAG by base64-stuffing images into text | Use LlamaParse + multi-modal retrieval primitives |

| A9 | Wrap retrieval in a custom agent when a Router suffices | Default to RouterQueryEngine; escalate to Agent only with justification |

| A10 | Ingest once at deploy, never reconcile | IngestionPipeline + docstore + UPSERTS_AND_DELETE |

Boundaries — when not to use LlamaIndex

• B1: Tiny static corpus (<100k tokens) → prompt-stuff with caching.
• B2: Pure structured/tabular data → DuckDB/SQL/BI. (LlamaIndex only when NL2SQL+RAG hybrid.)
• B3: Hard real-time / sub-100ms retrieval → raw vector store SDK, not a RAG framework.
• B4: Complex multi-agent orchestration → LangGraph or CrewAI leads; embed LlamaIndex retrievers as tools.
• B5: Highly specialized parsing requirements + team has engineering budget → custom stack (Unstructured.io + pgvector + custom retriever) gives more control.

PR-review smells (instant red flags)

• from llama_index import ServiceContext → A4.
• index.as_query_engine(similarity_top_k=20) without a rerank postprocessor → A6.
• SentenceSplitter(chunk_size=4096) → likely A1.
• Settings.embed_model = ... in >1 file → A4 drift.
• IngestionPipeline(...) without docstore= → A10.
• A Workflow with no events or loops → over-engineered; should be a QueryEngine.
• An agent with a single retrieval tool → A9; should be a QueryEngine or RouterQueryEngine.

生态对照 (Ecosystem Context)

Decision rubric

Q1. Primarily extracting from messy documents (PDFs, slides, tables, scans)?
   YES → LlamaIndex (+ LlamaParse) leads.
Q2. Primary challenge is multi-step agentic orchestration with many non-retrieval tools?
   YES → LangGraph / CrewAI leads; use LlamaIndex retrievers as tools.
Q3. Corpus small (<100k tokens) and static?
   YES → No framework; prompt-stuff with caching.
Q4. Pure structured/tabular data?
   YES → SQL/DuckDB/BI. Use LlamaIndex only for hybrid NL2SQL+RAG.
DEFAULT → LlamaIndex remains lead; layer LangGraph only if agentic logic emerges.

Head-to-head highlights

| Vs | LlamaIndex wins when | Other wins when |

|---|---|---|

| LangChain | Retrieval quality and ingestion are the bottleneck; document-heavy | Orchestration is complex; many non-retrieval tools |

| Haystack | Modern LLM-centric docs; multi-modal; broader index taxonomy | YAML-configurable pipelines; classical IR feel |

| Raw vector store | Need >2 of {SentenceSplitter, IngestionPipeline, Reranker, Eval, Synthesizer} | Truly minimal RAG; team wants no framework |

| DSPy | Want structured retrieval infrastructure | Want automatic prompt optimization |

| LangGraph (for agents) | Retrieval-heavy with light agency (Workflows ergonomic here) | Many states, complex multi-agent state machines |

| CrewAI / AutoGen | (different category) | Multi-agent collaboration is the goal |

The normative hybrid (2025-2026)

> Most production teams converge on: LlamaIndex for retrieval & ingestion; LangGraph (or LlamaIndex Workflows) for orchestration; LangSmith / Phoenix for observability.

References

• references/R1-architecture.md — 5-layer model deep dive, Index taxonomy, Settings/Workflows
• references/R2-sop-workflow.md — full 8-stage RAG bootstrap protocol
• references/R3-dilemma-cases.md — 5 dilemma cases in full
• references/R4-anti-patterns.md — 13 official failure modes + 10 anti-patterns + boundaries
• references/R5-ecosystem-context.md — comparison matrix, hybrid patterns
• intermediate/operation_candidates.json — machine-readable operation list

Primary sources (cited inline above)

• developers.llamaindex.ai/python/framework/ (architecture homepage)
• developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/
• developers.llamaindex.ai/python/framework/optimizing/rag_failure_mode_checklist/ (official 13 failure modes)
• developers.llamaindex.ai/python/framework/module_guides/indexing/index_guide/
• developers.llamaindex.ai/python/framework/module_guides/loading/ingestion_pipeline/
• llamaindex.ai/blog/evaluating-the-ideal-chunk-size-for-a-rag-system-using-llamaindex-6207e5d3fec5
• llamaindex.ai/blog/llamaindex-enhancing-retrieval-performance-with-alpha-tuning-in-hybrid-search-in-rag-135d0c9b8a00
• llamaindex.ai/blog/towards-long-context-rag
• llamaindex.ai/blog/announcing-workflows-1-0-a-lightweight-framework-for-agentic-systems
• docs.llamaindex.ai/en/stable/module_guides/supporting_modules/service_context_migration/
• github.com/run-llama/llama_index (README, issues #12200, #13792, #6465)
• cookbook.openai.com/examples/evaluation/evaluate_rag_with_llamaindex
• learn.deeplearning.ai/courses/building-agentic-rag-with-llamaindex/
• ibm.com/think/topics/llamaindex-vs-langchain
• statsig.com/perspectives/llamaindex-rag-retrieval
• tianpan.co/blog/2026-04-12-hybrid-search-production-bm25-dense-embeddings