RAG Blueprint

NVIDIA RAG Blueprint

Purpose

Use this skill for NVIDIA RAG Blueprint operations: deployment, configuration,

troubleshooting, shutdown, and feature management across Docker, Helm, and

library deployments.

Instructions

• Match the user request to the intent routing table below.
• Read the referenced playbook before making changes.
• Use repository docs and deployment config files as the source of truth.
• Verify the affected service or workflow after changes.

Prerequisites

• NVIDIA RAG Blueprint repository checkout.
• Docker/Compose or Kubernetes/Helm for deployments.
• Python 3.11+ for library workflows.
• NVIDIA GPU tooling for self-hosted NIM services.

Autonomy Principles

• Auto-detect everything: GPU, VRAM, drivers, Docker, CUDA, disk, OS, ports, existing services, NGC key, repo state.
• If it can be checked with a command, check it — don't ask the user.
• Ask only when user action is required: providing an API key, confirming data deletion, or choosing between equally valid options.
• Once analysis is done, route to the correct workflow and execute.

Intent Detection

Determine what the user wants and route immediately:

| User Intent | Action |

|-------------|--------|

| Deploy, install, set up, start RAG | Read and follow references/deploy.md |

| Configure, enable, change, toggle a feature | Use the Configure section below |

| Troubleshoot, debug, fix, error, unhealthy | Read and follow references/troubleshoot.md |

| Stop, shutdown, tear down, clean up | Read and follow references/shutdown.md |

If the intent is ambiguous, infer from context (e.g., "RAG isn't working" → troubleshoot; "get RAG running" → deploy). Only ask if genuinely unclear.

Configure

Requires a running RAG deployment. If services are not running, deploy first via references/deploy.md.

Match the user's request to a reference file, then read and follow it:

| Feature Keywords | Reference |

|-----------------|-----------|

| VLM, VLM embeddings, image captioning | references/configure/vlm.md |

| NeMo Guardrails | references/configure/guardrails.md |

| Agentic RAG, planning/execution agent, agentic streaming, stage events | references/configure/agentic-rag.md |

| Query rewriting, decomposition, multi-turn | references/configure/query-and-conversation.md |

| Ingestion (text-only, audio, Nemotron Parse, OCR, batch CLI, NV-Ingest, volume mount, performance) | references/configure/ingestion.md |

| Search, retrieval, hybrid search, multi-collection, metadata, filters, Elasticsearch filters, reranker, topK, accuracy/performance | references/configure/search-and-retrieval.md |

| LLM/embedding/ranking model changes, vector DB, Milvus/Elasticsearch auth, service keys, model profiles, ports/GPU | references/configure/models-and-infrastructure.md |

| Reasoning, thinking mode, reasoning_content, self-reflection, prompts, generation params (tokens, temperature, citations), per-request LLM params | references/configure/reasoning-and-generation.md |

| Summarization | references/configure/summarization.md |

| Observability (tracing, Zipkin, Grafana, Prometheus) | references/configure/observability.md |

| Multimodal query (image + text) | references/configure/multimodal-query.md |

| Data catalog (collection/document metadata) | references/configure/data-catalog.md |

| User interface (UI settings, reasoning panel, metadata filters) | references/configure/user-interface.md |

| API reference (endpoints, schemas) | references/configure/api-reference.md |

| Evaluation (RAGAS metrics) | references/configure/evaluation.md (and skill rag-eval) |

| MCP server & client, agent toolkit | references/configure/mcp.md |

| Migration (version upgrades) | references/configure/migration.md |

| Notebooks (setup and catalog) | references/configure/notebooks.md |

Configure Flow

• Match the user's request to a reference file from the table above.

• Detect what's running:

   echo "=== NIM ===" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(nim-llm|nemotron-(vlm-)?embedding|nemotron-ranking|nemotron-vlm|nemotron-3-nano-omni|page-elements|graphic-elements|table-structure|nemotron-ocr)' || echo "NO_LOCAL_NIMS"; echo "=== RAG ===" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(rag-server|ingestor-server|elasticsearch|milvus|seaweedfs|lancedb)' || echo "NO_DOCKER_RAG"; echo "=== K8S ===" && kubectl get pods -n rag 2>/dev/null | head -5 || echo "NO_K8S"; echo "=== LIBRARY ===" && ps aux 2>/dev/null | grep -E '(nvidia_rag|uvicorn.*rag)' | grep -v grep || echo "NO_LIBRARY"

• Use this table to determine platform, deployment type, and where config lives:

| Local NIMs running? | RAG services running? | Deployment Type | Config Location |

|---------------------|-----------------------|-----------------|-----------------|

| Yes (Docker) | Any | Self-hosted | deploy/compose/.env |

| No | Yes (Docker) | NVIDIA-hosted | deploy/compose/nvdev.env |

| Yes (K8s pods) | Any | Self-hosted | values.yaml (NIM sections) |

| No | Yes (K8s pods) | NVIDIA-hosted | values.yaml (envVars) |

| — | Library processes | Library mode | notebooks/config.yaml |

| No | No | Not running | Deploy first via references/deploy.md |

Tell the user what you detected and ask to confirm. Example: "I see local NIM containers running (nim-llm-ms, nemotron-vlm-embedding-ms) — this is a self-hosted deployment. Config file is deploy/compose/.env. Correct?"

• Check current feature state before changing anything — read the config location from step 3, then cross-check the live service:
• Docker: docker exec rag-server env 2>/dev/null | grep -E "<VAR_NAME>"
• Helm: kubectl get pod -n rag -l app=rag-server -o jsonpath='{.items[0].spec.containers[0].env}' 2>/dev/null

If the config file and live service disagree, tell the user the service has stale config and will need a restart.

• If the feature needs extra GPUs, check availability against hardware restrictions (see below):

   nvidia-smi --query-gpu=index,name,memory.total,memory.used --format=csv,noheader 2>/dev/null || echo "NO_GPU"

• Read the reference file and apply changes:
• Docker: edit the env file (uncomment to enable, re-comment to disable — the env file is the source of truth). Then restart the affected service:

     source <env-file> && docker compose -f deploy/compose/<compose-file> up -d

| Service | Compose File |

|---------|-------------|

| rag-server | docker-compose-rag-server.yaml |

| ingestor-server | docker-compose-ingestor-server.yaml |

| Elasticsearch, Milvus, etcd, SeaweedFS | vectordb.yaml |

| NIM containers (LLM, embedding, ranking, VLM, OCR, parse, audio, extraction) | nims.yaml |

| guardrails | docker-compose-nemo-guardrails.yaml |

| observability (Grafana, Prometheus, Zipkin) | observability.yaml |

• Helm: edit values.yaml, then upgrade: helm upgrade rag <chart> -n rag -f values.yaml
• Library: edit notebooks/config.yaml, then restart the Python process

• Verify:
• Docker: docker ps --format "table {{.Names}}\t{{.Status}}" | head -20; curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null | head -1
• Helm: kubectl get pods -n rag; kubectl rollout status deployment/rag-server -n rag --timeout=120s
• Library: curl -s http://localhost:8081/v1/health 2>/dev/null | head -1

• If restart fails, read references/troubleshoot.md. If multiple features requested, repeat from step 1 for each.

Examples

• "Deploy RAG" -> route to references/deploy.md.
• "Enable VLM" -> route to references/configure/vlm.md.
• "RAG is unhealthy" -> route to references/troubleshoot.md.
• "Stop RAG" -> route to references/shutdown.md.

Limitations

• Operational guidance only applies to this RAG Blueprint repository.
• Live deployment changes require a running Docker, Helm, or library target.
• Secrets such as NGC_API_KEY must be supplied by the user environment.

Troubleshooting

| Error / signal | What to do |

|----------------|------------|

| Services are not running | Follow references/deploy.md before configuring features. |

| Restart or health check fails | Follow references/troubleshoot.md. |

| User requests teardown | Follow references/shutdown.md and confirm destructive cleanup. |

When User Says "Configure" Without Specifics

Run steps 2–3 above, then read the identified config file to list what's currently enabled:

grep -E "^(export )?(ENABLE_|APP_)" <config-file> 2>/dev/null | sort

Summarize what's running and enabled, then ask which feature to change.

Hardware Restrictions

Read docs/support-matrix.md for current GPU requirements per deployment mode.

Read docs/service-port-gpu-reference.md for port mappings and GPU assignments.

| GPU | Feature Restrictions |

|-----|---------------------|

| B200 | No VLM, No Guardrails, No Nemotron Parse. May need multi-GPU LLM (LLM_MS_GPU_ID). |

| RTX PRO 6000 | No Nemotron Parse. No Audio on Helm. |