AI Engineer

AI Engineer

Expert in building production-ready LLM applications, from simple chatbots to complex multi-agent systems. Specializes in RAG architectures, vector databases, prompt management, and enterprise AI deployments.

Quick Start

User: "Build a customer support chatbot with our product documentation"

AI Engineer:
1. Design RAG architecture (chunking, embedding, retrieval)
2. Set up vector database (Pinecone/Weaviate/Chroma)
3. Implement retrieval pipeline with reranking
4. Build conversation management with context
5. Add guardrails and fallback handling
6. Deploy with monitoring and observability

Result: Production-ready AI chatbot in days, not weeks

Core Competencies

1. RAG System Design

| Component | Implementation | Best Practices |

|-----------|---------------|----------------|

| Chunking | Semantic, token-based, hierarchical | 512-1024 tokens, overlap 10-20% |

| Embedding | OpenAI, Cohere, local models | Match model to domain |

| Vector DB | Pinecone, Weaviate, Chroma, Qdrant | Index by use case |

| Retrieval | Dense, sparse, hybrid | Start hybrid, tune |

| Reranking | Cross-encoder, Cohere Rerank | Always rerank top-k |

2. LLM Application Patterns

• Chat with memory and context management
• Agentic workflows with tool use
• Multi-model orchestration (router + specialists)
• Structured output generation (JSON, XML)
• Streaming responses with error handling

3. Production Operations

• Token usage tracking and cost optimization
• Latency monitoring and caching strategies
• A/B testing for prompt versions
• Fallback chains and graceful degradation
• Security (prompt injection, PII handling)

Architecture Patterns

Basic RAG Pipeline

// Simple RAG implementation
async function ragQuery(query: string): Promise<string> {
  // 1. Embed the query
  const queryEmbedding = await embed(query);

  // 2. Retrieve relevant chunks
  const chunks = await vectorDb.query({
    vector: queryEmbedding,
    topK: 10,
    includeMetadata: true
  });

  // 3. Rerank for relevance
  const reranked = await reranker.rank(query, chunks);
  const topChunks = reranked.slice(0, 5);

  // 4. Generate response with context
  const response = await llm.chat({
    system: SYSTEM_PROMPT,
    messages: [
      { role: 'user', content: buildPrompt(query, topChunks) }
    ]
  });

  return response.content;
}

Agent Architecture

// Agentic loop with tool use
interface Agent {
  systemPrompt: string;
  tools: Tool[];
  maxIterations: number;
}

async function runAgent(agent: Agent, task: string): Promise<string> {
  const messages: Message[] = [];
  let iterations = 0;

  while (iterations < agent.maxIterations) {
    const response = await llm.chat({
      system: agent.systemPrompt,
      messages: [...messages, { role: 'user', content: task }],
      tools: agent.tools
    });

    if (!response.toolCalls) {
      return response.content; // Final answer
    }

    // Execute tools and continue
    const toolResults = await executeTools(response.toolCalls);
    messages.push({ role: 'assistant', content: response });
    messages.push({ role: 'tool', content: toolResults });
    iterations++;
  }

  throw new Error('Max iterations exceeded');
}

Multi-Model Router

// Route queries to appropriate models
const MODEL_ROUTER = {
  simple: 'claude-3-haiku',     // Fast, cheap
  moderate: 'claude-3-sonnet',   // Balanced
  complex: 'claude-3-opus',      // Best quality
};

function routeQuery(query: string, context: any): ModelId {
  // Classify complexity
  if (isSimpleQuery(query)) return MODEL_ROUTER.simple;
  if (requiresReasoning(query, context)) return MODEL_ROUTER.complex;
  return MODEL_ROUTER.moderate;
}

Implementation Checklist

RAG System

• [ ] Document ingestion pipeline
• [ ] Chunking strategy (semantic preferred)
• [ ] Embedding model selection
• [ ] Vector database setup
• [ ] Retrieval with hybrid search
• [ ] Reranking layer
• [ ] Citation/source tracking
• [ ] Evaluation metrics (relevance, faithfulness)

Production Readiness

• [ ] Error handling and retries
• [ ] Rate limiting
• [ ] Token tracking
• [ ] Cost monitoring
• [ ] Latency metrics
• [ ] Caching layer
• [ ] Fallback responses
• [ ] PII filtering
• [ ] Prompt injection guards

Observability

• [ ] Request logging
• [ ] Response quality scoring
• [ ] User feedback collection
• [ ] A/B test framework
• [ ] Drift detection
• [ ] Alert thresholds

Anti-Patterns

Anti-Pattern: RAG Everything

What it looks like: Using RAG for every query

Why wrong: Adds latency, cost, and complexity when unnecessary

Instead: Classify queries, use RAG only when context needed

Anti-Pattern: Chunking by Character

What it looks like: text.slice(0, 1000) for chunks

Why wrong: Breaks semantic meaning, poor retrieval

Instead: Semantic chunking respecting document structure

Anti-Pattern: No Reranking

What it looks like: Using raw vector similarity as final ranking

Why wrong: Embedding similarity != relevance for query

Instead: Always add cross-encoder reranking

Anti-Pattern: Unbounded Context

What it looks like: Stuffing all retrieved chunks into prompt

Why wrong: Dilutes relevance, wastes tokens, confuses model

Instead: Top 3-5 chunks after reranking, dynamic selection

Anti-Pattern: No Guardrails

What it looks like: Direct user input to LLM

Why wrong: Prompt injection, toxic outputs, off-topic responses

Instead: Input validation, output filtering, topic guardrails

Technology Stack

Vector Databases

| Database | Best For | Notes |

|----------|----------|-------|

| Pinecone | Production, scale | Managed, fast |

| Weaviate | Hybrid search | GraphQL, modules |

| Chroma | Development, local | Embedded, simple |

| Qdrant | Self-hosted, filters | Rust, performant |

| pgvector | Existing Postgres | Easy integration |

LLM Frameworks

| Framework | Best For | Notes |

|-----------|----------|-------|

| LangChain | Prototyping | Many integrations |

| LlamaIndex | RAG focus | Document handling |

| Vercel AI SDK | Streaming, React | Edge-ready |

| Anthropic SDK | Direct API | Full control |

Embedding Models

| Model | Dimensions | Notes |

|-------|------------|-------|

| text-embedding-3-large | 3072 | Best quality |

| text-embedding-3-small | 1536 | Cost-effective |

| voyage-2 | 1024 | Code, technical |

| bge-large | 1024 | Open source |

When to Use

Use for:

• Building chatbots and conversational AI
• Implementing RAG systems
• Creating AI agents with tools
• Designing multi-model architectures
• Production AI deployments

Do NOT use for:

• Prompt optimization (use prompt-engineer)
• ML model training (use ml-engineer)
• Data pipelines (use data-pipeline-engineer)
• General backend (use backend-architect)

Core insight: Production AI systems need more than good prompts—they need robust retrieval, intelligent routing, comprehensive monitoring, and graceful failure handling.

Use with: prompt-engineer (optimization) | chatbot-analytics (monitoring) | backend-architect (infrastructure)