Instrument LLM Analytics Agent Skill

>- Add PostHog LLM analytics to trace AI model usage. Use after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. Also handles initial PostHog SDK setup if not yet installed.

65k tokens

context cost

the whole folder, loaded on every use

files

instructions only

copies elsewhere

how many repositories repackaged it

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/PostHog/skills --skill instrument-llm-analytics

What comes with it

252 079 bytes besides the instruction

references/README.md

references/anthropic.md

references/autogen.md

references/azure-openai.md

references/basics.md

references/calculating-costs.md

references/cerebras.md

references/cohere.md

references/crewai.md

references/deepseek.md

references/dspy.md

references/fireworks-ai.md

references/google.md

references/groq.md

references/helicone.md

references/hugging-face.md

references/instructor.md

references/langchain.md

references/langgraph.md

references/litellm.md

references/llamaindex.md

references/manual-capture.md

references/mastra.md

references/mirascope.md

references/mistral.md

references/ollama.md

references/openai-agents.md

references/openai.md

references/openrouter.md

references/perplexity.md

references/portkey.md

references/pydantic-ai.md

references/semantic-kernel.md

references/smolagents.md

references/together-ai.md

references/traces.md

references/vercel-ai.md

references/xai.md

The instruction itself

4 sections, as written by the author

Add PostHog LLM analytics

Use this skill to add PostHog LLM analytics that trace AI model usage in new or changed code. Use it after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. If PostHog is not yet installed, this skill also covers initial SDK setup. Supports any provider or framework.

Supported providers: OpenAI, Azure OpenAI, Anthropic, Google, Cohere, Mistral, Perplexity, DeepSeek, Groq, Together AI, Fireworks AI, xAI, Cerebras, Hugging Face, Ollama, OpenRouter.

Supported frameworks: LangChain, LlamaIndex, CrewAI, AutoGen, DSPy, LangGraph, Pydantic AI, Vercel AI, LiteLLM, Instructor, Semantic Kernel, Mirascope, Mastra, SmolAgents, OpenAI Agents.

Proxy/gateway: Portkey, Helicone.

Instructions

Follow these steps IN ORDER:

STEP 1: Analyze the codebase and detect the LLM stack.

Look for LLM provider SDKs (openai, anthropic, google-generativeai, etc.) and AI frameworks (langchain, llamaindex, crewai, etc.) in dependency files and imports.
Look for lockfiles to determine the package manager.
Check for existing PostHog or observability setup. If PostHog is already installed and LLM tracing is configured, skip to STEP 4 to add tracing for any new LLM calls.

STEP 2: Research instrumentation. (Skip if PostHog LLM tracing is already set up.)

2.1. Find the reference file below that matches the detected provider or framework — it is the source of truth for callback setup, middleware configuration, and event capture. Read it now.

2.2. If no reference matches, use manual-capture.md as a fallback — it covers the generic event capture approach that works with any provider.

STEP 3: Install the PostHog SDK. (Skip if PostHog is already set up.)

Add the PostHog SDK and any required callback/integration packages.
Do not manually edit dependency files — use the package manager's install command.
Always install packages as a background task. Don't await completion; proceed with other work immediately.

STEP 4: Add LLM tracing.

Instrument LLM calls to capture input tokens, output tokens, model name, latency, and costs for every generation.
Follow the provider-specific reference for the exact callback/middleware setup.
Do not alter the fundamental architecture of existing files. Make additions minimal and targeted.
You must read a file immediately before attempting to write it.

STEP 5: Link to users.

Associate LLM generations with identified users via distinct IDs when possible.

STEP 6: Set up environment variables.

Check if the project already has PostHog environment variables configured (e.g. in .env, .env.local, or framework-specific env files). If valid values already exist, skip this step.
If the PostHog API key is missing, use the PostHog MCP server's projects-get tool to retrieve the project's api_token. If multiple projects are returned, ask the user which project to use. If the MCP server is not connected or not authenticated, ask the user for their PostHog project API key instead.
For the PostHog host URL, use https://us.i.posthog.com for US Cloud or https://eu.i.posthog.com for EU Cloud.
Write these values to the appropriate env file using the framework's naming convention.
Reference these environment variables in code instead of hardcoding them.

Reference files

references/openai.md - Openai observability installation - docs
references/azure-openai.md - Azure openai observability installation - docs
references/README.md - PostHog.ai
references/anthropic.md - Anthropic ai observability installation - docs
references/google.md - Google ai observability installation - docs
references/cohere.md - Cohere ai observability installation - docs
references/mistral.md - Mistral ai observability installation - docs
references/perplexity.md - Perplexity ai observability installation - docs
references/deepseek.md - Deepseek ai observability installation - docs
references/groq.md - Groq ai observability installation - docs
references/together-ai.md - Together ai observability installation - docs
references/fireworks-ai.md - Fireworks ai observability installation - docs
references/xai.md - Xai observability installation - docs
references/cerebras.md - Cerebras ai observability installation - docs
references/hugging-face.md - Hugging face ai observability installation - docs
references/ollama.md - Ollama ai observability installation - docs
references/openrouter.md - Openrouter ai observability installation - docs
references/langchain.md - Langchain ai observability installation - docs
references/llamaindex.md - Llamaindex ai observability installation - docs
references/crewai.md - Crewai observability installation - docs
references/autogen.md - Autogen ai observability installation - docs
references/dspy.md - Dspy ai observability installation - docs
references/langgraph.md - Langgraph ai observability installation - docs
references/pydantic-ai.md - Pydantic ai observability installation - docs
references/vercel-ai.md - Vercel ai SDK observability installation - docs
references/litellm.md - Litellm ai observability installation - docs
references/instructor.md - Instructor ai observability installation - docs
references/semantic-kernel.md - Semantic kernel ai observability installation - docs
references/mirascope.md - Mirascope ai observability installation - docs
references/mastra.md - Mastra ai observability installation - docs
references/smolagents.md - Smolagents ai observability installation - docs
references/openai-agents.md - Openai agents SDK observability installation - docs
references/portkey.md - Portkey ai observability installation - docs
references/helicone.md - Helicone ai observability installation - docs
references/manual-capture.md - Manual capture ai observability installation - docs
references/basics.md - Ai observability basics - docs
references/traces.md - Traces - docs
references/calculating-costs.md - Calculating llm costs - docs

Each provider reference contains installation instructions, SDK setup, and code examples specific to that provider or framework. Find the reference that matches the user's stack.

If the user's provider isn't listed, use manual-capture.md as a fallback — it covers the generic event capture approach that works with any provider.

Key principles

Environment variables: Always use environment variables for PostHog and LLM provider keys. Never hardcode them.
Minimal changes: Add LLM analytics alongside existing LLM calls. Don't replace or restructure existing code.
Trace all generations: Capture input tokens, output tokens, model name, latency, and costs for every LLM call.
Link to users: Associate LLM generations with identified users via distinct IDs when possible.
One provider at a time: Only instrument the provider(s) the user is actually using. Don't add instrumentation for providers not present in the codebase.

Other skills for the same job

different authors, same section of the catalogue

Datacommons Client

by christophacham

×3

Work with Data Commons, a platform providing programmatic access to public statistical data from global sources. Use this skill when working with demographic data, economic indicators, health statistics, environmental data, or any public datasets available through Data Commons. Applicable for querying population statistics, GDP figures, unemployment rates, disease prevalence, geographic entity resolution, and exploring relationships between statistical entities.

Neuropixels neural recording analysis. Load SpikeGLX/OpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen/IBL curation, AI-assisted visual analysis, for Neuropixels 1.0/2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

20k tokens

Senior Data Scientist

by ComeOnOliver

×3

World-class data science skill for statistical modeling, experimentation, causal inference, and advanced analytics. Expertise in Python (NumPy, Pandas, Scikit-learn), R, SQL, statistical methods, A/B testing, time series, and business intelligence. Includes experiment design, feature engineering, model evaluation, and stakeholder communication. Use when designing experiments, building predictive models, performing causal analysis, or driving data-driven decisions.

Python interface to OpenMS for mass spectrometry data analysis. Use for LC-MS/MS proteomics and metabolomics workflows including file handling (mzML, mzXML, mzTab, FASTA, pepXML, protXML, mzIdentML), signal processing, feature detection, peptide identification, and quantitative analysis. Apply when working with mass spectrometry data, analyzing proteomics experiments, or processing metabolomics datasets.

Parallel/distributed computing. Scale pandas/NumPy beyond memory, parallel DataFrames/Arrays, multi-file processing, task graphs, for larger-than-RAM datasets and parallel workflows.

35k tokens

Tensorboard

by Orchestra-Research

×1

Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit

How to use it

Copy the folder

Take posthog/instrument-llm-analytics from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.