Azure Aigateway

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

> To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

| Category | Triggers |

|----------|----------|

| Model Governance | "semantic caching", "token limits", "load balance AI", "track token usage" |

| Tool Governance | "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" |

| Agent Governance | "content safety", "jailbreak detection", "filter harmful content" |

| Configuration | "add Azure OpenAI backend", "configure my model", "add AI Foundry model" |

| Testing | "test AI gateway", "call OpenAI through gateway" |

Quick Reference

| Policy | Purpose | Details |

|--------|---------|---------|

| azure-openai-token-limit | Cost control | Model Policies |

| azure-openai-semantic-cache-lookup/store | 60-80% cost savings | Model Policies |

| azure-openai-emit-token-metric | Observability | Model Policies |

| llm-content-safety | Safety & compliance | Agent Policies |

| rate-limit-by-key | MCP/tool protection | Tool Policies |

Get Gateway Details

# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>

Test AI Endpoint

GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

Common Tasks

Add AI Backend

See references/patterns.md for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>

Apply AI Governance Policy

Recommended policy order in <inbound>:

• Authentication - Managed identity to backend
• Semantic Cache Lookup - Check cache before calling AI
• Token Limits - Cost control
• Content Safety - Filter harmful content
• Backend Selection - Load balancing
• Metrics - Token usage tracking

See references/policies.md for complete example.

Troubleshooting

| Issue | Solution |

|-------|----------|

| Token limit 429 | Increase tokens-per-minute or add load balancing |

| No cache hits | Lower score-threshold to 0.7 |

| Content false positives | Increase category thresholds (5-6) |

| Backend auth 401 | Grant APIM "Cognitive Services User" role |

See references/troubleshooting.md for details.

References

• Detailed Policies - Full policy examples
• Configuration Patterns - Step-by-step patterns
• Troubleshooting - Common issues
• AI-Gateway Samples
• GenAI Gateway Docs

SDK Quick References

• Content Safety: Python | TypeScript
• API Management: Python | .NET