microsoft/deployment-e2e-testing
Guide for writing Aspire deployment end-to-end tests. Use this when asked to create, modify, or debug deployment E2E tests that deploy to Azure.
npx skills add https://github.com/microsoft/aspire --skill deployment-e2e-testingThis skill provides patterns and practices for writing end-to-end tests that deploy Aspire applications to real Azure infrastructure.
Deployment E2E tests extend the CLI E2E testing patterns with actual Azure deployments. They use the Hex1b terminal automation library to drive the Aspire CLI and verify that deployed applications work correctly.
Location: tests/Aspire.Deployment.EndToEnd.Tests/
Supported Platforms: Linux only (Hex1b requirement).
Prerequisites:
Deployment tests build on the CLI E2E testing skill. Before working with deployment tests, familiarize yourself with:
Key differences from CLI E2E tests:
| Aspect | CLI E2E Tests | Deployment E2E Tests |
|--------|---------------|----------------------|
| Duration | 5-15 minutes | 15-45 minutes |
| Resources | Local only | Azure resources |
| Authentication | None | Azure OIDC/CLI |
| Cleanup | Temp directories | Azure resource groups |
| Triggers | PR, push | Nightly, manual, deploy-test/* |
DeploymentE2ETestHelpers (Helpers/DeploymentE2ETestHelpers.cs): Terminal factory and environment helpersDeploymentE2EAutomatorHelpers (Helpers/DeploymentE2EAutomatorHelpers.cs): Async extension methods on Hex1bTerminalAutomator for deployment scenariosHex1bAutomatorTestHelpers (shared): Common async extension methods on Hex1bTerminalAutomator (WaitForSuccessPromptAsync, AspireNewAsync, etc.)AzureAuthenticationHelpers (Helpers/AzureAuthenticationHelpers.cs): Azure auth and resource namingDeploymentReporter (Helpers/DeploymentReporter.cs): GitHub step summary reportingSequenceCounter (Helpers/SequenceCounter.cs): Prompt tracking (same as CLI E2E)Each deployment test:
aspire newaspire deploypublic sealed class MyDeploymentTests(ITestOutputHelper output)
{
[Fact]
public async Task DeployMyScenario()
{
// 1. Validate prerequisites
var subscriptionId = AzureAuthenticationHelpers.TryGetSubscriptionId();
if (string.IsNullOrEmpty(subscriptionId))
{
Assert.Skip("Azure subscription not configured.");
}
if (!AzureAuthenticationHelpers.IsAzureAuthAvailable())
{
if (DeploymentE2ETestHelpers.IsRunningInCI)
{
Assert.Fail("Azure auth not available in CI.");
}
Assert.Skip("Azure auth not available. Run 'az login'.");
}
// 2. Setup
var resourceGroupName = AzureAuthenticationHelpers.GenerateResourceGroupName("my-scenario");
var workspace = TemporaryWorkspace.Create(output);
var startTime = DateTime.UtcNow;
try
{
// 3. Build terminal and run deployment
using var terminal = DeploymentE2ETestHelpers.CreateTestTerminal();
var pendingRun = terminal.RunAsync(TestContext.Current.CancellationToken);
var counter = new SequenceCounter();
var auto = new Hex1bTerminalAutomator(terminal, defaultTimeout: TimeSpan.FromSeconds(500));
await auto.PrepareEnvironmentAsync(workspace, counter);
if (DeploymentE2ETestHelpers.IsRunningInCI)
{
await auto.SourceAspireCliEnvironmentAsync(counter);
}
await auto.AspireNewAsync("MyProject", counter, useRedisCache: false);
// ... deployment steps ...
await auto.TypeAsync("exit");
await auto.EnterAsync();
await pendingRun;
// 4. Report success
var duration = DateTime.UtcNow - startTime;
DeploymentReporter.ReportDeploymentSuccess(
nameof(DeployMyScenario),
resourceGroupName,
deploymentUrls,
duration);
}
catch (Exception ex)
{
DeploymentReporter.ReportDeploymentFailure(
nameof(DeployMyScenario),
resourceGroupName,
ex.Message);
throw;
}
finally
{
// 5. Cleanup Azure resources
await CleanupResourceGroupAsync(resourceGroupName);
}
}
}
| Method | Description |
|--------|-------------|
| PrepareEnvironmentAsync(workspace, counter) | Sets up the terminal environment with custom prompt and workspace directory |
| InstallAspireCliFromPullRequestAsync(prNumber, counter) | Downloads and installs the Aspire CLI from a PR build artifact |
| InstallAspireCliReleaseAsync(counter) | Installs the latest released Aspire CLI |
| SourceAspireCliEnvironmentAsync(counter) | Adds ~/.aspire/bin to PATH so the aspire command is available |
These extend Hex1bTerminalAutomator and are used alongside the shared Hex1bAutomatorTestHelpers methods (WaitForSuccessPromptAsync, AspireNewAsync, etc.) documented in the CLI E2E Testing Skill.
Tests use OIDC (Workload Identity Federation) for authentication:
- name: Azure Login (OIDC)
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ vars.ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION }}
The test code automatically detects CI and uses DefaultAzureCredential which picks up the OIDC session.
Authenticate with Azure CLI before running tests:
# Login to Azure
az login
# Set your subscription
az account set --subscription "your-subscription-id"
# Set environment variable
export ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION="your-subscription-id"
# Run tests
dotnet test tests/Aspire.Deployment.EndToEnd.Tests/
// Check if auth is available
if (!AzureAuthenticationHelpers.IsAzureAuthAvailable())
{
Assert.Skip("Azure auth not available");
}
// Get subscription ID
var subscriptionId = AzureAuthenticationHelpers.GetSubscriptionId();
// Generate unique resource group name
var rgName = AzureAuthenticationHelpers.GenerateResourceGroupName("my-test");
// Result: "aspire-e2e-my-test-20240115-abc12345"
// Check auth type
if (AzureAuthenticationHelpers.IsOidcConfigured())
{
// Using OIDC (CI)
}
else
{
// Using Azure CLI (local)
}
Resource groups are named with a consistent pattern for easy identification and cleanup:
{prefix}-{testname}-{date}-{runid}
Example: aspire-e2e-aca-starter-20240115-12345678
Components:
ASPIRE_DEPLOYMENT_TEST_RG_PREFIX (default: aspire-e2e)Tests automatically write to the GitHub step summary:
// Report success with URLs
DeploymentReporter.ReportDeploymentSuccess(
testName: "DeployStarterToACA",
resourceGroupName: "aspire-e2e-...",
deploymentUrls: new Dictionary<string, string>
{
["Dashboard"] = "https://dashboard.azurecontainerapps.io",
["Web Frontend"] = "https://webfrontend.azurecontainerapps.io"
},
duration: TimeSpan.FromMinutes(15));
// Report failure
DeploymentReporter.ReportDeploymentFailure(
testName: "DeployStarterToACA",
resourceGroupName: "aspire-e2e-...",
errorMessage: "Deployment timed out",
logs: "Full deployment logs...");
Tests generate recordings for debugging:
var recordingPath = DeploymentE2ETestHelpers.GetTestResultsRecordingPath(nameof(MyTest));
// CI: $GITHUB_WORKSPACE/testresults/recordings/MyTest.cast
// Local: /tmp/aspire-deployment-e2e/recordings/MyTest.cast
Always cleanup Azure resources in a finally block:
private static async Task CleanupResourceGroupAsync(string resourceGroupName)
{
var process = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = "az",
Arguments = $"group delete --name {resourceGroupName} --yes --no-wait",
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false
}
};
process.Start();
await process.WaitForExitAsync();
if (process.ExitCode != 0)
{
var error = await process.StandardError.ReadToEndAsync();
throw new InvalidOperationException($"Cleanup failed: {error}");
}
}
Deployment tests are triggered by:
maindeploy-test/* - For rapid iterationTo iterate quickly during development:
# Create a protected branch
git checkout -b deploy-test/my-feature
# Make changes
# ...
# Push to trigger workflow
git push origin deploy-test/my-feature
| Variable | Required | Description |
|----------|----------|-------------|
| ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION | Yes | Azure subscription ID |
| ASPIRE_DEPLOYMENT_TEST_RG_PREFIX | No | Resource group prefix (default: aspire-e2e) |
| AZURE_DEPLOYMENT_TEST_TENANT_ID | CI | Azure AD tenant ID (GitHub secret) |
| AZURE_DEPLOYMENT_TEST_CLIENT_ID | CI | OIDC app client ID (GitHub secret) |
| AZURE_DEPLOYMENT_TEST_SUBSCRIPTION_ID | CI | Azure subscription ID (GitHub secret) |
var subscriptionId = AzureAuthenticationHelpers.TryGetSubscriptionId();
if (string.IsNullOrEmpty(subscriptionId))
{
Assert.Skip("Subscription not configured");
}
if (!AzureAuthenticationHelpers.IsAzureAuthAvailable())
{
Assert.Skip("Azure auth not available");
}
var rgName = AzureAuthenticationHelpers.GenerateResourceGroupName("my-test");
DeploymentReporter.ReportDeploymentSuccess(...);
// or
DeploymentReporter.ReportDeploymentFailure(...);
try
{
// ... deployment test ...
}
finally
{
await CleanupResourceGroupAsync(resourceGroupName);
}
// DON'T
var subscriptionId = "12345-abcde-...";
// DO
var subscriptionId = AzureAuthenticationHelpers.GetSubscriptionId();
// DON'T - cleanup might not run
await DeployAsync();
await CleanupAsync(); // Skipped if deploy throws!
// DO - always cleanup
try
{
await DeployAsync();
}
finally
{
await CleanupAsync(); // Always runs
}
Local: Ensure Azure CLI is authenticated:
az login
az account show
CI: Check OIDC configuration:
AZURE_CLIENT_ID secret is setAZURE_TENANT_ID secret is setDeployments can take 15-30+ minutes. If tests timeout:
WaitUntil callsFind and cleanup orphaned test resources:
# List all test resource groups
az group list --query "[?starts_with(name, 'aspire-e2e')]" -o table
# Delete specific resource group
az group delete --name aspire-e2e-xxx --yes
The test tenant rotates every ~90 days. When rotation occurs:
deployment-testing environmentAZURE_DEPLOYMENT_TEST_CLIENT_ID, AZURE_DEPLOYMENT_TEST_TENANT_ID, AZURE_DEPLOYMENT_TEST_SUBSCRIPTION_IDTake microsoft/deployment-e2e-testing from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.