google/dev-testing
Capsem testing policy and workflow. Use whenever running tests, writing new tests, or verifying changes work. Covers focused native tests, developer smoke feedback, the single public full release gate, TDD red-green-refactor, adversarial security testing, coverage policy, and mandatory end-to-end VM validation. For VM-specific tests see dev-testing-vm, for hypervisor tests see dev-testing-hypervisor, for frontend tests see dev-testing-frontend.
npx skills add https://github.com/google/capsem --skill dev-testingRead tests/README.md before adding or moving test fixtures. Test-only config
belongs under tests/fixtures/, not root config/.
| Command | What | VM? |
|---------|------|-----|
| just test | Everything: unit/coverage, cross-compile, frontend, Python/VM integration, injection, benchmarks, Linux install, and exact-package clean-Tart macOS install/glow-up | Yes |
| just smoke | Focused developer feedback: repack, sign, boot, doctor, MCP, and service integration | Yes |
just test is the single release source of truth. just smoke is useful
developer feedback, but it never qualifies or releases anything. During TDD
run the smallest native pytest, cargo, pnpm, or script command directly;
release commands must execute full just test.
The full gate is a construction boundary, not the edit loop. During TDD,
reproduce the failure with the smallest focused test, run that test red/green,
and batch adjacent parity fixes before paying for the complete gate. Run it
once when the forward-fix source state is ready; any later production or gate
change needs one new complete run.
just test deliberately accepts committed or uncommitted developer work. It
records HEAD plus a digest of every tracked and untracked non-ignored source
byte, then fails if either changes while the gate runs. Generated output stays
under ignored build directories. The local proof therefore covers the exact
source state the developer asked to test without forcing commit choreography.
Automatic gate benchmark output belongs under ignored
target/test-benchmarks/. Historical benchmark publication uses the owning
pytest/benchmark command and an explicit review; it is not a Just convenience
recipe.
Release CI reuses the same checked-in private modules as local just test:
_test-fast_test-static_test-artifacts_test-functional_test-glowup_test-release-contractsThe build scope is selective; the quality scope is not. Binary CI builds only
packages and resolves every channel profile by recorded digest. Profile CI
builds exactly one channel/profile and resolves the current package by recorded
digest. The resolved complementary artifacts are staged into the production
test harness, not replaced by source-built substitutes.
Every activated pairing must pass artifact validation, all VM suites,
Winterfell/MCP lifecycle, IronBank, injection, integration, benchmarks, full
capsem-doctor, exact native install, and update glow-up. A staged profile
whose minimum package is not yet satisfied may run only the self-consistency,
integrity, isolation, and boot proof; the following binary lane must run the
complete functional and glow-up modules before activation.
Both macOS and Linux must install their exact publishable native packages,
including their real post-install scripts, before publication. Notarization and
the public stable-to-nightly switch/upgrade glow-up remain mandatory
end-to-end proof.
On Apple Silicon macOS, just test owns the pre-publication macOS package
boundary through scripts/macos_release_glowup.py: it builds the package with the
production assembler, installs that exact file in a disposable headless Tart
guest, verifies the receipt, app bundle, complete binary cohort, service and
gateway health, then extracts the same package on the physical Mac and boots a
real Capsem guest VM from its exact binary/profile payload to a shell marker.
The local package is unsigned; its postinstall ad-hoc signs the installed
Mach-O payload with the required entitlements. Local qualification must not
load Developer ID material or create a signing keychain. The tagged publication
workflow alone signs, notarizes, staples, and installs the final publishable
package.
Tart macOS guests do not support nested virtualization, so these are two
explicit halves of one script rather than a claimed nested proof. just smoke
deliberately excludes Tart and therefore cannot be used for release.
Rust is pinned to 1.97.1 across the workspace file, workflow steps,
host-builder, and bootstrap. Change all pin surfaces together in a deliberate
toolchain-bump PR. RustSec and JavaScript bulk advisories are blocking in
just smoke, local just test, ordinary CI, and both release lanes,
as well as the scheduled/manual audit. A new advisory fails the candidate
until it is remediated or explicitly reviewed in checked-in scanner policy.
Linux proof is host-aware: a cross-built non-host package receives structural
validation in qualification and exact native installation in its tagged release
job. CAPSEM_REQUIRE_LINUX_DEB_PROOF=1 must not reject that non-host package
before the host package reaches its mandatory KVM proof. The hosted arm64 runner
does not expose /dev/kvm, so it proves exact package/service operation while
the x86_64 runner additionally owns the guest-shell marker.
Expensive harnesses need a cheap clean-environment bootstrap proof at the
start of just test, before Docker/Colima or artifact preparation. The one
private _test-fast module is also called by just smoke, ordinary CI, and
both release lanes. It owns YAML/workflow and source syntax, source contracts,
dependency audits, Clippy, Python lint/type checks, and JavaScript/frontend
checks; no caller may reproduce a subset inline. Only a green fast gate may
build the Linux install-test image. That preflight must use a
container-owned UV_PROJECT_ENVIRONMENT and prove python -m pytest launches
before VM, package, or asset work consumes hours. Keep an ordering contract.
It is fail-fast infrastructure
validation only: the later Docker/systemd install E2E remains mandatory and
must still exercise the installed package and post-install behavior.
Read references/local-ci-parity.md before editing any release workflow, gate
recipe, or CI job. It holds the Ironbank parity rule (every portable release
gate must be owned by just test), scanner/tool pinning, Docker platform and
prune discipline, runtime/disk budgets, and the source-guard contracts.
Write tests first:
Without a failing test first, it's easy to write tests that pass by accident or don't actually verify the behavior you intended.
Every non-trivial feature slice needs evidence in all of these categories before it can be called done. A green unit suite or a benchmark is not a substitute for functional or end-to-end proof.
| Category | What it proves | Minimum expectation |
|----------|----------------|---------------------|
| Unit/contract | Pure logic, parser state machines, schema migration, helper APIs | Red/green tests for normal and edge behavior at the smallest useful boundary |
| Functional | The feature works through its production-facing API, not just private helpers | Exercise the real module boundary with realistic inputs and assert outputs plus side effects |
| Adversarial | The feature preserves security, privacy, and policy invariants when attacked | Malformed, oversized, denied, missing, racing, timeout, permission, and leak-prevention cases |
| E2E/VM | The user-visible path works in a real Capsem session | Boot/run a VM or use the black-box CLI/MCP/service path, then inspect externally visible behavior |
| Telemetry | Audit data is present, accurate, and queryable | Query session.db or logger readers for required rows, fields, decisions, errors, and attribution |
| Performance | Hot paths stayed inside the accepted budget | Benchmarks or timing assertions with recorded numbers and regression criteria |
If a category is genuinely impossible or deliberately deferred, record it as missing with a reason, owner, and follow-up task. Silent deferral is the bug. "Covered by later E2E" is not enough unless the tracker names the later test and the current milestone is explicitly scoped as internal-only.
For policy, MITM, MCP, telemetry, networking, filesystem, process lifecycle, or sandbox-boundary work, the functional slice matrix is mandatory. The tests should prove not only that the happy path succeeds, but also that enforcement happens at the intended boundary: a blocked MCP tool does not dispatch, a blocked return does not leak, a denied URL does not reach the network, a malformed frame does not poison the stream, and telemetry records the truth.
Use /ironbank for release-critical VM, network, model, MCP, credential
broker, package-manager, doctor, benchmark, and security acceptance proof.
Ironbank lives in tests/ironbank/ and is full black-box: tests are written
from public contracts, CLI help, docs, generated schemas, hermetic fixtures,
route responses, logs, DB rows, and installed package metadata. Do not inspect
Rust/product internals to decide expected behavior.
Ironbank cannot use:
skip, skipif, slow, optional markers, or public-network dependenciesOne deterministic stimulus must prove the whole ledger path: client result,
parsed facts, CEL/security decision, detection/enforcement rows, protocol DB
rows, structured logs, status counters, UDS route, HTTP route, and UI-facing
JSON. Every emitted DB/log/route field is exact-value asserted, covered by a
typed invariant, or explicitly marked not applicable. Unknown fields fail the
test until the field ledger is updated.
Package-manager tests prove function. Installing zstd, for example, means
compressing known bytes, decompressing them, and comparing the exact output;
not just checking dpkg output.
Telemetry and security ledgers are database-owned. Service routes, UI handlers,
MCP helpers, and benchmark harnesses must not build their own logged-data
projection caches and must not open SQLite directly. They may own query intent
(for example the fields a route needs), but they call the logger DB object to
execute it. The logger DB object owns connection threads, mem/disk table
layout, write buffering, flush, reload-from-disk behavior, WAL tuning, and
future FTS5/search tables.
Do not move route-specific SQL into DbWriter or turn the DB layer into a pile
of route helper methods such as stats_detail_payload() just to hide SQL. The
boundary is execution and storage mechanics:
db.ready().await?;
db.query(sql, params).await?;
db.write(event).await?;
db.write(event).await means the DB object accepted the event into its
producer buffer. Tests that assert read-after-write rows must use the DB flush
barrier or shutdown/reopen. Do not paper over visibility with sleeps, route
projections, or direct SQLite readers.
Empty table means empty result. Missing table or column means the schema
contract is broken and must fail loudly; never add compatibility branches that
treat missing ledger shape as empty data.
Regression tests must guard the boundary. If a route needs ledger data, add a
test that proves the route uses the DB object and a source guard that rejects
raw rusqlite opens, direct DbReader::open, and service-owned projection
state in production route code. Add a companion guard that prevents
route-specific DB writer methods or missing-schema fallbacks from being
introduced.
crates/capsem-mock-server is the single reusable local fixture server for
benchmarks, doctor, protocol recording/replay, gateway/integration tests, and
Ironbank. It owns mock protocol responses and deterministic local upstream
behavior. Tests may launch it through scripts/mock_server.py,
tests/helpers/mock_server.py, or CAPSEM_MOCK_SERVER_BASE_URL.
Do not add another local HTTP/MCP/OAuth/model mock server for a feature. Extend
the shared mock server and its fixtures instead, then assert the route through
the relevant black-box test.
just test runs the python suite under pytest -n 4 --dist=loadfile. Four real VMs boot simultaneously. This is the canary, not just a speed-up. We ship Capsem as a multi-VM sandbox for AI agents -- if our own test suite cannot safely boot 4 concurrent VMs, real users running an agent farm will hit the exact same bug. Treat any concurrency flake as a Capsem-side bug, not a test-tuning problem:
validate_vm_name / registry, not "isolate test names better"Anti-patterns when a test flakes under -n 4:
time.sleep() to "let things settle" -- masking a raceserial so it runs alone -- defeating the dogfooding signalThe exception is a true timing or benchmark probe whose assertion is the
measured number. Those tests must already be marked serial and just test
runs them immediately after the -n 4 canary. That is not a flake escape
hatch: it prevents another benchmark file from stealing the same Apple VZ
launch budget and corrupting the number we are trying to publish.
The host has plenty of headroom (48 GB RAM, 14 cores; 4 VMs at 2 GB / 2 CPU each = 8 GB / 8 cores). If concurrency surfaces a flake, fix the product, then re-run. Bumping -n higher (8, 12) is the natural follow-on once n=4 is stable -- real users will run more.
If a previous just test -n 4 run was interrupted (ctrl-C, pytest-xdist worker death, host crash) and the NEXT run flakes with "vm-ready never asserted", UDS "connection refused", or mysterious HTTP 500s -- the cause is companion processes from the interrupted run still alive under PID 1. pkill -f "target/debug/capsem-(service|process|gateway|tray|mcp)" will make the flake vanish, but that is cleanup-after-the-fact. The fix is on the COMPANION side: every spawned companion (gateway, tray, and any new one) must use capsem-guard::install(parent_pid, lock_path) to enforce (a) refuse-standalone, (b) singleton, (c) self-exit on parent death. See /dev-rust-patterns lesson 18. Regression tests live in tests/capsem-service/test_companion_lifecycle.py -- never remove them; when adding a new companion, extend that file.
Never pkill -f capsem- with a broad pattern during test debugging: capsem- matches --crate-name capsem-core in running rustc/cargo invocations and will SIGKILL the compiler mid-build. Use a binary-path pattern like pkill -f "target/debug/capsem-(service|process|gateway|tray|mcp)" instead.
Apple's Virtualization.framework does not tolerate overlapping checkpoint
lifecycle operations (saveMachineStateToURL and restoreMachineStateFromURL)
on sibling VMs, and teardown must not cross those checkpoint edges. Capsem uses
ServiceState::save_restore_lock plus the host-wide VzHostLock flock:
cold starts and teardown take shared/read guards, save and restore take
exclusive/write guards. The rail holds even when pytest-xdist spawns one
capsem-service per worker, while independent cold starts can still run
together for the boot-latency gate.
Do not demote suspend/resume, lifecycle, provisioning, or teardown tests to
-n 1 to sidestep VZ races. just test at -n 4 is the contract; if a
concurrent run sees restore permission errors, loop-device corruption,
connection-refused startup races, or readiness misses, fix the lifecycle rail.
Full context and failure signatures live in
docs/src/content/docs/gotchas/concurrent-suspend-resume.md.
Capsem is a security product. Every security-relevant feature needs tests that actively try to break invariants. Think like an attacker:
Stress-test boundary conditions. Write tests for the attacks you'd attempt yourself.
When touching security-relevant code, check these invariants have test coverage:
| Invariant | What to test | Where |
|-----------|-------------|-------|
| VirtioFS share is guest/ only | session_dir/guest/ exists, symlinks resolve, host-only files (session.db, serial.log) are outside the share | capsem-core::lib::tests |
| UDS sockets are 0600 | After bind, verify permissions exclude other users | capsem-process |
| Process env is cleared | env_clear() called, only allowlisted vars passed | capsem-service spawn tests |
| No process::exit on guest I/O | Control channel close causes loop break, not exit | capsem-process |
| Sensitive logs are 0600 | serial.log created with restricted permissions | capsem-process |
| Gateway auth on all routes | Every route except GET / returns 401 without token | capsem-gateway::auth::tests |
| Auth rate limiting | 429 after threshold, resets after window | capsem-gateway::auth::tests |
| CORS rejects external origins | Only localhost/127.0.0.1/tauri allowed | capsem-gateway::tests |
| Body size limit | 413 for >10MB payloads | capsem-gateway::proxy::tests |
| VM ID validation | Path traversal (../), dots, spaces, null bytes rejected | capsem-gateway::terminal::tests |
| Rootfs read-only | profile rootfs asset mounted ro, guest binaries 555 | capsem-doctor in-VM tests |
| Suspend reports errors | IPC failure and timeout both return 500, not silent success | capsem-service tests |
If all test fixtures wait/poll before asserting, the tests will never catch server-side race conditions. For every endpoint that talks to a VM socket, write at least one test that calls it IMMEDIATELY after provision (no wait_exec_ready, no ready_vm fixture). The server must handle readiness internally.
Pattern to avoid (masks the bug -- server never needs wait logic because client always waits):
fixture calls provision -> fixture polls wait_exec_ready -> test calls exec
Required test pattern (catches the bug -- if server doesn't wait, test fails):
test calls provision -> test immediately calls exec -> server handles wait
See tests/capsem-service/test_svc_exec_ready.py for the regression tests that enforce this.
wait_exec_ready (in tests/helpers/service.py, tests/helpers/mcp.py, tests/capsem-gateway/test_gw_e2e.py) makes one exec call with the server-side timeout passed through. The server's handle_exec calls wait_for_vm_ready internally, which polls until the VM is ready. Do NOT add client-side retry loops -- that creates a double-wait where each retry can block for the full server timeout (30s client retries x 30s server wait = pathological cascade). One wait, one place.
tests/capsem-serial/test_boot_timing.py::test_exec_latency_within_gate asserts that provision-to-first-exec completes within EXEC_LATENCY_GATE. If this test fails, investigate boot time (process.log boot_timeline spans), not the wait mechanism.
tests.rs file, not inline mod tests { ... }. See the next subsection.crates/capsem-core/tests/guest/artifacts/diagnostics/test_*.py (see dev-testing-vm)frontend/src/lib/__tests__/ (see dev-testing-frontend)tests/test_*.pytests/capsem-*/ directories, each with its own conftest.py and pytest markertests/ironbank/ (black-box only; no Rustimplementation-derived expectations)
tests.rs patternEvery Rust module keeps its unit tests in a sibling tests.rs, not an inline mod tests { ... } block. The parent module declares:
// foo.rs OR foo/mod.rs
// ... production code ...
#[cfg(test)]
mod tests;
and the tests go in tests.rs in the same directory:
// tests.rs -- sibling of foo.rs or child of foo/
use super::*;
#[test]
fn roundtrip() { ... }
Why. Inline #[cfg(test)] mod tests { ... } blocks are appended at the bottom of prod files and commonly hit 50–99% of the file's line count. That means every Read, grep, and scroll to reach production code walks past thousands of test lines first. Several modules in this codebase hit 4,000+ lines that way before extraction. Agents and humans both read faster when prod code isn't buried.
Mechanics.
tests.rs is a submodule of the parent file -- use super::*; works, private items are visible, #[cfg(test)] on the mod tests; declaration still gates compilation.lib.rs, foo.rs), put tests.rs next to them in the same src/ directory.foo/mod.rs, put tests.rs inside foo/.mod tests block (e.g. #[allow(unused_imports)]) move onto the declaration: #[cfg(test)]\n#[allow(unused_imports)]\nmod tests;.Extraction recipe (for any remaining inline mod tests { ... }):
{ and }) into a new sibling tests.rs.#[cfg(test)] mod tests; (plus any attributes that were on the original).cargo test -p <crate> -- should pass identically.When to push back. If you see a new PR or agent output adding an inline mod tests { ... } block, request it be moved to tests.rs before merge. Exceptions are narrow: tiny helper modules under ~50 lines total where inline tests plus prod code fit on one screen, or a module that's already a test-only helper.
A Python contract asserting that some Rust test *exists* must read the sibling
tests.rs, never the production .rs the test moved out of. Use
tests/rust_sources.py:
from rust_sources import production, sibling_tests
assert "pub enum Status" in production(RELEASE_GRAPH) # prod symbol
assert "release_graph_enums_reject_unknown" in sibling_tests(RELEASE_GRAPH)
Keep the two sources separate. Several contracts assert a symbol is
*absent* from production ("Removed" not in source), and a test module
legitimately names the thing it proves is rejected -- concatenating them lets a
fixture falsify a claim about shipped code.
sibling_tests() resolves mod tests; the way Rust does (foo.rs →
foo/tests.rs; main.rs/lib.rs/mod.rs → tests.rs beside it) and raises
when the module is missing rather than passing on an empty string. The helper is
not named tests_of on purpose: pytest collects test*, so the obvious name
becomes a phantom failing test in every importer.
tests/test_rust_test_name_assertions.py enforces this repo-wide and fails in
seconds. It resolves each assertion's target through the AST, per function
scope, so a contract that legitimately names a relocated test while asserting it
against a test module or spec document is not flagged.
Why it matters. This layout change broke sixteen contracts under
tests/capsem-release/, then five more under tests/capsem-install/ that run
only inside the Docker install gate -- invisible until forty minutes into a
release run. Nothing about the failure pointed at a moved function; it read as a
broken release.
All Python integration tests live under tests/capsem-*/ and use pytest markers. Each suite has a dedicated just recipe.
| Suite | Directory | Marker | VM? | What it tests |
|-------|-----------|--------|-----|---------------|
| Service API | capsem-service/ | integration | Yes | HTTP endpoints: provision, list, info, exec, logs, file I/O, delete |
| CLI | capsem-cli/ | integration | Yes | CLI subcommands via subprocess |
| MCP | capsem-mcp/ | mcp | Yes | MCP server black-box (stdio, tool routing) |
| Session DB | capsem-session/ | session | Yes | Telemetry: net/model/tool/mcp/fs/snapshot events |
| Snapshots | capsem-snapshots/ | snapshot | Yes | Auto/manual snapshots, revert |
| Isolation | capsem-isolation/ | isolation | Yes | Multi-VM filesystem + network isolation |
| Security | capsem-security/ | security | Yes | Binary perms, codesigning, asset integrity, env blocklist |
| Config | capsem-config/ | config | Yes | Limits, resource bounds, hot-reload |
| Bootstrap | capsem-bootstrap/ | bootstrap | No | Setup flow, dev tools, asset checks |
| Stress | capsem-stress/ | stress | Yes | 5 concurrent VMs, rapid create/delete |
| Build chain | capsem-build-chain/ | build_chain | Yes | cargo build -> codesign -> pack -> manifest -> boot |
| Guest | capsem-guest/ | guest | Yes | Network, services, filesystem, env inside guest |
| Cleanup | capsem-cleanup/ | cleanup | Yes | Process killed, socket removed, session dir removed |
| Codesign | capsem-codesign/ | codesign | No | All binaries signed, entitlements present (FAIL not skip) |
| Serial | capsem-serial/ | serial | Yes | Console logs, boot timing < 30s |
| Session lifecycle | capsem-session-lifecycle/ | session_lifecycle | Yes | DB exists, schema, events, survives shutdown |
| Config runtime | capsem-config-runtime/ | config_runtime | Yes | CPU/RAM applied in guest, blocked domains |
| Recipes | capsem-recipes/ | recipe | No | just run-service, just doctor, cargo build |
| Recovery | capsem-recovery/ | recovery | Yes | Stale socket/instances, orphaned process, double service |
| Rootfs artifacts | capsem-rootfs-artifacts/ | rootfs | No | Artifact files, build context, doctor consistency |
| Session exhaustive | capsem-session-exhaustive/ | session_exhaustive | Yes | Per-table data validation, cross-table FK integrity |
| Install | capsem-install/ | install | No | Native package installer: layout, auto-launch, service install, manifest placement, update, uninstall, lifecycle, reinstall, error paths |
just test is the only public complete/release gate and just smoke is the
only public focused composite. Suite-specific and install/package rails are
private implementation details; run an individual pytest/cargo/pnpm command
directly for focused diagnosis instead of adding another public composite.
Public Just recipes, Capsem CLI command paths, and service HTTP method/path
pairs are exact approval-gated surfaces. Any change must pass
tests/test_public_surface_contract.py and requires explicit approval before
editing config/public-surface.toml.
Read references/test-matrix.md for the per-crate Rust CI matrix and the
Python suite map (which suites run versus collect in PR CI, smoke, and the full
gate).
| Component | Floor | Enforced | Where |
|-----------|------:|:--------:|-------|
| Rust workspace | 63% | --fail-under-lines 63 | CI (cargo llvm-cov), just test |
| Python selected CI suite | 85% | --cov-fail-under=85 | Ordinary CI |
| Python full suite | 85% | --cov-fail-under=85 | just test |
| capsem-service | 80% | Codecov component | codecov.yml |
| capsem-mcp | 80% | Codecov component | codecov.yml |
| capsem-gateway | 80% | Codecov component | codecov.yml |
| capsem (CLI) | 80% | Codecov component | codecov.yml |
cargo llvm-cov via just test (floor: 63% line coverage)just test suite both enforce 85%.codecov.yml maps components to code paths. Update it when files or directories are added, moved, or renamed.Read references/mcp-debug-tools.md for interactive VM debugging through the
capsem MCP server: tool table, one-shot vs iterative workflows, and common
session-DB queries. MCP tools are for fast targeted checks; just recipes are
for comprehensive validation before committing.
After any change touching guest binaries, network policy, telemetry, MCP, or VM lifecycle:
just exec "capsem-doctor" -- verifies sandbox integrity inside the VMpython3 scripts/check_session.py that net_events, model_calls, tool_calls, tool_responses, fs_events, dns_events, and security_rule_events are populated correctly for the exercised protocolsNever dismiss a test failure as "pre-existing" or "unrelated." Every failure must be investigated. Follow the dev-debugging workflow:
Never take the last line of a multi-part result as the result. Two shapes of
one mistake, both of which report success while the thing measured failed:
$? after a pipe is the pipe's status. just test | tail reports what
tail did. Redirect, then read the code separately:
just test > /tmp/gate.log 2>&1; echo "EXIT=$?"
tail -n1 across a multi-part result returns the last part, not the whole.
cargo test -p capsem-service runs three test binaries; the last prints
0 passed, so | tail -1 reads as though the crate had no tests while 91 and
264 passed above it. Aggregate instead of sampling:
cargo test -p capsem-service 2>&1 | grep -E "^test result:" # every binary
Both errors are silent and both flatter you: one turns a failed gate into a
pass, the other turns a passing crate into a phantom regression. If a command
can emit more than one verdict, read them all.
Read the *first* real error, not the recipe cascade under it — grep -aE "^FAILED|^E " lands on the cause, while the trailing error: Recipe ... failed lines are only the unwind.
tests/test_exit_status_integrity.py keeps this out of committed recipes,
scripts, and workflows, and requires set -o pipefail in any bash recipe that
pipes. It cannot see an agent's ad-hoc shell — that part is on you.
Redirect Capsem paths with paths::CapsemPathsGuard::redirect(root). It sets
CAPSEM_HOME, CAPSEM_RUN_DIR, and CAPSEM_ASSETS_DIR from one root and
restores on drop, so a fixture cannot set one and inherit the rest.
Read logs with telemetry::read_log_tail, including in assertions: a test that
opens a *.log path directly stops exercising what the product does the moment
that stream rotates.
Both are enforced by tests/test_path_and_log_wrappers_are_mandatory.py. See
/dev-rust-patterns "One rule, one function" for why.
just test exports CAPSEM_HOME, CAPSEM_RUN_DIR, CAPSEM_TEST_PROFILE, and
CAPSEM_BENCHMARK_OUTPUT_ROOT. A test that reads ambient state passes in your
shell and fails in the gate:
CAPSEM_HOME="$PWD/target/test-home/.capsem" \
CAPSEM_RUN_DIR="$PWD/target/test-home/.capsem/run" \
cargo test -p <crate>
A fixture that overrides CAPSEM_HOME must override CAPSEM_RUN_DIR too —
the run dir takes precedence over the home-derived default, so setting only the
first sends production code to the ambient run directory while the fixture
writes into a temp one. Bisect by exporting one variable at a time; that names
the culprit in two runs instead of guessing.
A number copied next to a rule drifts from it silently. Three separate gate failures in one session traced to this: a coverage floor asserted as 65 after it moved to 63, a guest kernel check demanding major >= 7 after the pin moved to 6.18, and a Docker fixture simulating 30 GiB free as "plenty" after the floor rose to 40.
Each read as a broken product, and each surfaced minutes-to-an-hour into a gate rather than at the edit. Derive the value from its source, or name it once and pin config and contract together:
floor = tomllib.loads(BUILD_CONFIG.read_text())["rails"]["assets"]["minimum_free_gib"]
ample_kib = (floor + 10) * 1024 * 1024 # follows the floor; never restates it
Prove it derives rather than hardcodes: change the source value and confirm the test *follows* instead of breaking.
cargo test --test platform_gating scans all .rs files under crates/ for macOS-only and Linux-only symbols (libc::clonefile, AppleVzHypervisor, KvmHypervisor, FICLONE, etc.) and verifies they appear inside #[cfg(target_os = "...")] blocks. This catches ungated platform APIs before they reach CI. Run this test when adding any platform-specific code.
Extract logic into capsem-core -- never embed business logic in the app layer where it's coupled to Tauri. If you can't test something without booting a VM or launching the GUI, it belongs in core.
Take google/dev-testing from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.