Test Quality Analysis Skill for Claude
Detect test smells, overmocking, flaky tests, and coverage issues. Analyze test effectiveness, maintainability, and reliability. Use when reviewing tests or improving test quality.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/secondsky/claude-skills --skill test-quality-analysisThe instruction itself
18 sections, as written by the authorTest Quality Analysis
Expert knowledge for analyzing and improving test quality - detecting test smells, overmocking, insufficient coverage, and testing anti-patterns.
Core Dimensions
- Correctness: Tests verify the right behavior
- Reliability: Tests are deterministic, not flaky
- Maintainability: Tests are easy to understand
- Performance: Tests run quickly
- Coverage: Tests cover critical code paths
- Isolation: Tests don't depend on external state
Test Smells
Overmocking
Problem: Mocking too many dependencies makes tests fragile.
// ❌ BAD: Overmocked
test('calculate total', () => {
const mockAdd = vi.fn(() => 10)
const mockMultiply = vi.fn(() => 20)
// Testing implementation, not behavior
})
// ✅ GOOD: Mock only external dependencies
test('calculate order total', () => {
const mockPricingAPI = vi.fn(() => ({ tax: 0.1 }))
const total = calculateTotal(order, mockPricingAPI)
expect(total).toBe(38)
})
Detection: More than 3-4 mocks, mocking pure functions, complex mock setup.
Fix: Mock only I/O boundaries (APIs, databases, filesystem).
Fragile Tests
Problem: Tests break with unrelated code changes.
// ❌ BAD: Tests implementation details
await page.locator('.form-container > div:nth-child(2) > button').click()
// ✅ GOOD: Semantic selector
await page.getByRole('button', { name: 'Submit' }).click()
Flaky Tests
Problem: Tests pass or fail non-deterministically.
// ❌ BAD: Race condition
test('loads data', async () => {
fetchData()
await new Promise(resolve => setTimeout(resolve, 1000))
expect(data).toBeDefined()
})
// ✅ GOOD: Proper async handling
test('loads data', async () => {
const data = await fetchData()
expect(data).toBeDefined()
})
Poor Assertions
// ❌ BAD: Weak assertion
test('returns users', async () => {
const users = await getUsers()
expect(users).toBeDefined() // Too vague!
})
// ✅ GOOD: Strong, specific assertions
test('creates user with correct attributes', async () => {
const user = await createUser({ name: 'John' })
expect(user).toMatchObject({
id: expect.any(Number),
name: 'John',
})
})
Analysis Tools
# Vitest coverage (prefer bun)
bun test --coverage
open coverage/index.html
# Check thresholds
bun test --coverage --coverage.thresholds.lines=80
# pytest-cov (Python)
uv run pytest --cov --cov-report=html
open htmlcov/index.html
Best Practices Checklist
Unit Test Quality (FIRST)
- [ ] Fast: Tests run in milliseconds
- [ ] Isolated: No dependencies between tests
- [ ] Repeatable: Same results every time
- [ ] Self-validating: Clear pass/fail
- [ ] Timely: Written alongside code
Mock Guidelines
- [ ] Mock only external dependencies
- [ ] Don't mock business logic or pure functions
- [ ] Use real implementations when possible
- [ ] Limit to 3-4 mocks per test maximum
Coverage Goals
- [ ] 80%+ line coverage for business logic
- [ ] 100% for critical paths (auth, payment)
- [ ] All error paths tested
- [ ] Boundary conditions tested
Test Structure (AAA Pattern)
test('user registration', async () => {
// Arrange
const userData = { email: '[email protected]' }
// Act
const user = await registerUser(userData)
// Assert
expect(user.email).toBe('[email protected]')
})
Code Review Checklist
- [ ] Tests verify behavior, not implementation
- [ ] Assertions are specific and meaningful
- [ ] No flaky tests (timing, ordering issues)
- [ ] Proper async/await usage
- [ ] Test names clearly describe behavior
- [ ] Minimal code duplication
- [ ] Critical paths have tests
- [ ] Both happy path and error cases covered
Common Anti-Patterns
Testing Implementation Details
// ❌ BAD
const spy = vi.spyOn(Math, 'sqrt')
calculateDistance()
expect(spy).toHaveBeenCalled() // Testing how, not what
// ✅ GOOD
const distance = calculateDistance({ x: 0, y: 0 }, { x: 3, y: 4 })
expect(distance).toBe(5) // Testing output
Mocking Too Much
// ❌ BAD
const mockAdd = vi.fn((a, b) => a + b)
// ✅ GOOD: Use real implementations
import { add } from './utils'
// Only mock external services
const mockPaymentGateway = vi.fn()
See Also
vitest-testing- TypeScript/JavaScript testingplaywright-testing- E2E testingmutation-testing- Validate test effectiveness
Other skills for the same job
different authors, same section of the catalogue
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup
Use when implementing any feature or bugfix, before writing implementation code
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
Expert guidance for systematic backtesting of trading strategies. Use when developing, testing, stress-testing, or validating quantitative trading strategies. Covers "beating ideas to death" methodology, parameter robustness testing, slippage modeling, bias prevention, and interpreting backtest results. Applicable when user asks about backtesting, strategy validation, robustness testing, avoiding overfitting, or systematic trading development.
Cloud laboratory platform for automated protein testing and validation. Use when designing proteins and needing experimental validation including binding assays, expression testing, thermostability measurements, enzyme activity assays, or protein sequence optimization. Also use for submitting experiments via API, tracking experiment status, downloading results, optimizing protein sequences for better expression using computational tools (NetSolP, SoluProt, SolubleMPNN, ESM), or managing protein design workflows with wet-lab validation.
This skill should be used for time series machine learning tasks including classification, regression, clustering, forecasting, anomaly detection, segmentation, and similarity search. Use when working with temporal data, sequential patterns, or time-indexed observations requiring specialized algorithms beyond standard ML approaches. Particularly suited for univariate and multivariate time series analysis with scikit-learn compatible APIs.
How to use it
Take secondsky/test-quality-analysis from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.