Test Suite Prioritizer

# Recent commit changes
git diff --name-only HEAD~1 HEAD

# Uncommitted changes
git diff --name-only

# Changes in current branch vs main
git diff --name-only main...HEAD

# Files in last N commits
git diff --name-only HEAD~N HEAD

src/auth/login.py
src/payment/processor.py
src/models/user.py
tests/test_auth.py

Changed: src/auth/login.py
Covered by:
  - tests/test_auth.py::test_login_success
  - tests/test_auth.py::test_login_failure
  - tests/integration/test_user_flow.py::test_complete_registration
  - tests/e2e/test_authentication_flow.py::test_full_auth_cycle

Changed: src/payment/processor.py
Covered by:
  - tests/test_payment.py::test_process_payment
  - tests/test_payment.py::test_refund
  - tests/integration/test_checkout.py::test_complete_purchase

Priority Score = Base Score × Change Impact Factor

Base Score factors:
- Direct coverage of changed file: +10 points
- Indirect coverage (imports changed file): +5 points
- Integration test covering changed area: +3 points
- Test in same module: +2 points

Change Impact Factor (multiply):
- Critical path code (auth, payments): ×1.5
- Recently failed test: ×1.3
- High historical failure rate: ×1.2
- Core business logic: ×1.2
- UI/cosmetic changes: ×0.8

# Example: test_login_success
base_score = 10  # Directly tests login.py (changed)
impact_factor = 1.5  # Auth is critical path
priority_score = 10 × 1.5 = 15

# Example: test_user_flow
base_score = 5 + 3  # Imports login.py + integration test
impact_factor = 1.2  # Core business logic
priority_score = 8 × 1.2 = 9.6

# Example: test_ui_styling
base_score = 2  # Same module as changed file
impact_factor = 0.8  # UI/cosmetic
priority_score = 2 × 0.8 = 1.6

# Test Execution Priority List

## Summary
- Total tests analyzed: 45
- High priority (score ≥ 10): 8 tests
- Medium priority (score 5-9): 15 tests
- Low priority (score < 5): 22 tests
- Estimated time for high priority tests: ~2 minutes

## Recommended Execution Order

### Priority 1: Critical Tests (Run First)

1. **tests/test_auth.py::test_login_success** (Score: 15.0)
   - Directly tests modified file: src/auth/login.py
   - Critical path: Authentication
   - Reason: Core security functionality changed

2. **tests/test_auth.py::test_login_failure** (Score: 15.0)
   - Directly tests modified file: src/auth/login.py
   - Critical path: Authentication
   - Reason: Error handling for authentication changed

3. **tests/test_payment.py::test_process_payment** (Score: 15.0)
   - Directly tests modified file: src/payment/processor.py
   - Critical path: Payment processing
   - Reason: Payment logic modified, high business impact

### Priority 2: High Impact Tests

4. **tests/integration/test_user_flow.py::test_complete_registration** (Score: 9.6)
   - Indirect coverage: Imports src/auth/login.py
   - Integration test: End-to-end user flow
   - Reason: Validates auth changes in realistic scenario

5. **tests/integration/test_checkout.py::test_complete_purchase** (Score: 9.6)
   - Indirect coverage: Imports src/payment/processor.py
   - Integration test: Full checkout flow
   - Reason: Validates payment changes with real workflow

6. **tests/test_models.py::test_user_model** (Score: 7.0)
   - Related file: src/models/user.py modified
   - Dependency: Auth and payment depend on user model
   - Reason: Foundation for changed functionality

### Priority 3: Supporting Tests

7. **tests/test_validators.py::test_email_validation** (Score: 4.0)
   - Indirect dependency: Used by auth module
   - Reason: Input validation for authentication

8. **tests/e2e/test_authentication_flow.py::test_full_auth_cycle** (Score: 3.9)
   - E2E coverage: Complete authentication flow
   - Reason: Comprehensive validation but slow to execute

[... remaining tests with lower scores ...]

## Quick Commands

Run high priority tests only (8 tests, ~2 min):

Run critical tests only (3 tests, ~30 sec):

## Time Estimates

- Critical tests (3): ~30 seconds
- High priority tests (8): ~2 minutes
- All prioritized tests (23): ~5 minutes
- Full test suite (45): ~15 minutes

## Recommendations

1. **Pre-commit**: Run critical tests (score ≥ 15) before committing
2. **CI fast lane**: Run high priority tests (score ≥ 9) in first stage
3. **Full validation**: Run all tests in parallel after merge

## 1-Minute Quick Check (Top 3 Critical Tests)

pytest tests/test_auth.py::test_login_success \
       tests/test_auth.py::test_login_failure \
       tests/test_payment.py::test_process_payment

These cover the most critical changed functionality.

## Post-Refactor Validation

Changed file: src/utils/helpers.py (used by 25 modules)

Priority:
1. All tests directly importing helpers.py (18 tests)
2. Integration tests using helper functions (8 tests)
3. Tests in dependent modules (25 tests)

Reason: Wide-reaching refactor requires comprehensive validation.

## Note: Flaky Tests Detected

The following tests have high failure rates but low priority scores:
- tests/test_cache.py::test_concurrent_access (30% failure rate)
- tests/test_api.py::test_rate_limiting (25% failure rate)

Recommendation:
- Fix flaky tests before relying on prioritization
- Consider quarantining or mocking timing-dependent code

# .github/workflows/ci.yml example
jobs:
  quick-tests:
    name: Critical Tests (Fast Feedback)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run critical tests
        run: |
          # Run tests with score ≥ 15 first
          pytest -v tests/test_auth.py tests/test_payment.py

  full-tests:
    name: Complete Test Suite
    runs-on: ubuntu-latest
    needs: quick-tests  # Only run if critical tests pass
    steps:
      - uses: actions/checkout@v2
      - name: Run all tests
        run: pytest

# Pre-commit hook (.git/hooks/pre-commit)
#!/bin/bash

# Get changed files
changed_files=$(git diff --cached --name-only)

# Run prioritized tests for changed files
if echo "$changed_files" | grep -q "src/auth/"; then
    pytest tests/test_auth.py || exit 1
fi

if echo "$changed_files" | grep -q "src/payment/"; then
    pytest tests/test_payment.py || exit 1
fi

# pytest conftest.py - custom marker
import pytest

def pytest_collection_modifyitems(config, items):
    # Mark tests based on priority
    for item in items:
        if "test_auth" in item.nodeid or "test_payment" in item.nodeid:
            item.add_marker(pytest.mark.critical)

# Run only critical tests
# pytest -m critical

# Example: Analyze test failure history from CI
# tests with >10% historical failure rate get boosted priority

Historical Failures:
- test_concurrent_access: 12 failures / 100 runs = 12% → Priority boost ×1.2
- test_database_connection: 5 failures / 100 runs = 5% → No boost

# Find all test files
find . -name "test_*.py" -o -name "*_test.py"

# Get test functions
pytest --collect-only -q

# Analyze imports in tests
grep -r "^import\|^from" tests/

# Find test files
find . -name "*.test.js" -o -name "*.spec.js"

# Get test list
npm test -- --listTests

# Analyze coverage
npm test -- --coverage --coverageReporters=json

# Find test classes
find . -name "*Test.java"

# Run with verbose output
mvn test -X

# Get test reports
cat target/surefire-reports/*.xml

Top 3 tests to run (30 seconds):
1. tests/test_auth.py::test_login_success
2. tests/test_auth.py::test_login_failure
3. tests/test_auth.py::test_session_creation

Reasoning: Direct tests of changed functionality, fast execution.

Stage 1 - Critical Tests (2 min):
- All auth tests (5 tests)
- All payment tests (4 tests)
- Core user model tests (3 tests)

Stage 2 - Integration Tests (5 min):
- User flow integration (8 tests)
- API integration (6 tests)

Stage 3 - Full Suite (parallel, 10 min):
- Remaining tests (120 tests)

Reasoning: Fast feedback on critical paths, comprehensive validation in parallel.

Must-run tests (2 min):
1. Tests directly covering changed files (8 tests)
2. Integration tests for critical paths (4 tests)

Nice-to-run tests (3 min):
3. Related unit tests (15 tests)

Skip for now:
- E2E tests (too slow)
- Unrelated tests (no impact)

Reasoning: Maximum validation coverage in minimum time.