Mutation testing ile test suite kalitesini olc. Stryker, mutmut, go-mutesting destegi.
npx skills add https://github.com/vibeeval/vibecosystem --skill mutation-testing
Mutation testing, test suite'inin kalitesini olcen bir tekniktir. Kaynak kodda kucuk degisiklikler (mutasyonlar) yapilir ve testlerin bu degisiklikleri yakalayip yakalamadigina bakilir.
Code coverage "kodun ne kadari calistiriliyor?" sorusunu yanitlar.
Mutation testing "testler gercekten bir seyi kontrol ediyor mu?" sorusunu yanitlar.
%100 code coverage'a sahip ama assertion'i olmayan testler mutation testing'de FAIL alir.
# Install
npm install --save-dev @stryker-mutator/core
npx stryker init
# Jest runner
npm install --save-dev @stryker-mutator/jest-runner
# Vitest runner
npm install --save-dev @stryker-mutator/vitest-runner
# TypeScript support
npm install --save-dev @stryker-mutator/typescript-checker
Config (stryker.config.mjs):
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
export default {
mutate: [
'src/**/*.ts',
'!src/**/*.test.ts',
'!src/**/*.spec.ts',
'!src/**/*.d.ts',
'!src/**/index.ts'
],
testRunner: 'jest',
checkers: ['typescript'],
reporters: ['html', 'clear-text', 'progress', 'json'],
coverageAnalysis: 'perTest',
thresholds: {
high: 80,
low: 60,
break: null // Set to 60 to fail CI on low kill ratio
},
timeoutMS: 60000,
concurrency: 4
};
Run:
npx stryker run
# Report: reports/mutation/mutation.html
pip install mutmut
Config (pyproject.toml):
[tool.mutmut]
paths_to_mutate = "src/"
tests_dir = "tests/"
runner = "python -m pytest -x --tb=short -q"
dict_synonyms = "Struct,NamedStruct"
Run:
# Full run
mutmut run
# Results
mutmut results
# Show specific mutant
mutmut show 42
# HTML report
mutmut html
go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest
Run:
# Full run
go-mutesting ./...
# Specific package
go-mutesting ./pkg/calculator/...
# With score threshold
go-mutesting --score 0.8 ./...
a + b -> a - b, a * b, a / b
a * b -> a / b, a + b
a++ -> a--
Neyi test eder: Matematiksel hesaplamalarin dogrulugu
a > b -> a >= b
a < b -> a <= b
a >= b -> a > b
a <= b -> a < b
Neyi test eder: Boundary condition'lar, off-by-one hatalari
true -> false
a && b -> a || b
a || b -> a && b
!a -> a
Neyi test eder: Boolean logic, branch coverage
if (condition) -> if (!condition)
while (x > 0) -> while (x <= 0)
Neyi test eder: Kontrol akisinin dogrulugu
return x -> return 0
return true -> return false
return "hello" -> return ""
return obj -> return null
Neyi test eder: Return value assertion'lari
"hello" -> ""
"hello" -> "Stryker was here!"
Neyi test eder: String handling, empty string kontrolu
doSomething(); -> (removed)
x = calculate() -> (removed)
Neyi test eder: Side effect'lerin test edilip edilmedigi
| Seviye | Kill Ratio | Anlami |
|--------|-----------|--------|
| Mukemmel | 90%+ | Test suite cok guclu |
| Iyi | 80-89% | Kabul edilebilir, kucuk iyilestirmeler |
| Orta | 60-79% | Ciddi iyilestirme gerekli |
| Zayif | < 60% | Test suite guvenilemez |
Hedef: Her projede minimum %80 kill ratio
Bir mutant survive ettiyse su adimlari takip et:
Dosya: src/calculator.ts:15
Original: if (balance > 0) { ... }
Mutant: if (balance >= 0) { ... }
Durum: SURVIVED
balance === 0 durumunu test etmiyorit('should handle zero balance', () => {
const result = processBalance(0);
expect(result).toBe('no_funds'); // Bu test mutant'i oldurur
});
npx stryker run --mutate "src/calculator.ts"
Survived mutant > -> >= ise:
// Her boundary icin 3 test yaz: altinda, ustunde, tam sinirda
it('rejects when below minimum', () => expect(validate(-1)).toBe(false));
it('rejects at exact minimum', () => expect(validate(0)).toBe(false));
it('accepts above minimum', () => expect(validate(1)).toBe(true));
Survived mutant return x -> return 0 ise:
// Testlerde return value'yu MUTLAKA assert et
const result = calculate(5, 3);
expect(result).toBe(8); // Spesifik deger kontrolu
Survived mutant && -> || ise:
// Her boolean kombinasyonu test et
it('fails when only A is true', () => expect(check(true, false)).toBe(false));
it('fails when only B is true', () => expect(check(false, true)).toBe(false));
it('passes when both are true', () => expect(check(true, true)).toBe(true));
it('fails when both are false', () => expect(check(false, false)).toBe(false));
Survived mutant statement removal ise:
// Side effect'leri de test et
calculate(5);
expect(mockLogger.info).toHaveBeenCalledWith('Calculated: 5');
expect(mockMetrics.increment).toHaveBeenCalledWith('calculations');
Survived mutant !x -> x ise:
// Her iki yolu da test et
it('handles truthy input', () => expect(process(true)).toBe('A'));
it('handles falsy input', () => expect(process(false)).toBe('B'));
name: Mutation Testing
on:
pull_request:
branches: [main]
schedule:
- cron: '0 2 * * 0' # Haftalik tam tarama
jobs:
mutation-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- run: npm ci
- run: npx stryker run
- uses: actions/upload-artifact@v4
with:
name: mutation-report
path: reports/mutation/
- name: Check kill ratio
run: |
SCORE=$(cat reports/mutation/mutation.json | jq '.schemaVersion' -r)
# Custom threshold check script
mutation-test:
stage: test
script:
- npm ci
- npx stryker run
artifacts:
paths:
- reports/mutation/
expire_in: 7 days
only:
- merge_requests
allow_failure: true # Ilk baslarken, sonra kaldir
PR'larda sadece degisen dosyalari mutate et:
- name: Get changed files
id: changed
run: |
FILES=$(git diff --name-only origin/main...HEAD -- '*.ts' | grep -v test | tr '\n' ',')
echo "files=$FILES" >> $GITHUB_OUTPUT
- name: Run incremental mutation
if: steps.changed.outputs.files != ''
run: npx stryker run --mutate "${{ steps.changed.outputs.files }}"
Sadece degisen dosyalari mutate et:
# Stryker
npx stryker run --mutate "src/changed-file.ts"
# mutmut
mutmut run --paths-to-mutate src/changed_module/
Stryker'da coverageAnalysis: 'perTest' kullan. Her mutant sadece ilgili testlerle calistirilir.
Sonsuz donguye giren mutant'lar icin makul timeout:
timeoutMS: 60000, // 60 saniye max
timeoutFactor: 1.5 // Normal surenin 1.5 kati
CPU sayisina gore paralel calistir:
concurrency: 4 // veya os.cpus().length - 1
Onceki sonuclari cache'le:
incremental: true,
incrementalFile: 'reports/stryker-incremental.json'
Bazi mutasyonlar kodun davranisini degistirmez:
// Original
const i = 0;
// Mutant (equivalent - davranis ayni)
const i = -0;
Cozum: Equivalent mutant'lari rapordan cikar, survived olarak sayma.
while (true) veya for(;;) gibi durumlar:
Cozum: Timeout ayarini dogru yap, timeout mutant'larini "killed" say.
Buyuk codebase'lerde saatlerce surebilir:
Cozum: Incremental mode, per-test coverage, parallelism kullan.
Mutant, baska testleri de etkiler:
Cozum: Testlerin bagimsiz oldugunu dogrula, shared state kullanma.
Flaky testler mutant'lari yanlis killed gosterebilir:
Cozum: Once flaky testleri duzelt, sonra mutation test calistir.
Config dosyalarini mutate etmenin anlami yok:
Cozum: mutate pattern'indan config, constants, types dosyalarini haric tut.
Bu skill su durumlarda aktive olur:
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup
Use when implementing any feature or bugfix, before writing implementation code
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
Expert guidance for systematic backtesting of trading strategies. Use when developing, testing, stress-testing, or validating quantitative trading strategies. Covers "beating ideas to death" methodology, parameter robustness testing, slippage modeling, bias prevention, and interpreting backtest results. Applicable when user asks about backtesting, strategy validation, robustness testing, avoiding overfitting, or systematic trading development.
Cloud laboratory platform for automated protein testing and validation. Use when designing proteins and needing experimental validation including binding assays, expression testing, thermostability measurements, enzyme activity assays, or protein sequence optimization. Also use for submitting experiments via API, tracking experiment status, downloading results, optimizing protein sequences for better expression using computational tools (NetSolP, SoluProt, SolubleMPNN, ESM), or managing protein design workflows with wet-lab validation.
This skill should be used for time series machine learning tasks including classification, regression, clustering, forecasting, anomaly detection, segmentation, and similarity search. Use when working with temporal data, sequential patterns, or time-indexed observations requiring specialized algorithms beyond standard ML approaches. Particularly suited for univariate and multivariate time series analysis with scikit-learn compatible APIs.
Take vibeeval/mutation-testing from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.
The instructions reference pip, npm, npx, go.
Without those the skill loads but fails at the first command.