Find Untested Sources Agent Skill

Find Untested Sources

Purpose

Coverage tools answer "which lines were executed?" — they require a green build

and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The

question this skill answers is different and much cheaper:

> _Which source files have no test file referencing any of their declared

> types/symbols?_

That's the question an agent asks before writing a new test — and it can be

answered statically in a few seconds by parsing source files, with **no build,

no dependency resolution, and no compilation**. The output is a deterministic

test-pairing map that lets the agent pick the next file to test without reading

the entire codebase first.

Two engines — pick one

This skill ships two interchangeable analyzers with a compatible JSON contract:

| Engine | Script | Use when |

|--------|--------|----------|

| Roslyn (C#) | scripts/Find-UntestedSources.cs | The repo is .NET-only. Parses every .cs file with the Roslyn syntax API and does strict namespace disambiguation, so it is materially more accurate on duplicated short names like Settings or Context. |

| tree-sitter (polyglot) | scripts/find_untested_sources.py | The repo is not exclusively C#, or you want one tool across Python, TypeScript/JavaScript, Go, Java, Rust, Ruby, and C#. |

For a .NET-only repository, prefer the Roslyn engine — its namespace-aware

pairing beats the polyglot engine's identifier overlap.

Required workflow

• Use the narrowest repository or package root named by the caller. Do not scan

a parent workspace when the request identifies a subdirectory.

• Execute the appropriate analyzer once. Do not replace analyzer execution with

manual globbing, filename matching, or visual inspection.

For polyglot analysis, pass --include-tested when the answer must distinguish

paired sources from unpaired sources.

• Base the result on the analyzer's JSON. Preserve its paired/unpaired

classification and suggested relative path; do not guess a different path.

• When the caller named a subdirectory, prefix analyzer-relative paths with

that subdirectory so reported paths are workspace-relative.

• Report the requested result plus the static-pairing coverage caveat. Do not

append build, package-install, test-run, or coverage commands. When paired

sources exist, name their covering test files so the unpaired classification

is auditable.

When to Use

• User asks "where should I add tests?", "which files have no tests?", "find

untested code", "give me a test gap list", "what's the next file to test".

• Before invoking a test-generation agent, to produce a prioritized worklist.
• After generating tests, to verify each new test file pairs to a source file.
• To enumerate "weakly paired" source files (only one referring test) for

follow-up depth checks.

When Not to Use

• Line/branch coverage — use coverage-analysis.
• CRAP-score / risk hotspots — use coverage-analysis.
• Are existing tests strong? — use test-gap-analysis (mutation reasoning)

or assertion-quality.

Roslyn engine (C#)

Prerequisites

• .NET SDK that supports file-based apps (dotnet run script.cs). Pinned in the

repo's global.json (SDK 11 preview or later).

• No internet access required beyond the initial NuGet restore of

Microsoft.CodeAnalysis.CSharp on first run.

Usage

# From the skill folder
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> [--top N]

# Save the report
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> > pairing.json

# Iterate the untested list, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested | Select-Object -First 10 source, decl_count, suggested_test_path

Diagnostics go to stderr; JSON goes to stdout.

Output schema

{
  "repo": "<absolute path>",
  "elapsed_ms": 8883,
  "counts": {
    "source_files": 3036,
    "test_files": 867,
    "untested_files": 1852,
    "paired_files": 1184
  },
  "untested": [
    {
      "source": "src/Foo/Bar.cs",
      "decl_count": 8,            // # of type declarations in the file
      "suggested_test_path":      // mirror of source under a discovered test project
        "tests/Foo.Tests/Bar/BarTests.cs"
    }
  ],
  "source_to_tests": {
    "src/Foo/Baz.cs": [
      "tests/Foo.Tests/BazTests.cs",
      "tests/Foo.IntegrationTests/Scenarios/BazScenarios.cs"
    ]
  }
}

How it works

• File discovery — recursive walk pruning bin/, obj/, node_modules/,

.git/, .vs/, packages/, and any dotted subdir. Skips generated files

(.g.cs, .Designer.cs, .AssemblyInfo.cs).

• Test vs source classification — walks up to the nearest .csproj and

marks it a test project if the project name ends in .Tests, .Test,

.UnitTests, .IntegrationTests, .E2E, .EndToEnd, .Spec, .Specs, or

the content references Microsoft.NET.Test.Sdk, MSTest.Sdk,

Microsoft.Testing.Platform, xunit, NUnit, TUnit, or

<IsTestProject>true</IsTestProject>.

• Source index (parallel) — parse each source file with

CSharpSyntaxTree.ParseText (syntax only, no compilation); record every

BaseTypeDeclarationSyntax / DelegateDeclarationSyntax as

(ShortName, EnclosingNamespace, FilePath).

• Test scan (parallel) — parse each test file, collect using directives +

enclosing namespace, walk every IdentifierToken, look it up in the

short-name index, and disambiguate strictly: an identifier is attributed

only if the declaration's namespace matches one of the test file's using

directives, the enclosing namespace, or a prefix of them. This avoids noise

where common names like Settings or Context match every project.

• Pairing & suggestion — invert into source → [tests]. Build a

production-to-test project map from <ProjectReference> entries; for each

untested source, mirror its in-project relative path under the referencing

test project to suggest a path.

• JSON emit — ordered by declaration count desc, then alphabetical.

Polyglot engine (tree-sitter)

Prerequisites

• Python 3.10+.
• pip install tree-sitter-language-pack (single self-contained wheel that

bundles parsers for 300+ languages and the high-level process() API). No

native build, no per-language grammar install.

Usage

# From the skill folder
python scripts/find_untested_sources.py <repo-root>

# Restrict to a language (repeatable)
python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript

# Truncate the report (top 20 by declared API surface)
python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json

# Iterate, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path

Pass --include-tested to additionally emit tested_sources (omitted by

default to keep the payload small for LLM consumption). Diagnostics go to

stderr; JSON goes to stdout.

Output schema

{
  "repo_root": "<absolute path>",
  "summary": {
    "source_files": 3138,
    "test_files": 761,
    "tested_source_files": 1419,
    "untested_source_files": 1719,
    "orphan_test_files": 15,
    "languages": ["csharp"]
  },
  "untested_sources": [
    {
      "path": "src/Foo/Bar.cs",
      "language": "csharp",
      "declaration_count": 8,
      "declarations": ["Bar", "BarOptions", "IBar", "..."],
      "suggested_test_path": "src/Foo/BarTests.cs"
    }
  ],
  "orphan_tests": [
    { "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }
  ]
}

How it works

• File discovery — recursive walk pruning common build/vendor dirs (bin,

obj, node_modules, target, dist, build, vendor, __pycache__,

.venv, .git, …) and generated files (.d.ts, .g.cs, .Designer.cs,

_pb2.py, *.min.js, AssemblyInfo.cs, …).

• Language detection — detect_language_from_path maps the extension to a

supported language; unknown extensions are skipped.

• Test-vs-source classification — per-language path heuristics:

| Language | Test rule |

|---|---|

| Python | path contains tests//test/; or filename starts with test_ or ends _test.py; or conftest.py. |

| JS/TS/TSX | path contains __tests__, tests, test, spec, e2e; or filename contains .test./.spec.. |

| Go | filename ends _test.go. |

| Java | path contains test/tests; or filename ends Test.java/Tests.java. |

| Rust | path contains tests//benches/. |

| C# | path contains tests/; or project segment ends .Tests/.Test/.UnitTests/.IntegrationTests; or filename ends Tests/Test. |

| Ruby | path contains spec//test/; or filename ends _spec.rb/_test.rb. |

• Per-file extraction — `process(text, ProcessConfig(structure, imports,

symbols))` returns declared items, raw import statements, and a flat declared

-name list.

• Pairing — for each test file, union import resolution (per language,

e.g. Python from pkg.mod import x → pkg/mod.py; Java import a.b.C; →

a/b/C.java; C# using is namespace-not-file, so a no-op) with **identifier

overlap** (word-like tokens, length ≥ 4, matched against declared names).

• JSON emit — untested_sources ordered by declaration count descending.

Limitations (be honest with the agent)

Both engines are static, parse-only heuristics that trade a little accuracy for

orders-of-magnitude lower cost than coverage. Known gaps:

• Reflection / DI-resolved types referenced only via a string name or

container resolution won't be detected — the type's short name never appears

in the test source.

• Extension methods invoked as instance methods (C#): the declaring static

class is not named, so its file is not credited.

• var, target-typed new(), pattern matching lose the type token; the

file-level union usually still catches it through other references.

• Short identifier names (polyglot, < 4 chars) are dropped to avoid noisy

pairings on names like id, db, Tag.

• Monorepo path aliases (TS path mapping, Java module-info) are not

resolved; a suffix-match fallback may pick the wrong source if two files share

a trailing path segment.

For these cases, run actual coverage (coverage-analysis) on the unpaired

candidates the agent has already triaged.

Always label the final result as a static pairing heuristic, not evidence of

line or branch coverage. Include that caveat even when every requested source

file has an obvious matching or missing test.

Outputs the agent should consume

• untested[*].source / untested_sources[*].path — pick the next source file

to test (highest declaration count first).

• *.suggested_test_path — drop-in target for the new test file; the Roslyn

engine honors the test project that already <ProjectReference>s the source's

project, so dotnet sln add is not needed. The polyglot engine may suggest a

co-located test when no test root is discoverable; that is a valid fallback,

but prefer an established repository test directory when one exists.

• source_to_tests (Roslyn) / --include-tested tested_sources (polyglot) —

verify a newly written test file lands in the list for the intended source.

• orphan_tests (polyglot) — tests that don't reference any same-language

source file; useful for triaging stale or integration-only tests.