Run Solution Analyzer Agent Skill

Run Solution Analyzer

This skill covers running the Solutions Analyzer locally against this clone of

Azure-Sentinel. Each script writes CSVs or docs to a directory you choose; the

scripts themselves are agnostic to where that directory lives.

If your workflow includes publishing the generated CSVs/docs to additional

repositories, GitHub Pages, or Kusto, those targets are user-specific and

should be recorded in agent memory (workspace or user scope) rather than in

this skill.

Scripts Overview

| Script | Purpose | Invocation |

| ------------------------------------ | ------------------------------------------------------------------ | ------------------------------------------------- |

| map_solutions_connectors_tables.py | Mapper: generates all CSV data | Run directly |

| generate_connector_docs.py | Doc generator: markdown + HTML pages, static + interactive indexes | Run directly |

| generate_asim_browser.py | ASIM browser: interactive HTML schema browser | Run directly |

| upload_to_kusto.py | Upload CSVs to Azure Data Explorer (Kusto) cluster | Run directly |

| collect_table_info.py | Fetch table reference data from Microsoft docs | Auto-invoked by mapper (--force-refresh=tables) |

| collect_asim_fields.py | Fetch ASIM field/schema definitions from Microsoft docs | Auto-invoked by mapper (--force-refresh=asim) |

| generate_interactive_docs.py | Generate index.html and HTML entity pages | Auto-invoked by generate_connector_docs.py |

All scripts run from Tools/Solutions Analyzer.

Default Output Location

By default the mapper writes CSVs into Tools/Solutions Analyzer/ in the

current branch. The doc generator and ASIM browser require an explicit

--output-dir so they never pollute the source tree.

Sync the Working Branch With origin/master

Always do this before any analyzer run. A working branch that has drifted

from master makes the mapper analyze a stale source tree — every connector

merged to master after the divergence point can appear as a new "added"

connector in subsequent diffs, even though no mapper logic changed.

git fetch origin master
git merge origin/master --no-edit

• Use git merge, not git rebase, so subsequent

git log --first-parent origin/master analysis (used by compare_runs.py

and friends) still reflects merge dates correctly.

• Resolve any conflicts before running the mapper.
• Do this even for incremental/docs-only runs: docs link to GitHub paths that

may have moved.

After merging, verify the working tree is clean of unintended changes:

git status --short | Select-Object -First 20

Cache Invalidation Model

The mapper's analysis cache is keyed by (analysis_type, file_path) and

stores the source file mtime. When a source file is modified (including via

git merge), its mtime changes and the entry is automatically re-analyzed on

the next run. Source-content changes do NOT require --force-refresh.

--force-refresh exists for two distinct reasons:

• Analyzer logic changed (mapper code modified) — invalidate cached

results for the affected analysis types so they get recomputed from current

source. This is the local-only case.

• Network-fetched data is stale — re-fetch tables / marketplace / ASIM

schema fields from external sources. This is the network case and is

reserved for explicit full refresh.

| Type | Triggers network fetch? | When to refresh |

| ------------- | ---------------------------------------------------------- | ------------------------------------------------------ |

| parsers | No | Mapper parser-analysis logic changed |

| solutions | No | Mapper solution-analysis logic changed |

| standalone | No | Mapper standalone-content logic changed |

| asim | Yes — re-runs collect_asim_fields.py against MS docs | ASIM analyzer logic changed AND network refresh wanted |

| marketplace | Yes — Azure Marketplace API | Marketplace data is stale |

| tables | Yes — re-runs collect_table_info.py against MS docs | Table reference info is stale |

Note: asim is hybrid — it invalidates the local ASIM-parser analysis cache

and re-runs collect_asim_fields.py. There is currently no flag to do

only the local half.

After a git merge from master

Default: just run the mapper with no flags. Mtime-based invalidation re-analyzes

every file that changed during the merge:

python map_solutions_connectors_tables.py

Network-fetched CSVs (tables_reference.csv, asim_fields.csv,

asim_entity_fields.csv, asim_logical_types.csv,

asim_vendors_products.csv, marketplace cache) will be reused from cache.

That's expected — they shouldn't drift just because master moved forward.

Full Mapper Run (Invalidating All Caches)

Use only when explicitly asked to do a full/clean run, when "invalidate all

caches" is requested, or when analyzer logic was modified across multiple

analysis types.

cd "Tools/Solutions Analyzer"
python map_solutions_connectors_tables.py --force-refresh=all

• --force-refresh=all clears all 6 cache types (asim, parsers, solutions,

standalone, marketplace, tables) AND re-runs collect_asim_fields.py and

collect_table_info.py against the network.

• --force-refresh=all-offline clears parsers, solutions, standalone,

AND asim (which still re-runs collect_asim_fields.py despite the name;

only marketplace/tables are skipped).

• Multiple types: --force-refresh=parsers,solutions,standalone (purely

local, no network calls).

• Run with isBackground: false and timeout: 0.
• Do NOT truncate/filter output (no Select-Object piping).

Known issue (as of 2026-04-28): collect_asim_fields.py fails with HTTP

404 because the Microsoft Sentinel docs were migrated out of

MicrosoftDocs/azure-docs. The mapper logs a warning and proceeds, leaving

asim_fields.csv, asim_entity_fields.csv, asim_logical_types.csv, and

asim_vendors_products.csv empty/stale. Until the script is updated to fetch

from the new source, avoid --force-refresh=all and --force-refresh=asim

unless you accept stale ASIM schema CSVs.

Generate Documentation

python generate_connector_docs.py --output-dir <DOCS_DIR> [--skip-input-generation] [--html-output-dir <HTML_DIR>] [--html-docs-path <REL_PATH>] [--html-index-url <URL>]

• --output-dir — mandatory, never omit.
• --skip-input-generation — skip the embedded mapper invocation when CSVs

are already fresh.

• --html-output-dir, --html-docs-path, --html-index-url — control where

index.html and the interactive entity pages are written and how they

link back to the index. These are typically only set when publishing to a

static-site / GitHub Pages location; defaults are sensible for a local

preview.

• Run with isBackground: false and timeout: 0. Do NOT truncate output.

Generate ASIM Browser

python generate_asim_browser.py --output-dir <DIR> [--docs-base-path <REL_PATH>] [--index-url <URL>] [--link-extension <EXT>]

--docs-base-path and --index-url make sense only when the browser is

deployed alongside an interactive docs index. For a standalone local

preview, --output-dir alone is enough.

When to Re-Run the Mapper Before Docs

Run mapper before generating docs if:

• The mapper script itself was modified.
• Any override YAML file in the overrides/ folder was modified.
• You specifically need to refresh the CSV data.
• You are explicitly asked to run the mapper.

Otherwise, just run the doc generator with --skip-input-generation.

Caching and Logging

• Cache: Tools/Solutions Analyzer/.cache/
• Logs: Tools/Solutions Analyzer/.logs/
• Log file: Tools/Solutions Analyzer/.logs/map_solutions_connectors_tables.log

Watching Mapper Progress While It Runs

The mapper writes timestamped progress lines ([ N.Ns] ...) to its own log

file in real time. To see live status during a run, tail that file directly:

Get-Content "Tools/Solutions Analyzer/.logs/map_solutions_connectors_tables.log" -Tail 12

**Do NOT rely on ... | Tee-Object -FilePath <log> | Select-Object -Last N

for live progress.** PowerShell pipelines buffer until the upstream command

finishes, so the Tee'd file stays empty until the entire run completes

(including the post-processing/CSV-write phase, which can be many minutes

after the per-parser progress lines were emitted). The mapper's own

.logs/map_solutions_connectors_tables.log is unbuffered and updates within

seconds.

Upload to Kusto

python upload_to_kusto.py -c <cluster-url> -d <database> --solution-analyzer --source-dir ./

• --solution-analyzer uploads the standard set of CSVs defined in

SOLUTION_ANALYZER_FILES.

• --source-dir ./ uses local CSVs instead of downloading from GitHub.
• Without --source-dir, downloads CSVs from the public Azure-Sentinel

GitHub repo.

• --dry-run shows what would be uploaded without making changes.
• Requires Azure CLI authentication (az login).

The cluster URL and database name are environment-specific and not part of

this skill.

User-Specific Publishing Pipelines

If you maintain a publishing pipeline (e.g. CSV output worktree, separate

docs repository, GitHub Pages site, specific Kusto cluster), record the

concrete paths, URLs, and the order of copy/commit/push steps in agent

memory — for example, in /memories/repo/ for a workspace-scoped note or in

/memories/ for a cross-workspace one. The agent should consult that memory

to resolve any placeholders above before running.