Algo Nlp Lda Agent Skill

Implement LDA topic modeling to discover latent topics in document collections. Use this skill when the user needs to extract topics from a text corpus, categorize documents by theme, or explore thematic structure — even if they say 'what are the main topics', 'topic extraction', or 'document clustering by theme'.

8k tokens

context cost

the whole folder, loaded on every use

files

instructions only

copies elsewhere

how many repositories repackaged it

223

stars on the repo

on the repository, not the skill itself

Install

one command, takes just this skill from the repository

npx skills add https://github.com/asgard-ai-platform/skills --skill algo-nlp-lda

What comes with it

28 673 bytes besides the instruction

examples/sample_scenario.md

references/advanced-lda.md

references/topic-evaluation.md

The instruction itself

14 sections, as written by the author

LDA Topic Modeling

Overview

Latent Dirichlet Allocation models each document as a mixture of topics and each topic as a distribution over words. Discovers K latent topics from a corpus without supervision. Uses Gibbs sampling or variational inference. Complexity: O(N × K × iterations) where N = total word tokens.

When to Use

Trigger conditions:

Discovering latent themes in a large document collection
Organizing/categorizing documents by automatically discovered topics
Exploratory text analysis when categories are unknown

When NOT to use:

When categories are known (use supervised classification)
For short texts (tweets, titles) — too few words per document for reliable topic assignment
When you need semantic understanding (use embeddings)

Algorithm

IRON LAW: The Number of Topics K Must Be Chosen, Not Discovered
LDA does NOT tell you how many topics exist. K is a hyperparameter.
Too few topics: overly broad, mixed themes. Too many: fragmented,
redundant topics. Use coherence score (C_v) to compare K values,
but the final choice requires human judgment on topic interpretability.

Phase 1: Input Validation

Preprocess: tokenize, remove stop words, apply lemmatization. Build document-term matrix. Filter: remove terms appearing in <5 or >50% of documents.

Gate: Clean DTM, vocabulary size reasonable (1K-50K terms).

Phase 2: Core Algorithm

Choose K (start with √(N/2), try range K=5,10,15,20,...)
Set hyperparameters: α = 50/K (document-topic density), β = 0.01 (topic-word density)
Run LDA (Gibbs sampling: 1000+ iterations, or variational inference)
Extract: topic-word distributions (top 10-20 words per topic) and document-topic distributions

Phase 3: Verification

Evaluate: topic coherence (C_v score, higher is better), manual inspection of top words per topic, check for "junk" topics (mixed incoherent words).

Gate: Coherence score acceptable, topics are humanly interpretable.

Phase 4: Output

Return topics with top words and document assignments.

Output Format

{
  "topics": [{"id": 0, "label": "finance", "top_words": ["revenue", "profit", "quarter", "growth"], "coherence": 0.55}],
  "doc_topics": [{"doc_id": "d1", "dominant_topic": 0, "topic_distribution": [0.7, 0.1, 0.2]}],
  "metadata": {"K": 10, "coherence_avg": 0.48, "documents": 5000, "vocabulary": 8000}
}

Examples

Sample I/O

Input: 1000 news articles, K=5

Expected: Topics like: {politics, sports, technology, business, entertainment} with coherent top words per topic.

Edge Cases

| Input | Expected | Why |

|-------|----------|-----|

| Very short documents | Poor topic assignment | Too few words for reliable mixture estimation |

| Homogeneous corpus | 1-2 topics dominate | All documents are similar, limited topic diversity |

| K=1 | Single topic = corpus vocabulary | Degenerate case, no discrimination |

Gotchas

Stop words MUST be removed: LDA will create "junk" topics dominated by common words ("the", "is", "and") if stop words remain.
Topic labeling is manual: LDA gives word distributions, NOT topic names. You must interpret and label topics based on top words.
Reproducibility: Gibbs sampling is stochastic. Different random seeds give different topics. Run multiple times and check stability.
Dynamic topics: Standard LDA assumes topics are static. For evolving corpora (news over years), use Dynamic Topic Models.
Hyperparameter sensitivity: Low α produces documents with fewer, more distinct topics. Low β produces topics with fewer, more specific words. Tune or use automatic methods.

References

For coherence metrics and K selection, see references/topic-evaluation.md
For dynamic and correlated topic models, see references/advanced-lda.md

Other skills for the same job

different authors, same section of the catalogue

Matplotlib

by christophacham

×3

Low-level plotting library for full customization. Use when you need fine-grained control over every plot element, creating novel plot types, or integrating with specific scientific workflows. Export to PNG/PDF/SVG for publication. For quick statistical plots use seaborn; for interactive plots use plotly; for publication-ready multi-panel figures with journal styling, use scientific-visualization.

Create stunning, animation-rich HTML presentations from scratch or by converting PowerPoint files. Use when the user wants to build a presentation, convert a PPT/PPTX to web, or create slides for a talk/pitch. Helps non-designers discover their aesthetic through visual exploration rather than abstract choices.

> This skill orchestrates autonomous discovery of brand materials across enterprise platforms (Notion, Confluence, Google Drive, Box, SharePoint, Figma, Gong, Granola, Slack). It should be used when the user asks to "discover brand materials", "find brand documents", "search for brand guidelines", "audit brand content", "what brand materials do we have", "find our style guide", "where are our brand docs", "do we have a style guide", "discover brand voice", "brand content audit", or "find brand assets".

6k tokens

Matplotlib Scientific Plotting

by BioTender-max

×1

Low-level Python plotting for scientific figures: publication-quality line, scatter, bar, heatmap, contour, 3D; multi-panel layouts; fine control of every element. PNG/PDF/SVG export. Use seaborn for quick stats, plotly for interactive.

4k tokens

Baoyu Format Markdown

by ComeOnOliver

×1

Formats plain text or markdown files with frontmatter, titles, summaries, headings, bold, lists, and code blocks. Use when user asks to "format markdown", "beautify article", "add formatting", or improve article layout. Outputs to {filename}-formatted.md.

Enterprise-grade PowerPoint deck generation system using evidence-based prompting techniques, workflow enforcement, and constraint-based design. Use when creating professional presentations (board decks, reports, analyses) requiring consistent visual quality, accessibility compliance, and integration of complex data from multiple sources. Implements html2pptx workflow with spatial layout optimization, validation gates, and multi-chat architecture for 30+ slide decks.

PPTX HTML Fidelity Audit

by nexu-io

Audit a python-pptx export against its source HTML deck, identify layout/content drift (footer overflow, cropped content, missing italic/em, lost styling, off-rhythm spacing), and re-export with strict footer-rail + cursor-flow layout discipline. Use this skill whenever the user has a .pptx that was generated from an HTML slide deck and asks to compare/audit/verify/fix the export — including phrases like "compare ppt with html", "fidelity audit", "fix the pptx", "ppt is cut off", "footer overlap", "italic missing in pptx", "re-export the deck", "pptx-html-fidelity-audit", or any case where a python-pptx → HTML round-trip needs verification or repair. Also trigger when the user shows you a deck.html and a deck.pptx side by side and is debugging visual differences.

15k tokens scripts

How to use it

Copy the folder

Take asgard-ai-platform/algo-nlp-lda from the repository into ~/.claude/skills for personal use, or into .claude/skills inside a project.

Check the name does not clash

The agent identifies a skill by the name field in its header. Two skills with the same name cannot sit side by side — one of them will be ignored.