Script Aware Romanization Pipeline Agent Skill

A script-aware romanization pipeline

One entry point takes a line of text and a set of enabled languages, and returns a Latin-script

reading of that line or null:

// adapted — LyricsRomanizer.romanize's multi-line signature, collapsed to one line here
fun romanize(line: String, enabled: Set<RomanizationLanguage>): String?

Everything interesting is in how it decides *which* of several very different transliteration

strategies to run, per line, not per song: a lyric sheet routinely interleaves an original-script

line with a plain-English one, and romanizing the English half produces nonsense. Detection therefore

runs fresh on every line, and the twelve supported languages split cleanly into two families —

arithmetic/table scripts that need nothing beyond commonMain, and two scripts (Japanese, Chinese)

whose readings genuinely depend on surrounding words and therefore need a real dictionary behind an

expect/actual boundary. Getting that boundary as narrow as possible is most of the design.

Traps

**Detecting script by Unicode code-block ranges works — but only in a specific order, not any

order.** A block-range test (char in '぀'..'ヿ' for kana, and so on for Han, Hangul,

Devanagari, Gurmukhi, Cyrillic) needs no ICU and no locale, but a Japanese line is written mostly in

kanji with only a few kana particles — counting characters and taking a majority calls it Chinese.

The fix is a priority order, not a count: kana anywhere at all proves the line is Japanese (Chinese

text contains none), so the kana check must run, and win, before the Han check. Any per-script

detector built from "which characters are present" needs this kind of precedence rule wherever two

scripts' ranges can co-occur in the same line.

**One Unicode block can hold several languages that romanize differently, and the block alone cannot

tell them apart.** Seven Cyrillic-alphabet languages here share one range. Resolving *which* one a

line is in has to come from letters each language has that the others do not — ordered

most-distinctive first: two languages that share almost their whole alphabet except for one

language's few unique letters must have those unique letters checked before the shared ones, or the

shared letters match first and the unique ones never get a chance. Skip the disambiguation step

entirely when only one candidate language is enabled — there is nothing to guess, and running a

letter-frequency heuristic on a known-single-language line only risks getting it wrong for no reason.

When nothing distinctive matches, fall back to a single hardcoded default, not a computed "closest

alphabet" comparison: the source reaches for Russian specifically — its own comment calls it the most

common of the seven and the one carrying none of the others' unique letters — and only when Russian

itself is not enabled does it fall through to cyrillicEnabled.first(), whichever language that

happens to be in iteration order, with no comparison run at all (RomanizationLanguage.kt:71-73, 99).

**A script that looks alphabetic can still be an abugida, and a lookup table alone cannot transliterate

one.** Devanagari and Gurmukhi consonants carry an inherent vowel that a following mark can replace or

delete — a plain Map<Char, String> gets the base consonant right and the vowel wrong on every

syllable that isn't the default case. This needs a small state machine reading one character of

lookahead, not a bigger map; see combining-chars-break-char-literals for the specific complication

this creates once one of those marks is a combining character rather than its own letter.

**Only make a script's transliteration depend on a real dictionary when the rule genuinely cannot be

local to one character.** Two of twelve languages here need a platform library — Japanese, because a

kanji's reading depends on the word it's in, and Chinese, because some characters are heteronyms whose

pronunciation depends on the surrounding word. Every *other* script resolves per character or by pure

arithmetic and needs nothing beyond commonMain: a syllabic script's alphabet is a small map, and a

syllable-block script decomposes with integer arithmetic instead of a table at all

((codepoint - base) / consonantCount / vowelCount, or similar, covers the whole script from three

short tables). Reach for a dictionary only for cross-word disambiguation and morphology — properties a

per-character table cannot express by construction. Even where a dictionary IS used, a heteronym

looked up one character at a time returns several valid readings with no ranking between them —

taking the first is a guess; the honest fix is a word-aware lookup, not tuning which array index gets

picked.

**A dictionary-backed language can still be a two-stage pipeline where only the first stage needs the

dictionary.** Getting from kanji to a phonetic reading needs a real morphological analyzer; getting

from that reading to a Latin-script one is still pure table lookup plus a little lookahead — so it

belongs in shared, dictionary-free code, testable without the platform dependency the first stage

needs. Collapsing both into "the platform-specific implementation" denies every other platform a stage

it could have run itself.

**Inside that shared table-lookup stage, a multi-character token must be tried before its own prefix

is tried alone.** A syllable spelled with two characters that together make one sound is a single

token in the output table; matching one character at a time first consumes the first of the pair as

its own (wrong) sound and leaves the second stranded. Check known multi-character sequences starting

at the current position *before* falling back to the single-character table, every time the read

position advances — the shorter match is not a safe default merely because it comes first in a naive

loop.

Every reason a romanizer might have "nothing to add" should collapse to one signal, not several.

A line already in Latin, a language switched off in settings, a platform with no library for that

script, and a transliteration that happens to equal its own input are four situations with the same

correct behavior: show nothing extra. Returning null for all four — rather than throwing for the

unsupported-platform case — lets every caller be one if (result != null) instead of a when over

failure reasons, and lets a platform ship with a script's dictionary simply absent, not stubbed.

**"The dictionary is not downloaded yet" and "this platform has no dictionary at all" should be the

same return value, not two.** A library that ships its dictionary as an on-demand download has a

window where the feature is temporarily unavailable rather than permanently unavailable — resist the

urge to add a third state for it. Returning the same null for "not ready yet" as for "not supported

here" means every caller that already handles the permanent case handles the temporary one for free,

with no extra branch anywhere — see on-demand-dictionary-asset for the provider side of this same

null, where the download-not-finished check actually lives.

**Two consumers that both need to answer "is this character script X" must call the same function, not

duplicate the range check.** A script detector and a per-character transliterator over the same script

are tempting to write as two separate range checks — one per change that touches either. Duplicated

tests drift the moment one is edited without the other; a single shared predicate cannot.

A specific third-party formatting flag combination throws instead of formatting. A romanization

library asked for both diacritic tone marks and a particular "how to spell a Latin letter with no

Latin equivalent" option throws a checked exception rather than silently producing something wrong —

worth knowing before it surfaces from production traffic instead of from reading the docs. Confirm the

exact incompatible pairing for your own formatting library before wiring up every option combination.

Verifying it

Run from the root of a Kotlin Multiplatform module with a similar split (adapt file names):

# 1. the platform-specific surface really is just the two dictionary-dependent functions
grep -n "expect object\|fun " service/lyricsService/src/commonMain/kotlin/org/simpmusic/lyrics/romanization/PlatformRomanizer.kt

# 2. the platform with no library at all still compiles — both functions just return null
grep -n "actual fun" service/lyricsService/src/iosMain/kotlin/org/simpmusic/lyrics/romanization/PlatformRomanizer.ios.kt

# 3. script detection order: kana is checked (and wins) before Han
grep -n "fun detectScript" -A8 service/lyricsService/src/commonMain/kotlin/org/simpmusic/lyrics/romanization/RomanizationLanguage.kt

# 4. same-block disambiguation: single-candidate shortcut, then most-distinctive-first ordering
grep -n "size == 1" -A6 service/lyricsService/src/commonMain/kotlin/org/simpmusic/lyrics/romanization/RomanizationLanguage.kt

# 5. the two-character token is tried before the single-character table, every position
grep -n "val pair = \|val youon = \|val single = " service/lyricsService/src/commonMain/kotlin/org/simpmusic/lyrics/romanization/KanaRomanizer.kt

# 6. the incompatible formatting combination is a real, catchable exception type (adjust the jar glob to your formatting library)
JAR=$(find ~/.gradle/caches -iname "pinyin4j-*.jar" ! -iname "*sources*" ! -iname "*javadoc*" | head -1)
javap -p -classpath "$JAR" net.sourceforge.pinyin4j.format.exception.BadHanyuPinyinOutputFormatCombination

Pass condition for 1: exactly two function signatures inside the expect object. For 2: both

actual fun bodies are a bare null, no imports of a native library. For 3: the kana check is the

first line in the function, before the Han check. For 4: a size == 1 early return precedes a list

literal whose comment says "most-distinctive first." For 5: pair and youon are computed and

checked before single is even read. For 6: javap prints the exception class — confirming it is a

real, named type the calling code can be shown to guard against, not a hypothetical.