Train Character Lora

Train a Character LoRA (local, Flux.1-dev)

Overview

The trainer runs ostris ai-toolkit's run.py inside a headless GPU Docker container,

driven entirely through train_* MCP tools — you (the LLM) are the UI. You generate the

dataset, launch the job, watch progress, and the finished LoRA lands in ComfyUI

models/loras/ + the LoRA catalog automatically.

• Base model: FLUX.1-dev (best proven character consistency; needs ~24GB VRAM with

quantization — RTX 4090 class).

• Phase-1 scope: character LoRAs only. Style/slider/edit and other bases come later.

The flow (tool sequence)

• train_doctor — preflight once per session. Checks docker daemon, --gpus all

GPU passthrough, trainer image, HF_TOKEN. If image:false → run train_build_image

(one-time, several minutes — CUDA + torch + ai-toolkit). If hfTokenSet:false, warn the

user: the first run downloads FLUX.1-dev (gated HF repo) and needs HF_TOKEN in the MCP

server env.

• train_prepare_dataset — stage the images. See "Dataset" below.
• train_start — launch. Returns a job id immediately; training runs detached.
• train_status {id} — poll progress (progress.step/totalSteps/loss, recent

samples, log tail). Poll on a slow cadence (every few minutes) — a 2000-step run is

roughly an hour on a 4090. Don't block on it.

• Done — status:"completed" means the .safetensors was copied to

models/loras/<name>.safetensors and upserted into the LoRA catalog (result has the

paths + catalog id). Verify by loading it in a Flux workflow (LoraLoaderModelOnly,

strength 1.0) with the trigger word in the prompt.

Dataset guidance

Call train_prepare_dataset with items: [{path, caption?}, ...] and a defaultCaption.

• 10–30 varied images of the subject: different angles, expressions, lighting,

backgrounds, distances (close-up + half-body + full-body). Variety beats count.

• Trigger word: pick something rare and stable (e.g. ohwx, zxc_person) — NOT a

real word. Use it as defaultCaption and pass it as trigger to train_start.

• Captions: describe what *changes* between images (pose, setting, clothing,

expression); the constant identity is learned from the images themselves. Start each

caption with the trigger word, e.g. `ohwx person sitting in a cafe, laughing, natural

light`. Keep them short and factual. When in doubt, the trigger word alone

(defaultCaption) is a workable baseline.

• Images are copied and renamed img_00001.<ext> etc. — source files are never modified.

Params (sane defaults — override sparingly)

| Param | Default | When to change |

|-------|---------|----------------|

| steps | 2000 | 200 for a smoke test; 1500–3000 real runs. More ≠ better (overbake = plasticky). |

| lr | 1e-4 | 5e-5 for a tighter/subtler identity. |

| rank | 16 | 32 for very detailed characters. |

| resolution | [512,768,1024] | [512] if VRAM-constrained. |

| quantize | true | Keep true on 24GB. |

| saveEvery / sampleEvery | 250 | Lower (100) to watch early progress. |

Monitoring & judgement

• train_status.progress.samples are host paths — look at them. (ai-toolkit prints no

saved-sample lines, so they populate at finalize from the output dir; mid-run you can look

directly in the job's output/<name>/samples/ folder.) Identity should be

recognizable by ~1/3 of the run; if samples stay generic past halfway, the run will

likely underfit — cancel (train_cancel) and check captions/trigger.

• Loss should trend down and stabilize (~0.1–0.3); wild spikes usually mean lr too high.
• Checkpoints save every saveEvery steps under the job's output/ dir, so a cancelled

run isn't a total loss.

Failure modes

• no_docker / no_image from train_start → run train_doctor, follow its hints.
• OOM / CUDA errors in the log tail → drop resolution to [512], keep quantize:true,

batch stays 1.

• handoff failed in job error → training itself finished; the LoRA is still under the

job's output/<name>/ dir — copy it into models/loras/ manually and upsert the catalog.

• First run is slow before step 1 — FLUX.1-dev download (~24GB) + latent caching. As

long as the log tail moves, it's fine. The HF cache persists across runs.