Eagle3 New Model

EAGLE3 New Model Configuration

Create tools/launcher/examples/<Org>/<Model>/hf_offline_eagle3.yaml by **copying the

closest existing example and adapting it**. Pick a reference with the same shape as the

target (dense vs MoE, similar size) from tools/launcher/examples/ — e.g. the Qwen3-8B

config for a dense model.

The pipeline is a 4-task config (task_0 data synthesis → task_1 hidden-state dump →

task_2 train → task_3 benchmark). The task structure, args, containers, and GPU/node

sizing are all visible in the existing examples — infer them from a reference rather than

hand-rolling. This file documents only the two things that are not obvious from the

examples: which dump backend to pick, and the model-specific gotchas.

Choosing the task_1 hidden-state dump backend

| Backend | Script | When to use |

|---------|--------|-------------|

| vLLM | common/eagle3/dump_offline_data_vllm.sh | Default. Broad coverage via vLLM's native hidden-state extractor. |

| HF | common/eagle3/dump_offline_data_hf.sh | VLMs / multimodal, custom-code models, sliding-window attention (TRT-LLM can't serve these). |

| TRT-LLM | common/eagle3/dump_offline_data.sh | Pure-text models with TRT-LLM support; pass --tp <TP> and --moe-ep <EP>. |

Rule of thumb: HF if the model is a VLM or uses sliding-window attention; vLLM

otherwise. TRT-LLM only when you specifically want its kernels for a supported plain-text model.

Model-specific adjustments

These are the non-obvious knobs that vary per model:

| Situation | What to change |

|---|---|

| Requires --trust-remote-code | Add to task_0 vLLM args (before the -- separator) and to task_3 benchmark args |

| MoE with large expert hidden dim | Increase intermediate_size in eagle_config.json to match moe_intermediate_size |

| Custom tokenizer (e.g. tiktoken) | Set TIKTOKEN_RS_CACHE_DIR env var in task_0 and task_1 |

After adapting the config, preview it with --dryrun before submitting.