feat(speculative): add serve_vllm for EAGLE-3 / P-EAGLE drafts

[khazic]

What

Adds nemo_automodel/components/speculative/serve_vllm.py, the vLLM companion to
serve_sglang. It serves an Automodel-trained EAGLE-3 / P-EAGLE drafter under
vLLM's parallel-drafting runtime.

Why

The P-EAGLE recipe (parallel_drafting: true) produces a head that SGLang cannot
serve today, so serve_sglang rejects it and points at vLLM. There was no vLLM
entry point, so a trained P-EAGLE draft could not actually be served or
benchmarked (the qwen_peagle_perfectblend.yaml recipe comment already notes
that acceptance can only be measured by running the head under vLLM). This closes
that gap.

How

serve_vllm resolves a recipe checkpoint (epoch_*/model[/consolidated]),
applies three fixups so vLLM can load the draft, then shells out to
vllm.entrypoints.openai.api_server with the right --speculative-config:

1. architectures: rewrite Automodel's LlamaEagle3DraftModel to vLLM's
   registered LlamaForCausalLMEagle3.
2. pard_token (P-EAGLE): vLLM's parallel-drafting proposer reads the masked-slot
   id from pard_token / ptd_token_id / dflash_config.mask_token_id, but the
   recipe writes only the top-level mask_token_id. The value is copied over,
   mirroring vLLM's own speculators update_peagle mapping.
3. weight keys: Automodel wraps the draft as self.model, so weights are saved as
   model.*; vLLM re-adds model. to every non-top-level weight, double-prefixing
   them. The prefix is stripped into a one-time vllm_export/ copy, leaving the
   source checkpoint untouched. A draft whose weights are already vLLM-standard
   skips the export and gets the config fixups in place.

The draft already stores the d2t / t2d vocab-remap buffers in the offset form
vLLM expects, so (unlike the SGLang path) no separate speculative-token-map file
is emitted.

Validation

Trained a standard P-EAGLE Qwen3-8B draft and served it through serve_vllm on
vLLM's parallel-drafting runtime: the draft loads and parallel drafting runs,
producing a measurable acceptance length end to end. New unit tests cover artifact
resolution, all three fixups, the export vs in-place split, export caching, and
CLI dispatch (--print-only, missing-vLLM exit, subprocess / execv dispatch).

Notes

• vLLM is intentionally not bundled with the container and not declared in
  pyproject.toml; the script exits with an install hint when it is missing.
• P-EAGLE inference requires vLLM >= 0.16 (parallel-drafting runtime).

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[HuiyingLi]

/ok to test 54bf059

[HuiyingLi]

/claude review

[claude]

Bug: the staleness check only compares the exported config's mtime against source weight files, but ignores the source config.json mtime. If a user re-trains and the source config.json is updated (e.g. mask_token_id changes, or parallel_drafting is toggled) without the weight files being touched, _export_is_fresh returns True and the export is reused with a stale pard_token / architectures.

Suggested change

	def _export_is_fresh(draft_dir: Path, export_dir: Path) -> bool:
	"""True when ``export_dir`` holds a complete config + weights newer than the source."""
	exported_config = export_dir / "config.json"
	if not exported_config.exists() or not _has_hf_weight_file(export_dir):
	return False
	src_weights = [*draft_dir.glob("model-*.safetensors"), draft_dir / "model.safetensors"]
	src_mtime = max((w.stat().st_mtime_ns for w in src_weights if w.exists()), default=0)
	return exported_config.stat().st_mtime_ns >= src_mtime
	exported_config = export_dir / "config.json"
	if not exported_config.exists() or not _has_hf_weight_file(export_dir):
	return False
	src_weights = [*draft_dir.glob("model-*.safetensors"), draft_dir / "model.safetensors"]
	src_config = draft_dir / "config.json"
	src_files = [w for w in src_weights if w.exists()] + ([src_config] if src_config.exists() else [])
	src_mtime = max((f.stat().st_mtime_ns for f in src_files), default=0)
	return exported_config.stat().st_mtime_ns >= src_mtime

[claude]

Missing coverage: the docstring says "touching the source weights rebuilds it" but the test only verifies the cache-hit path. Consider adding a second half that touches the source weight file and asserts the export is rebuilt:

    # Touch source weights -> export must be rebuilt.
    (model_dir / "model.safetensors").write_bytes((model_dir / "model.safetensors").read_bytes())
    resolve_draft_artifacts(str(model_dir))
    assert export_config.stat().st_mtime_ns > first_mtime, "stale export must be rebuilt after source changes"

[HuiyingLi]

Thank you!