Zeyu Ling, Qing Shuai, Teng Zhang, Shiyang Li, Bo Han, Changqing Zou

Text-to-motion generation has advanced rapidly, yet two challenges persist. First, existing motion autoencoders compress each frame into a single monolithic latent vector, entangling trajectory and per-joint rotations in an unstructured representation that downstream generators struggle to model faithfully. Second, text-to-motion, pose-conditioned generation, and long-horizon sequential synthesis typically require separate models or task-specific mechanisms, with autoregressive approaches suffering from severe error accumulation over extended rollouts.

We present PRISM, addressing each challenge with a dedicated contribution. (1) A joint-factorized motion latent space: each body joint occupies its own token, forming a structured 2D grid (time x joints) compressed by a causal VAE with forward-kinematics supervision. This simple change to the latent space, without modifying the generator, substantially improves generation quality, revealing that latent space design has been an underestimated bottleneck. (2) Noise-free condition injection: each latent token carries its own timestep embedding, allowing conditioning frames to be injected as clean tokens (timestep 0) while the remaining tokens are denoised. This unifies text-to-motion and pose-conditioned generation in a single model, and directly enables autoregressive segment chaining for streaming synthesis. Self-forcing training further suppresses drift in long rollouts. With these two components, we train a single motion generation foundation model that seamlessly handles text-to-motion, pose-conditioned generation, autoregressive sequential generation, and narrative motion composition, achieving state-of-the-art on HumanML3D, MotionHub, BABEL, and a 50-scenario user study.

Pretrained weights are hosted on Hugging Face and should be downloaded to pretrained_models/prism_1.4b:

pip install huggingface_hub
huggingface-cli download ZeyuLing/PRISM-TP2M-1.4B --local-dir pretrained_models/prism_1.4b

Or from Python:

from huggingface_hub import snapshot_download
snapshot_download("ZeyuLing/PRISM-TP2M-1.4B", local_dir="pretrained_models/prism_1.4b")

• Python >= 3.9
• PyTorch (CUDA recommended)
• transformers, diffusers, einops, mmengine
• SMPL/SMPL-X body model (for full mesh rendering)

This project is designed to run inside the versatilemotion repository.

cd /path/to/versatilemotion
pip install -r requirements.txt

All scripts assume you are in the versatilemotion repository root and checkpoint path is: opensource/prism/pretrained_models/prism_1.4b.

Use this script when you want one prompt -> one motion clip.

python opensource/prism/scripts/run_t2m.py \
  --checkpoint opensource/prism/pretrained_models/prism_1.4b \
  --prompt "A person walks forward and waves." \
  --output_dir outputs/t2m

Key arguments:

• --prompt: required text prompt.
• --num_frames: generated frame count (default 129).
• --guidance_scale: classifier-free guidance scale (default 5.0).
• --device: optional device (e.g., cuda:0).

Use this script when the first frame is given and you want controlled continuation.

python opensource/prism/scripts/run_tp2m.py \
  --checkpoint opensource/prism/pretrained_models/prism_1.4b \
  --first_frame_pose tpose \
  --prompt "The person begins to walk forward slowly and naturally." \
  --output_dir outputs/tp2m

Input modes:

• --first_frame_npz /path/to/cond.npz: provide your own first-frame condition.
• --first_frame_pose tpose: use built-in preset pose.

Preset poses: standing, tpose, squat, kneel, sit.
Exactly one of --first_frame_npz and --first_frame_pose must be provided.

Use this script for long-form generation with explicit segment prompts.

python opensource/prism/scripts/run_sequential.py \
  --checkpoint opensource/prism/pretrained_models/prism_1.4b \
  --prompts "A person waves." "A person walks." "A person bows." \
  --lengths 65 129 65 \
  --output_dir outputs/sequential

Key arguments:

• --prompts: required list of segment prompts.
• --lengths: optional per-segment frame counts; if omitted, all segments use 129.
• --guidance_scale: classifier-free guidance scale.

Use this script when you have a paragraph-like narrative instead of explicit segment list.

python opensource/prism/scripts/run_narrative.py \
  --checkpoint opensource/prism/pretrained_models/prism_1.4b \
  --text "A person walks in, waves, sits down, then stands and bows." \
  --no_rewriter \
  --output_dir outputs/narrative

Notes:

• Current script supports --no_rewriter mode robustly: it splits --text by semicolon (;) into segments.
• If you do not use semicolons, the full text is treated as one segment.
• --rewriter_url path is reserved, but external rewriter client logic is not implemented in this open-source script.

@article{ling2026prism,
  title={PRISM: Streaming Human Motion Generation with Per-Joint Latent Decomposition},
  author={Ling, Zeyu and Shuai, Qing and Zhang, Teng and Li, Shiyang and Han, Bo and Zou, Changqing},
  journal={arXiv preprint arXiv:2603.08590},
  year={2026},
  url={https://arxiv.org/abs/2603.08590}
}

See the versatilemotion repository for license terms.