Autoregressive Vision-Language-Action (VLA) models offer a scalable path to robot learning, yet existing action tokenizers treat tokenization as a compression problem, producing representations that are semantically misaligned with the autoregressive backbone. We propose CATok, a causal action tokenizer that reframes tokenization as a causally structured generative process.

CATok introduces a conditional annealing mechanism that extracts action tokens by progressively annealing a flow-matching process: each token is conditioned on all preceding tokens and encodes the residual reconstruction signal at a specific noise level, establishing a coarse-to-fine causal token space whose generative semantics are structurally aligned with autoregressive modeling. A token-conditioned flow-matching decoder built on Multimodal Diffusion Transformer (MMDiT) reconstructs continuous action chunks from these discrete tokens with the precision of hybrid diffusion-head architectures. This discrete bottleneck enforces knowledge insulation by design, cleanly separating high-level semantic reasoning from low-level motor execution without requiring explicit attention masking.

Extensive evaluations across three simulation benchmarks demonstrate that CATok consistently surpasses existing tokenization methods in reconstruction fidelity–compression tradeoff and inference efficiency, while improving VLA task success rate, establishing a high-performance, scalable foundation for purely autoregressive VLA systems.

CATok consists of three components: (1) a Dual-Stream Action Encoder that maps raw action chunks into a compact set of latent query tokens via co-attention with an MMDiT-based transformer; (2) a Bottleneck Vector Quantizer that discretizes latent tokens into a finite codebook using factorized codes; and (3) a Flow-Matching Decoder with Conditional Annealing that reconstructs high-fidelity action trajectories while enforcing a coarse-to-fine causal structure.

The key design is the conditional annealing mechanism: at flow-matching timestep t, the first ⌊tK⌋ token embeddings are progressively masked, forcing each token to specialize in a distinct generative stage—from coarse global guidance at high-noise stages to fine-grained residual corrections as the trajectory matures. This induces a causally ordered hierarchy naturally aligned with autoregressive action generation.

We evaluate CATok on three simulation benchmarks—LIBERO, SimplerEnv (Simpler-Bridge), and RoboTwin 2.0—against three representative tokenizer baselines: BIN (uniform per-dimension discretization), FAST (DCT-based compression with BPE tokenization), and OAT (learned fixed-length tokenizer with prefix-based decoding).

Higher task success rate. CATok achieves the highest success rates across all benchmarks (95.9% avg. on LIBERO), with the largest gains on long-horizon tasks (LIBERO-Long: 91.0% vs. 90.1% for FAST). Causal token ordering enables autoregressive policies to model long-range temporal dependencies more effectively.

Faster inference. CATok delivers 1.7× faster VLA inference than FAST and 3.9× faster than BIN, enabling more responsive real-time deployment.

Better fidelity–compression balance. CATok achieves a 3.6× stronger VRR×CR score than FAST at the same token count, demonstrating that causal structure improves reconstruction quality without sacrificing compression.

Higher training efficiency. CATok consistently reaches higher success rates at the same training step count, attributable to its stage-wise token factorization that makes each token prediction more informative for trajectory generation.

Causal Predictive Ordering. Per-token Shannon entropy decreases monotonically in the forward token order (Figure 6(a)), while reversing the order flips the trend—confirming the learned ordering is directional and causal. FAST and BIN show no such structure.

Stage-wise Generative Semantics. t-SNE of VQ embeddings (Figure 6(b)) reveals clear slot-dependent geometry: early tokens form smooth elongated regions, late tokens form compact separated clusters. Prefix reconstructions (Figure 6(c)) show coherent coarse-to-fine trajectory updates, confirming each token contributes a distinct generative increment.

We clarify and test three linked points. “Coarse-to-fine” refers to learned flow-stage-wise refinement—not a prescribed frequency or fixed-basis decomposition. A donor-swap intervention with CATok’s native velocity network tests whether individual token positions have distinct, stage-specific effects. Single-token-zeroing visualizations reveal how those roles map onto action dimensions: early tokens mainly encode coarse gripper-related task state, whereas late tokens encode fine joint-level trajectory details. Together, these analyses test both the flow-stage alignment and the functional granularity of the learned token roles.

We do not claim that CATok learns a prescribed basis, such as a low-to-high-frequency decomposition. Here, “coarse-to-fine” describes the generation process and the granularity of learned control information, not the spectral content of individual tokens.

The mechanism is straightforward. Flow matching is a gradual refinement process: the ODE evolves an action trajectory from a high-noise state toward the low-noise data state, so different flow times correspond to different refinement stages. Conditional annealing aligns token positions with those stages: as flow time changes, the suffix-masking schedule changes which token positions are available to the velocity network, so each position is trained to contribute within a particular flow-time range. Repeated training induces functional specialization: tokens associated with earlier stages encode coarse, task-level information, whereas tokens associated with later stages encode the details needed to refine the trajectory.

This yields two testable expectations: (i) each token’s influence should concentrate near its assigned flow stage, and (ii) token content should progress from coarse task-execution state to fine trajectory detail. Sections 2 and 3 test these expectations, respectively, using the native inference modules.

We use the trained CATok tokenizer checkpoint (K=32, codebook 4096, horizon 20), its MMDiT velocity network v_θ, and its ODE decoder D—the same modules used at inference, not a separately trained prefix decoder. We encode 128 held-out Franka action chunks (7-DoF + gripper) as Q = (q₁, …, q_K). For each token k, we replace its embedding with the same-position embedding from a donor chunk—keeping the intervention within the slot’s empirical distribution—while holding x_t, t, and the noise realization fixed, so the resulting change isolates token k’s effect on the velocity prediction. For each flow time t we measure Δ_k(t) = MSE(v_θ(x_t, t, Q^swap(k)), x₁−x₀) − MSE(v_θ(x_t, t, Q), x₁−x₀), where MSE is ℓ. Because conditional annealing masks token k outside its active range, we normalize its effect within that range.

Swapping a token changes the velocity field primarily at that token’s assigned flow stage:

• Schedule-consistent support. Token k is visible only for t < (k+1)/K, consistent with the suffix-masking schedule κ(t) = tK.
• Stage-specific peaks. Within its active range, each token’s peak influence occurs near its assigned stage, t* ≈ k/K, with corr(k, t*) = 0.985 (0.995 after grouping). The localized, stage-aligned responses show that token positions are functionally differentiated and not interchangeable—confirming expectation (i).

We next examine whether this stage specialization also exhibits the expected coarse-to-fine progression in action semantics. We zero each token embedding in turn, decode with the native ODE decoder, and visualize the resulting change in every action dimension—a complementary, trajectory-level view of what each token position controls.

Together, the three points establish that (1) conditional annealing defines a learned coarse-to-fine refinement process, (2) donor swaps verify that token effects are distinct and aligned with their assigned flow stages, and (3) single-token zeroing reveals the corresponding functional progression from early gripper/task-state information to late joint/trajectory-detail information.

To verify that CATok transfers to real-robot deployment under camera noise, lighting variation, and hardware differences, we design a suite of real-robot evaluation tasks. The VLA built on CATok as its tokenizer consistently outperforms the QwenFAST baseline, with significant gains in spatial generalization, color generalization, and long-horizon task completion.

We collect 362 trajectories of "pick up the yellow cup and place it on the plate." and 220 trajectories of "stack the cups." Using Qwen3.5-VL as the backbone, we train two VLAs that differ only in their tokenizer—QwenFAST and QwenCATok. CATok is pretrained on the collected real-robot data together with InternA1 joint data, so that its action reconstruction accuracy matches that of FAST. We evaluate on three task suites:

Figure 9. Scene layouts for the three real-robot task suites. Left to right: Pick-Spatial, Pick-Color, Stack-Long.

• Pick-Spatial. The task is to pick up the yellow cup and place it on the plate. We fix 10 relative cup-and-plate positions; each model rolls out twice per position (20 rollouts total) and we report the success rate. Tests spatial generalization.
• Pick-Color. The task is to pick up the red cup and place it on the plate. The cup-plate relative position is fixed, and only the yellow cup seen during training is replaced with a red cup; each model rolls out 10 times. The training set contains no trajectories of picking up a red cup. Tests color generalization.
• Stack-Long. A long-horizon task: the robot must first pick up the red cup and stack it onto the yellow cup, then pick up the pink cup and stack it again. The relative positions of the three cups are fixed; each model rolls out 40 times. Tests long-horizon task completion.

We deploy QwenCATok (our causal action tokenizer integrated into a VLA policy) alongside the QwenFAST baseline on a real Franka manipulator across three contact-rich tasks—pick, pick-red, and stack. Each cell below is a single, unedited end-to-end rollout.

These real-world gains corroborate the mechanisms identified in simulation. CATok's causally structured token space—each token encoding a semantically distinct, coarse-to-fine generative stage—aligns the discrete bottleneck with the autoregressive backbone, and this semantic alignment generalizes to real-world perturbations: spatial repositioning (Pick-Spatial, 20 rollouts, 0.45→0.65, +44% relative) and out-of-distribution color (Pick-Color, 10 rollouts, 0.30→0.50, +67% relative). The largest gain appears on Stack-Long (40 rollouts, 0.275→0.475, +73% relative), consistent with the same stage-wise causal factorization that drives training efficiency in simulation—by making each token prediction more informative, causal token ordering lets the policy better capture the long-range temporal dependencies required for compositional, long-horizon tasks. Overall, CATok's structural properties—semantic token alignment and stage-wise causal factorization—translate from simulation into robust and consistent real-world gains.