Advanced Machine Intelligence (AMI) is building a new breed of AI systems that understand the world, have persistent memory, can reason and plan, and are controllable and safe.
We’ve raised a $1.03B (~€890M) round from global investors who believe in our vision of universally
Interesting analysis! On the log-sum vs. sum-log: the DLM training objective, from a VI perspective, essentially forces posterior collapse (over ordering). This shifts burden from inference to generation. It enables things like infilling but also increases modeling difficulty.
Diffusion LLMs (DLLM) can do “any-order” generation, in principle, more flexible than left-to-right (L2R) LLM.
Our main finding is uncomfortable:
➡️ In real language, this flexibility backfires: DLLMs become worse probabilistic models than the L2R / R2L AR LMs.
This
Our new work on RL training–inference mismatch shows that simply reverting to FP16 makes RL much more stable across algos, models, and engines. Feels like we’re reaching a stage where we can rethink RL for LLMs again with a clean off-policy PG formulation.
FP16 can have a smaller training-inference gap compared to BFloat16, thus fits better for RL. Even the difference between RL algorithms vanishes once FP16 is adopted. Surprising!
🚨There May Not be Aha Moment in R1-Zero-like Training: oatllm.notion.site/oat-zero
A common belief about the recent R1-Zero-like training is that self-reflections *emerge* as a result of RL training. We carefully investigated and showed the opposite.
🧵
