Beyond Centralized Limits: The First Globally Distributed 100B+ Parameter AI Model.
Earlier this year, Intellect-1 (10B params) proved decentralized AI was possible.
Now, 0G has raised the bar 10x with DiLoCoX.
0G Research
10 posts
0G Research
@0G_Research
Research that builds infinite AI and the largest AI L1!
Joined July 2025
- We heard CT loves to hear from giga brains and read long-form content that dives deep into the tech being built. Giga brain deep dives on decentralised AI are incoming!
- One small step for 0G, one giant leap for mankind and AI! Read the paper:A major shift in the foundations of AI is underway, and 0G Labs is leading it. @sandy_carter 's latest article on @Forbes captures @0G_Research's latest breakthrough: 0G trains the largest distributed LLM yet - 100B+ parameters. Dive in → forbes.com/sites/digital-…
arxiv.orgDiLoCoX: A Low-Communication Large-Scale Training Framework for...The distributed training of foundation models, particularly large language models (LLMs), demands a high level of communication. Consequently, it is highly dependent on a centralized cluster with... - Replying to @0G_ResearchThis is a significant shift from “black-box APIs” to public goods for AI. Read our full blog on this groundbreaking discovery 👇
- Replying to @0G_ResearchThis changes the game: → Frontier-scale AI on commodity networks → Fully verifiable, transparent training → Models shaped by global contributors, not gated APIs → Composable foundation models for agents, apps & research
- 0gm, we will talk about a lot of research in 0Gtember!
- Replying to @0G_ResearchTraining large AI models has always been chained to centralized superclusters with ultra-fast interconnects. But the future can’t belong to a few who control the infrastructure. So we asked: What if frontier AI could be trained on everyday bandwidth?
- Replying to @0G_ResearchHow? Four key innovations: → Pipeline Parallelism: overlaps compute & comms → Dual Optimizer Policy: local updates + global alignment → One-Step-Delay Overlap: no wasted idle syncs → Adaptive Gradient Compression: smaller updates, same accuracy
- Replying to @0G_ResearchEnter DiLoCoX (Distributed Low-Communication Exchange). Co-developed with China Mobile, this framework enables training 107B-param models under 1gbps bandwidth constraints—achieving 357x greater communication efficiency than legacy AllReduce.
