Building the World's First AI-Driven Digital Organism
(AIDO)
GenBio AI develops multiscale AI foundation models to decode and simulate human biology. Our work aims to accelerate breakthroughs in medicine, biotech, and life sciences by building toward an interactive, programmable virtual cell and ultimately AIDO.
"To truly understand life, we must model and simulate it across every scale. GenBio AI's world model is our path toward a future where biology becomes computable, predictable, and ultimately programmable."
Eric Xing
Co-Founder and Chief Scientist
"When we can simulate biology with precision, we will unlock a new era of medicine where curing disease becomes an engineering problem rather than a decades-long gamble."
David Baker
Co-Founder
our work
Research
#
Paper
![Language models applied to protein sequences have become a panacea, enabling therapeutics development, materials engineering, and core biology research. De- spite the successes of protein language models, genome language models remain nascent. Recent studies suggest the bottleneck is data volume or modeling context size, since long-range interactions are widely acknowledged but sparsely anno- tated. However, it may be the case that even short DNA sequences are modeled poorly by existing approaches, and current models are unable to represent the wide array of functions encoded by DNA. To study this, we develop AIDO.DNA, a pretrained module for DNA representation in an AI-driven Digital Organism [1]. AIDO.DNA is a seven billion parameter encoder-only transformer trained on 10.6 billion nucleotides from a dataset of 796 species. By scaling model size while maintaining a short context length of 4k nucleotides, AIDO.DNA shows sub- stantial improvements across a breadth of supervised, generative, and zero-shot tasks relevant to functional genomics, synthetic biology, and drug development. Notably, AIDO.DNA outperforms prior encoder-only architectures without new data, suggesting that new scaling laws are needed to achieve compute-optimal DNA language models. Models and code are available through ModelGenerator in https://github.com/genbio-ai/AIDO and on Hugging Face.](https://genbio.ai/wp-content/uploads/2024/12/1-1.jpg)
![Proteins play a fundamental role in life. Understanding the language of proteins offers significant potential for gaining mechanistic insights into biological systems and introduces new avenues for treating diseases, enhancing agriculture, and safeguarding the environment. While large protein language models (PLMs) like ESM2-15B and xTrimoPGLM-100B have achieved remarkable performance in diverse protein understanding and design tasks, these models, being dense transformer models, pose challenges due to their computational inefficiency during training and deployment. In this work, we introduce AIDO.Protein, a pretrained module for protein representation in an AI-driven Digital Organism [1]. AIDO.Protein is also the first mixture-of-experts (MoE) model in the protein domain, with model size scales to 16 billion parameters. Leveraging a sparse MoE architecture with 8 experts within each transformer block and selectively activating 2 experts for each input token, our model is significantly more efficient in training and inference. Through pre-training on 1.2 trillion amino acids collected from UniRef90 and ColabfoldDB, our model achieves state-of-the-art results across most tasks in the xTrimoPGLM benchmark. Furthermore, on over 280 ProteinGym Deep Mutational Scanning (DMS) assays, our model achieves nearly 99% of the overall performance of the best MSA-based model and significantly outperforms the previously reported state-of-the-art models that do not utilize MSA. We also adapted this model for structure-conditioned protein sequence generation tasks and achieved new SOTA in this domain. These results indicate that AIDO.Protein can serve as a strong foundation model for protein understanding and design. Models and codes are available through ModelGenerator in https://github.com/genbio-ai/AIDO and on Hugging Face.](https://genbio.ai/wp-content/uploads/2024/12/2-1.jpg)
Use Cases
Unlock the future of biology with models simulating life from molecules to individuals
Drug Discovery
Accelerate drug development by simulating millions of treatments and predicting their effects on complex biological systems.
Read More
Disease Understanding
Equip researchers with models to delve into mechanisms of conditions like cancer and neurodegenerative diseases.
Read More
Personalized Medicine
Enable digital patient simulations to minimize adverse reactions and improve treatment effectiveness.
Read More
what we create
