Introducing Modular Domain Experts (MoDE), a new multi-domain adaptation technique for LLMs. MoDE independently trains experts on different domains and composes them to boost LLM performance on complex, multi-domain tasks.
Paper: arxiv.org/abs/2410.10181
