LLMs vs Humans

In the current era of LLM abundance, it is now possible to mix and match models based on desired price, performance, or inference speed using a technology called the LLM router. This ongoing project focuses on identifying semantic clusters or topics where LLMs of different sizes demonstrate distinct strengths and weaknesses. Using the Chatbot Arena dataset—which contains LLM response comparisons with winner/tie labels based on human feedback—we perform topic modeling to uncover these clusters. This work aims to enable more efficient and intelligent LLM routing.