Web search helps language models answer their questions. But what about the questions LLMs don’t know to ask?
Introducing PriorBench, a new kind of search benchmark. 🧵
Suppose you're concerned that you may have a serious disease. Your prior is a 20% chance of infection, but you don't know for sure. Doubting yourself, you approach an expert in the field. He consults the available data and informs you that exactly 20% of cases like yours are true
