Great read! Love the reachability vs. reliability framing: visual tools can reveal missing details, but using them well is another story. I feel like a lot of VQA is search + reasoning. If models learn what to look for next, how much of visual understanding is left unsolved?
Earlier this year, @a1zhang introduced the idea of compositional generalization, showing how harnesses can help LLMs generalize to harder tasks by decomposing them.
In our new blog post, we show that harnesses for Vision Language Models (VLMs) are compositional generalizers
GIF
