📣📣 Challenging new visual benchmark from our lab!
Is computer vision “solved”?
Not yet
Current models score 0% on ZeroBench
🧵1/6
CAML Lab
@CAML_Lab
- 🪡📢 New paper from our group! We explore the ability of frontier LLMs to follow threads of information through long context windows "Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?" Project page: needle-threading.github.io🎺New paper! "Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?" needle-threading.github.io 🧵(1/5)
- 📢📢 Check out this new paper from our group! A Practitioner's Guide to Continual Multimodal Pretraining: arxiv.org/abs/2408.14471🚀New Paper: "A Practitioner's Guide to Continual Multimodal Pretraining"! arxiv.org/abs/2408.14471 🌐Foundation models like CLIP need constant updates to stay relevant. How to do this in the real-world? Answer: Continual Pretraining!! We studied how to effectively do this.🧵👇
- 🚨New paper from our group introducing GRAB! GRAB is a challenging GRaph Analysis Benchmark for LMMs Project page: grab-benchmark.github.io Paper: arxiv.org/abs/2408.11817🎉📢New Paper! Introducing GRAB: A Challenging GRaph Analysis Benchmark for Large Multimodal Models grab-benchmark.github.io The highest-performing model scores just 21.7% A thread 🧵
- 🚨New work from our group introducing SciFIBench! We evaluate the scientific figure interpretation capabilities of 30 LMM, VLM and human baselines, including GPT-4o! Paper: arxiv.org/abs/2405.08807 Data: huggingface.co/datasets/jonat… Repo: github.com/jonathan-rober…Age-restricted adult content. This content might not be appropriate for people under 18 years old. To view this media, you’ll need to log in to X. Learn more
