Do harness evolution methods really find better harnesses or are they yet another way to improve performance by using more inference time compute? @yikewang_ and @TengX6 have an answer.
In this work, we revisit how automatic harness evolution should be evaluated.
Existing automatic harness evolution methods often search over harnesses using feedback from benchmark tasks and then report final performance on the same benchmark. This makes it difficult to tell
Can we train agents to make multiple attempts at solving tasks and learn from previous attempts? New work led by @TengX6 and @1t4chiii shows a simple MetaRL approach, where the agent reflects upon its prior episodes, can do this.
🚀 New work: Meta-Reinforcement Learning with Self-Reflection
LLM agents shouldn't just solve problems. They should learn from their own attempts. Most current RL methods optimize single independent trajectories.
Each attempt starts from scratch, with no mechanism to improve
Super excited to share our open interactive demo for DR Tulu-8B! It supports web and literature search with full transparency — you can see the model's thinking traces and tool outputs as it reasons through your query.
🔗 dr-tulu.org
📝 arxiv.org/abs/2511.19399
I have been trying out Telugu queries on the Indic LLM Arena over the last few days and most of the responses are surprisingly bad, with lots of hallucinations and sometimes even grammatical errors, even from strong (in English) models. Clearly there is a huge gap between English
For AI to be truly inclusive, it must understand more than just grammar—it must understand context.
@ai4bharat at @iitmadras had launched the Indic LLM Arena. This isn't just another leaderboard; it’s a public utility for:
✅ Developers: Test your models against real-world
Here's a really quick follow up to our recent big Olmo 3 release!
✨Olmo 3.1✨:
- a longer RL'ed 32B Think, with stronger math and reasoning skills
- a 32B Instruct, a larger Instruct model also with function calling
