Diane

@dianetc_

Figuring things out slowly.

P{Boston=.8,DC=.2}

dianetc.com

Joined March 2021

416 Following

1,046 Followers

Replies Reposts Media

Pinned
Diane
@dianetc_
Sep 28
I wrote a short blog on my experience testing Jev on reasoning-intensive regression problems. The results are within expectations (Jev is focused on classification after all), but the model has great attributes and is wicked fast! Read about it here: dianetc.com/musings/jev/
2
Diane
@dianetc_
Aug 19
Awesome blog! And they evaluated on the congress ToT split of OBLIQ-Bench
Weaviate AI Database
@weaviate_io
Aug 11
Scaling Test-Time Compute is the easiest way to get higher quality outputs from AI systems, but it comes at a cost! ⚖️ How much extra compute you want to trade for quality depends on your application. Introducing the effort parameter in Search Mode, giving you the power to
2
Diane
@dianetc_
Aug 18
Last fall, I spent a while trying to get a better understanding of the information retrieval (IR) landscape. This mostly involved a lot of research conversations, some half-baked ideas, and, for a while, not much tangible research progress. With OBLIQ-Bench being a few months
1
Diane
@dianetc_
May 6
We set out to build a better retriever, so we looked for the hardest IR benchmarks. For each, we asked how much headroom remained by running oracle reranking with a frontier LLM. Most had little room left! So we built OBLIQ-Bench to study much harder search queries than before.
Omar Khattab
9
Diane
@dianetc_
Jan 18
I tried treating similarity matrices as images and training a CNN to distinguish between gold docs and negatives. It was a fun little experiment and the results were unsurprising in hindsight (worked amazingly then failed spectacularly). details here: dianetc.github.io/musings/cnn_le…

Diane

Diane’s posts