oh that’s a GREAT take
Replying to @lateinteraction
Interesting view. Brain does agentic attention and sys1 recurrence. Transformer-based agents do the opposite.
Omar Khattab
@lateinteraction
- Intuition: Compaction is agentic recurrence (RNNs), whereas recursion (RLMs) is agentic attention. Recurrence maintains a constant-size state at all times by compressing the past. Attention keeps all context fully represented and then decides how to re-process it for each step.
- part of this is in how ~everyone with a platform here speaks about LLMs for the past few months i rely on LLMs as excitedly as anyone, but... are we using the same models, guys?! the frontier is still dumb as a brick 20%+ of the time and needs more hand-holding than a freshmanThe vibe and language here have been drifting wholesale away from sensible, nuanced takes on the technical topics that animate me. Been oscillating on whether I should essentially just check out other than amplifying releases, as I've done for the bulk of this year to date.
- .@dianetc_ has a paper on this, called Reasoning-intensive Regression: arxiv.org/pdf/2508.21762 the problem *is* solvable but not the way you thinkLLM review weirdness... Just renaming an uploaded pdf from "paper.pdf" to "paper_final_draft_pdf_ready_for_review.pdf" boosts average scores (gpt-5.6-terra)
- The vibe and language here have been drifting wholesale away from sensible, nuanced takes on the technical topics that animate me. Been oscillating on whether I should essentially just check out other than amplifying releases, as I've done for the bulk of this year to date.
