Freaky ass model ðŸ˜
Replying to @ChowdhuryNeil
update: even with simpler/more natural prompts, Inkling might still hit on you
Ainesh Chatterjee
@ain3sh
MTS @FactoryAI. I code often and sleep occasionally. I like systems. Agents are graphs. Harness is everything. Views are my own. #2 fan of Corvus Corax
Joined November 2023
- Excited to share about what I’ve been working on over the past year: quantifying the capacity of a neuron😅 This led to a mathematical framework we call HOPE, which lets us rigorously deconstruct what deep networks might have learned. Paper: arxiv.org/abs/2607.21366
- High art btwAge-restricted adult content. This content might not be appropriate for people under 18 years old. To view this media, you’ll need to log in to X. Learn more
- 1/ Standard transformers have a fundamental topological flaw: they cannot track dynamic states over time without running out of layers. Once a state representation reaches the top layer of the feedforward stack, the model's ability to update its belief collapses. 🧵
- 1/ Backprop is the engine of deep learning, but neuroscientists have insisted for decades that the brain can't do it. There are no dedicated "error" neurons or backward wiring. What if the brain doesn't compute error in space, but in time? 🧵
