This is one-shotting Recursive Self-Improvement (RSI)
What this probably means is that Fable is trained to inspect & edit weights of a model to improve it
CTO of Legora has Fable one-shot direct write a 100k param LLM.
Note that unlike a trained/grown model, the designed on only uses 4k params, zeroing the rest.
This indicates there’s a readable/writable logical structure that can be reasoned about for this task.
And Fable
Got fable to hand-write weights of a 100k param model to recognize real versus fake legal citations.
Basically one-shotted a full NN that’s better than training it
When I first told @MParakhin that we were planning to move to a monorepo he said wisely:
Moving to a Monorepo is an idea that every engineer absolutely hates, and that once its there, no engineer would ever want to work at a company that doesn't use one. We will definitely do
Don't agree with this take
Harness engineering bridges the gap between model capabilities and product capabilities
That's also where a lot of alpha lies
Hot take: harness engineering is (usually) stupid.
Just use the one the model was trained for, and write really good behavior prompts to control how it works.
Lean into how the models want to work.
