Caltech's @AnimaAnandkumar has an objection to scaling that is arithmetic rather than philosophical: for real physics, the transformer is already disqualified.
- Industrial simulation needs a context near 1e12, which puts attention at 1e24 operations, roughly twelve orders of
