watching your kid learn language is amazing. my daughter was trying to describe water as deep and called it "heavy". she was trying to say her water bottle was out of reach on a table and she said "it's too high".
every possible direction in ML has multiple OOMs of algorithmic gains coming in the next year or two:
- transformers aren't the final architecture
- gradient descent/backprop isn't the final learning algorithm
- sample efficiency will go past humans
- none of pre/mid/post
geneology is not for the weak: my great-great-grandfather is 'louis' on his headstone, 'morris' in the 1910 census, 'lewis' on his death certificate. his 'real' name is 'lipman' (!) which is on no documents afaict, but shows up on his kids' gravestones.
his wife yena was born