Do you work in AI?
Do you find things uniquely stressful right now, like never before?
Haver you ever suffered from a mental illness?
Read my personal experience of those challenges here:
Felix Hill
@FelixHill84
Research Scientist, Deepmind
I try to think hard about everything I tweet, esp on 90s football and 80s music
None of my opinions are really someone else's
- Lots of folks are talking about *emergence* in Deep Learning as if it's a new thing, that happens only in large language models at scale. It's not! It has been happening for decades and in very small networks. 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵
- You may smirk at the very notion of a royal family, but it takes a lot of bravery and vision to reform these things from the heart of the operation. I think history will look kindly on H.
- #GPT3 from @OpenAI showed an emergent ability in large neural language models for rapidly acquiring and using new words. We develop an agent that does this in a simulated 3D environment. arxiv.org/abs/2009.01719 1/N
- It is commonly said that models like BERT or GPT-2 don't really 'understand', but what does it actually mean to understand language? We try to answer this via a roadmap for human-like understanding of language in machines.
- Since GPT3, we have known that large Transformer language models are effective few-shot learners. But did you know that the nature of language itself makes language particularly rich pre-training data for inducing few-shot learning? 1/n
- Fantastic analogy from @geoffreyhinton deconstructing the strange but popular school of thought that because deep learning isn't perfect the solution must be to combine it with symbolic AI.
00:00 - The more I research, the more I'm convinced that Wittgenstein, Eleanor Rosch and David Rumelhart could have, collectively, designed the Transformer. 🧵
- I wrote a blog post about #DALL-E from OpenAI The way they approach *composition* is a strength of models like DALL-E not their weakness. In fact, composition it's the reason everyone is tweeting about it. fh295.github.io/noncomposition… @jackclarkSF @santoroAI
- The situation with reviewers of AI / ML papers demanding novelty has reached high farce Something needs to be done to stop this tyranny asap! If you have ever been found guilty of 'lack of novelty' (or are a serial offender, like me) read on. I have the solution: 🧵🧵👋🧵
