Do you work in AI?
Do you find things uniquely stressful right now, like never before?
Haver you ever suffered from a mental illness?
Read my personal experience of those challenges here:
Felix Hill
6,297 posts
Felix Hill
@FelixHill84
Research Scientist, Deepmind
I try to think hard about everything I tweet, esp on 90s football and 80s music
None of my opinions are really someone else's
- Lots of folks are talking about *emergence* in Deep Learning as if it's a new thing, that happens only in large language models at scale. It's not! It has been happening for decades and in very small networks. 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵
- You may smirk at the very notion of a royal family, but it takes a lot of bravery and vision to reform these things from the heart of the operation. I think history will look kindly on H.
- #GPT3 from @OpenAI showed an emergent ability in large neural language models for rapidly acquiring and using new words. We develop an agent that does this in a simulated 3D environment. arxiv.org/abs/2009.01719 1/N
- It is commonly said that models like BERT or GPT-2 don't really 'understand', but what does it actually mean to understand language? We try to answer this via a roadmap for human-like understanding of language in machines.
- Since GPT3, we have known that large Transformer language models are effective few-shot learners. But did you know that the nature of language itself makes language particularly rich pre-training data for inducing few-shot learning? 1/n
- Fantastic analogy from @geoffreyhinton deconstructing the strange but popular school of thought that because deep learning isn't perfect the solution must be to combine it with symbolic AI.
00:00 - The more I research, the more I'm convinced that Wittgenstein, Eleanor Rosch and David Rumelhart could have, collectively, designed the Transformer. 🧵
- I wrote a blog post about #DALL-E from OpenAI The way they approach *composition* is a strength of models like DALL-E not their weakness. In fact, composition it's the reason everyone is tweeting about it. fh295.github.io/noncomposition… @jackclarkSF @santoroAI
- The situation with reviewers of AI / ML papers demanding novelty has reached high farce Something needs to be done to stop this tyranny asap! If you have ever been found guilty of 'lack of novelty' (or are a serial offender, like me) read on. I have the solution: 🧵🧵👋🧵
- Replying to @jeremycorbynYou're right, of course. But don't you feel at least a bit responsible for not being able to win an election against such a terrible government?
- my hot take is that you should not advertise if ur paper was accepted but instead what ideas it has my even hotter take is that if you think that the 'number' of papers that you/your lab have submitted vs. accepted is something to be proud of then you should quit science asap
- One of the first 'meta analyses' I have seen in ML. These are common in the natural sciences. When you do one in ML, the conclusions are not pretty. But that is precisely why we urgently need more of them. Cite this, give authors awards etc... arxiv.org/abs/2106.15195
- Does anyone think that the way e.g. ChatGPT behaves -- and the fact it's a very simple architecture trained with a simple prediction objective -- ought to revolutionise how cognitive (neuro)scientists and philosophers of mind think about the human brain?
