Alec Radford

@AlecRad

San Francisco, CA

Pinned
Alec Radford
@AlecRad
Jun 11, 2018
What I've been working on for the past year! blog.openai.com/p/7fa97c36-611… Inspired by CoVE, ELMo, and ULMFiT we show that a single transformer language model can be finetuned to a wide variety of NLP tasks and performs very well with little tuning/tweaking.
50
Alec Radford
@AlecRad
Apr 25, 2019
This is a really fun live experiment with twitch chat predictably oscillating between love and hate based on the sample.
19
Alec Radford
@AlecRad
Feb 17, 2019
By the way - I think a valid (if extreme) take on GPT-2 is "lol you need 10,000x the data, 1 billion parameters, and a supercomputer to get current DL models to generalize to Penn Treebank."
15
Alec Radford
@AlecRad
Feb 11, 2019
The DL CV community is having a "oh wait, bags of local features are a really strong baseline for classification" moment with the BagNet paper. This has always been clear for text classification due to n-gram baselines. It took an embarrassingly long time for nets to beat them.
5
Alec Radford
@AlecRad
Nov 19, 2018
Nice discussion of the progress in NLU that's happening with BERT, OpenAI GPT, ULMFiT, ELMo, and more covered by @CadeMetz in the @nytimes I'm super excited to see how far this line of research will be able to get in the next few years!
Finally, a Machine That Can Finish Your Sentence (Published 2018)
From nytimes.com

Alec Radford’s posts