Pinned
deep learning research was the original vibe math
Zachary Lipton
12.7K posts
Zachary Lipton
@zacharylipton
Professor: CMU/@acmi_lab, Cofounder: @AbridgeHQ, Creator: @d2l_ai & approximatelycorrect.com, Relapsing 🎷
- Many ML practitioners have no idea where to start getting a foothold in learning theory. I certainly didn't. "Understanding Machine Learning" by Shai Shalev-Shwartz & Shai Ben-David is an absolute treasure. Beautiful hardcover & free download @ cs.huji.ac.il/~shais/Underst…
- I don’t know why so many people need to hear this, but if you actually want to do a PhD, don’t only apply to “top 10” schools. The cutthroat-ness of admissions falls off rapidly but the quality of faculty does not.
- By default, logistic regression in scikit-learn runs w L2 regularization on and defaulting to magic number C=1.0. How many millions of ML/stats/data-mining papers have been written by authors who didn't report (& honestly didn't think they were) using regularization?
- pro·fes·sor — noun ***the member of a research lab with the deepest intuitions about how LaTeX \vspace, page breaks, and figure placement work.***
- Well here goes. Our ICML Debates paper is live: "Troubling Trends in Machine Learning Scholarship". If anyone needs me, I'll be in witness protection. 🙄
- So much fucking respect to @ylecun for being the one big-tech blue chip name to step up & champion open research, open source & the startup community. Unflinchingly. For remembering what made this moment possible in the first place & putting his name on the line to protect it ❤️
- The field is already self-correcting. Good departments/labs are clearing their eyes, caring less about paper count, seeing through the noise. Don't worry so much about the ICML deadline. Slow down, relax, try to do work you're proud of, submit when it's ready.
- Our ***free deep learning book*** (d2l.ai) now has a ***modern NLP chapter draft***, complete w. the omnipresent BERT & friends. All drafted via interactive Jupyter notebooks.We have re-organized Chapter: NLP pretraining (d2l.ai/chapter_natura…) and Chapter: NLP applications (d2l.ai/chapter_natura…), and added sections of BERT (model, data, pretraining, fine-tuning, application) and natural language inference (data, model).
- When an entire sub-field of ML has failed (thus far) to produce any method that could actually be used, there is no "state of the art", only the "state of the game".
- Tomorrow, we launch "The Art of the Paper", a course on the principles, mechanics, & culture of scientific writing. While core to our work, this material seldom gets a formal treatment. I'm excited, nervous, & grateful to @mldcmu for the creative freedom. github.com/acmi-lab/cmu-1…
- Thrilled to release our greatest paper yet in Nature Machine Learning: ***Multimodal Multitask Multidomain Multi-Attention for Multilabel Classification with Multiple Adversaries*** arxiv.org/pdf/1234321234…
- If you follow known fraudster & plagiarist @sirajraval, ***please take a moment to unfollow him***. Especially prominent researchers, journalists, and institutions. Don’t confer credibility on this scoundrel and thereby help to defraud his students.
- Instead of receiving a boring paper diploma at the end of their PhDs, Google PhD graduates receive a priceless note handwritten by Jeff Dean on a napkin that says, “You have a PhD now.”
