Distillation is still possible through “Behavior Parroting”, see post below 👇
Cc: @natolambert
Sasha Sheng (Hiring) 🫶🏼
@hackgoofer
Mission, San Francisco
Joined August 2011
- 👎 loops Every seasoned engineers know this is a bad idea - if your goal is to build large complex systems. 😳😱loops should be renamed to GOTO statements - everything is global state (no scoping / isolation) - anything can happen at any time (impossible to reason about where you are in an execution + debug) - constraints are loose guidelines (rather than programmatic guarantees)
- The Original Scaling Law paper (Kaplan et al.) had a bug that caused the scaling law to be incorrect. Hopefully the OSS model builder community already knows this, but just in case if not. The co-creator of RLHF @CompleteSkeptic broke it down for you in this post:
- Every single day I live this life I feel truly grateful that I get to work with some of my favorite people here at TypeSafe AI. This is a group with real intellectual horsepower but also the kindest, and most courageous humans I know. 🥰 I feel inspired every single day.
- TypeSafe AI has the sickest swags! Come thru to our 5pm hack night tonight and win some. We are headed into the AI Engineer World Fair presenting a talk on “What’s next after RLHF”. @CompleteSkeptic loves a good debate. So please give him hard questions. 😏
