Log inSign up
Image
user avatar
Stephen Roller
Thinking Machines
@stephenroller
MTS @thinkymachines. previously pre-training @googledeepmind, @character_ai, and @aiatmeta.
NYC
stephenroller.com
Joined February 2008
1,259
Following
6,019
Followers
RepliesMedia
  • user avatar
    Stephen Roller
    Thinking Machines
    @stephenroller
    Apr 24
    Didn't have hash layers on my bingo card
    user avatar
    DeepSeek
    @deepseek_ai
    Apr 24
    🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length. 🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models. 🔹 DeepSeek-V4-Flash: 284B total / 13B active params.
    Image
  • user avatar
    Stephen Roller
    Thinking Machines
    @stephenroller
    Aug 12, 2025
    There’s a line of critique/reviewer feedback in quantization literature that the OPT models are too easy to quantize bc they’re undertrained; but all scales are trained for the same 300B tokens, making the 6.7B and smaller overtrained by Chinchilla estimates.
  • user avatar
    Stephen Roller
    Thinking Machines
    @stephenroller
    Jul 15, 2025
    We are moving incredibly fast. Come light up GPUs with us.
    user avatar
    Mira Murati
    @miramurati
    Jul 15, 2025
    Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're