Log inSign up
user avatar
Jacob Steinhardt
@JacobSteinhardt
Associate Professor of Statistics and EECS, UC Berkeley // Co-founder and CEO, @TransluceAI
Joined December 2011
82
Following
11.8K
Followers
  • Pinned
    user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Oct 24, 2024
    In July, I went on leave from UC Berkeley to found @TransluceAI, together with Sarah Schwettmann (@cogconfluence). Now, our work is finally public.
    user avatar
    Transluce
    @TransluceAI
    Oct 23, 2024
    Announcing Transluce, a nonprofit research lab building open source, scalable technology for understanding AI systems and steering them in the public interest. Read a letter from the co-founders Jacob Steinhardt and Sarah Schwettmann: transluce.org/introducing-tr…
    Image
    GIF
    54K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jul 8, 2020
    If our international students don't get a salary, I won't either. I pledge to donate my fall salary unless we fix U.S. immigration policy to allow international students (including incoming students) to be paid their stipend.
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Aug 18, 2022
    My student Kayo Yin needs your help. Her visa has been unnecessarily delayed, which would prevent her from coming to UC Berkeley to start her studies. Despite bringing all required documents, the @StateDept refused to process the visa and it could take months to re-process.
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jun 8, 2023
    Many people, including me, have been surprised by recent developments in machine learning. To be less surprised in the future, we should make and discuss specific projections about future models. In this spirit, I predict properties of models in 2030:
    Image
    What will GPT-2030 look like?
    From bounded-regret.ghost.io
    157K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jul 4, 2022
    In 2021, I created a forecasting prize to predict ML performance on benchmarks in June 2022 (and 2023, 2024, and 2025). June has ended, so we can see how the forecasters did:
    Image
    AI Forecasting: One Year In
    From bounded-regret.ghost.io
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    May 3, 2022
    This NYT article on Azalia and Anna's excellent chip design work is gross, to the point of journalistic malpractice. It platforms a bully while drawing an absurd parallel to @timnitGebru's firing. @CadeMetz should be ashamed. (not linking so it doesn't get more clicks)
    Image
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jun 21, 2022
    How fast can you run a transformer model? I spent an unreasonably large amount of time (and space) figuring out the answer:
    Image
    How fast can we perform a forward pass?
    From bounded-regret.ghost.io
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Feb 29, 2024
    Can we build an LLM system to forecast geo-political events at the level of human forecasters? Introducing our work Approaching Human-Level Forecasting with Language Models! Arxiv: arxiv.org/abs/2402.18563 Joint work with @dannyhalawi15, @FredZhang0, and @jcyhc_ai
    Image
    50K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Apr 4, 2023
    A core intuition I have about deep neural networks is that they are complex adaptive systems. This creates a number of control difficulties that are different from traditional engineering challenges:
    Image
    Complex Systems are Hard to Control
    From bounded-regret.ghost.io
    108K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Feb 20, 2023
    I'm back to blogging, with some new thoughts on emergence: bounded-regret.ghost.io/emergent-decep… I answer the question: what are some specific emergent "failure modes" for ML systems that we should be on the lookout for?
    Image
    Emergent Deception and Emergent Optimization
    From bounded-regret.ghost.io
    48K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jul 4, 2022
    Replying to @JacobSteinhardt
    To give an idea of just how much SOTA exceeded forecasters' expectations, here are the prediction intervals for the MATH and Massive Multitask benchmarks. Both outcomes exceeded the 95th percentile prediction.
    Image
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Jan 24, 2025
    In 2021, our research group released the MATH dataset. In the paper, we attribute the data to math contests released by the Mathematical Association of America (MAA), which is in the public domain. I've recently become aware that this is mistaken--while MATH contains MAA data...
    34K
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Dec 8, 2021
    Awesome to see @DeepMind 's recent language modeling paper include our forecasts as a comparison point! Hopefully more papers track progress relative to forecasts so that we can better understand the pace of progress in deep learning.
    Image
  • user avatar
    Jacob Steinhardt
    @JacobSteinhardt
    Feb 21, 2022
    On my blog, I've recently been discussing emergent behavior and in particular the idea that "More is Different". As part of this, I've compiled a list of examples across a variety of domains:
    Image
    Appendix: More Is Different In Other Domains
    From bounded-regret.ghost.io