Log inSign up
Thomas Fel
1,626 posts
Image
user avatar
Thomas Fel
@thomas_fel_
Interpretability, Visual Intelligence @GoodfireAI. Prev: @Harvard, @Google, @BrownUniversity (@tserre lab). Crêpe lover.
San Francisco, CA
thomasfel.me
Joined February 2017
944
Following
3,137
Followers
  • Pinned
    user avatar
    Thomas Fel
    @thomas_fel_
    May 7
    Happy to share my first post since joining Goodfire. Neural geometry has been my obsession for years, and our team here is building a really serious research agenda around it. I can't wait to share the series of papers coming over the next few weeks... Brace for shapes 🍩
    user avatar
    Goodfire
    @GoodfireAI
    May 7
    Neural networks might speak English, but they think in shapes. Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision. Starting today, we’re releasing a series of posts on this research agenda. 🧵
    Image
    00:00
    78K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Oct 14, 2025
    🕳️🐇Into the Rabbit Hull – Part I (Part II tomorrow) An interpretability deep dive into DINOv2, one of vision’s most important foundation models. And today is Part I, buckle up, we're exploring some of its most charming features.
    Image
    00:00
    64K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Sep 28, 2025
    Phenomenology → principle → method. From observed phenomena in representations (conditional orthogonality) we derive a natural instantiation. And it turns out to be an old friend: Matching Pursuit! 📄 arxiv.org/abs/2506.03093 See you in San Diego, @NeurIPSConf 🎉
    Image
    28K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Oct 29, 2024
    🎭Recent work shows that models’ inductive biases for 'simpler' features may lead to shortcut learning. What do 'simple' vs 'complex' features look like? What roles do they play in generalization? Our new paper explores these questions. arxiv.org/pdf/2407.06076 #Neurips2024
    Qualitative Analysis of “Meta-feature” (cluster of features) Complexity. (Left) A 2D UMAP projection displaying the 10,000 extracted features. The features are organized into 150 clusters using K-means clustering applied to the feature dictionary D. 30 clusters were selected for analysis of features at different complexity levels. (Right) For each Meta-feature cluster, we compute the average complexity score. This allows us to classify the features based on their complexity according to the model. Notably, simple features are often akin to color detectors (e.g., grass, sky) and detectors for low-frequency patterns (e.g., bokeh detector) or lines. In contrast, complex features encompass parts or structured objects, as well as features resembling shape.
    44K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Jul 20, 2025
    Great excuse to share something I really love: 1-Lipschitz nets. They give clean theory, certs for robustness, the right loss for W-GANs, even nicer grads for explainability!! Yet are still niche. Here’s a speed-run through some of my favorite papers on the field. 🧵👇
    Image
    Image
    user avatar
    cider
    @jeffreycider
    Jul 20, 2025
    optimization theorem: "assume a lipschitz constant L..." the lipschitz constant:
    60K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Oct 15, 2025
    🕳️🐇Into the Rabbit Hull – Part II Continuing our interpretation of DINOv2, the second part of our study concerns the geometry of concepts and the synthesis of our findings toward a new representational phenomenology: the Minkowski Representation Hypothesis
    Image
    57K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Mar 13, 2025
    Train your vision SAE on Monday, then again on Tuesday, and you'll find only about 30% of the learned concepts match. ⚓ We propose Archetypal SAE which anchors concepts in the real data’s convex hull, delivering stable and consistent dictionaries. arxiv.org/pdf/2502.12892…
    Image
    43K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Sep 4, 2023
    👋 Explain big vision model with 𝐂𝐑𝐀𝐅𝐓 🪄🐰 A method that 𝙖𝙪𝙩𝙤𝙢𝙖𝙩𝙞𝙘𝙖𝙡𝙡𝙮 extracts the most important concepts for your favorite pre-trained vision model. e.g., we automatically discover the most important concepts on a ResNet50 for rabbits: eyes, ears, fur. 🧶
    Image
    56K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Feb 22, 2024
    👋👨‍🍳🍵 After a year of cooking up a secret project, I'm thrilled to officially reveal: The 𝐋𝐄𝐍𝐒 𝐏𝐫𝐨𝐣𝐞𝐜𝐭. By combining modern tools of Explainable AI, how much can we explain a ResNet50? 🧶
    Image
    00:00
    42K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Sep 17, 2025
    Check out our COLM 2025 (oral) 🎤 SAEs reveal that VLM embedding spaces aren’t just "image vs. text" cones. They contain stable conceptual directions, some forming surprising bridges across modalities. 1/2
    Image
    10K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Mar 25, 2025
    An underrated paper. “Just because the information is here doesn’t mean it’s easily extractable.” Think about AES-256 a file —data is there, but the route back is complex. This is what we used to understand which features are easy vs. complex for NN arxiv.org/abs/2407.06076
    Image
    Image
    user avatar
    Jack Morris
    @jxmnop
    Mar 24, 2025
    # A new type of information theory this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else it says that we should measure the amount of information available in some representation based on how *extractable* it is,
    7.4K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Feb 5, 2023
    𝐇𝐨𝐥𝐢𝐬𝐭𝐢𝐜𝐚𝐥𝐥𝐲 𝐄𝐱𝐩𝐥𝐚𝐢𝐧𝐚𝐛𝐥𝐞 𝐕𝐢𝐬𝐢𝐨𝐧 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 arxiv.org/abs/2301.08669 Another important work for explainability. The authors reformulate the B-cos network for transformers by showing that each usual Vision Transformer blocks...
    Image
    13K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Apr 3, 2024
    Personal update: I'm truly honored and thrilled to announce that I'll be joining Harvard this September as one of the @KempnerInst Research Fellows! 🥳
    user avatar
    Kempner Institute at Harvard University
    @KempnerInst
    Apr 2, 2024
    Thrilled to announce the 2024 recipients of #KempnerInstitute Research Fellowships: Thomas Fel, Mikail Khona, Bingbin Liu, Isabel Papadimitriou, Noor Sajid, & Aaron Walsman! bit.ly/4aBQ6MS @Napoolar @KhonaMikail @BingbinL @isabelpapad @nsajidt @aaronwalsman
    Image
    12K
  • user avatar
    Thomas Fel
    @thomas_fel_
    Dec 10, 2023
    Super excited to be at @NeurIPSConf this year, presenting 4 articles focused on explainability! 😉 If you're attending the conference and would like to chat or take a coffee feel free to reach out! 👋☕
    Image
    Image
    Image
    12K

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up
Advertisement
Advertisement