{"id":790,"date":"2016-03-03T02:06:35","date_gmt":"2016-03-03T06:06:35","guid":{"rendered":"http:\/\/www.cogconfluence.com\/?page_id=790"},"modified":"2024-01-04T04:18:21","modified_gmt":"2024-01-04T08:18:21","slug":"who","status":"publish","type":"page","link":"https:\/\/www.cogconfluence.com\/","title":{"rendered":""},"content":{"rendered":"\n
\"\"<\/figure><\/div>\n\n\n\n

I’m a Research Scientist in MIT CSAIL with the MIT-IBM Watson AI Lab. I did my PhD in Brain and Cognitive Sciences at MIT, as an NSF Fellow working with Josh Tenenbaum and Antonio Torralba. My work investigates representations underlying intelligence in artificial<\/span> (and previously, biological) neural networks.<\/h2>\n\n\n\n
<\/div>\n\n\n\n

I am also broadly interested in creativity underlying the human relationship to the world: from the brain\u2019s fundamentally constructive role in sensory perception to the explicit creation of experiential worlds in art. I see this as a frontier for understanding intelligence and building intelligent machines. As a grad student I designed and co-taught MIT’s first course on Vision in Art and Neuroscience, which I continue to teach every fall. You can learn more about it here<\/a>.<\/p>\n\n\n\n

<\/div>\n\n\n\n
\n\n\n\n
<\/div>\n\n\n\n

Selected Recent Publications<\/h4>\n\n\n\n

FIND:<\/strong> A Function Description Benchmark for<\/strong> Evaluating Interpretability Methods. <\/strong>An interactive dataset of functions resembling subcomputations inside trained neural networks, for validating and comparing open-ended labeling tools, and a new method that uses Automated Interpretability Agents to interpret other systems.<\/em> S. Schwettmann<\/strong>*, T. Rott Shaham*, J. Materzynska, N. Chowdhury, S. Li, J. Andreas, D. Bau, A. Torralba. NeurIPS <\/em>2023.<\/a> <\/p>\n\n\n\n

Multimodal Neurons in Pretrained Text-Only Transformers<\/strong>. We find multimodal neurons in a transformer pretrained only on language. When image representations are aligned to the language model, these neurons activate on specific image features and inject related text into the model’s next token prediction. S. Schwettmann<\/strong>*, N. Chowdhury*, A. Torralba. ICCV CVCL Workshop <\/em>2023 (Oral).<\/a><\/p>\n\n\n\n

MILAN: Natural Language Descriptions of Deep Features<\/strong>. We introduce a procedure that automatically labels neurons in deep networks with open-ended, compositional, natural language descriptions of their function. E. Hernandez, S. Schwettmann<\/strong>, D. Bau, T. Bagashvili, A. Torralba, J. Andreas. Natural Language Descriptions of Deep Features. ICLR<\/em> 2022 (Oral).<\/a> <\/p>\n\n\n\n

<\/div>\n\n\n\n

Recent News<\/h4>\n\n\n\n

AI agents help explain other AI systems, MIT News<\/a><\/p>\n\n\n\n

Demystifying machine-learning systems, MIT News<\/a><\/p>\n\n\n\n

3Q: The Interface Between Art and Neuroscience, an interview with  MIT News<\/a><\/p>\n\n\n\n

How Artificial Intelligence Sees Art History Metropolitan Museum of Art<\/a><\/p>\n\n\n\n

<\/div>\n\n\n\n