Jonathan Richard Schwarz

I'm a Visiting Professor at Imperial College London and the Head of AI Research at Thomson Reuters (TR), leading TR's Foundational Research Team. In addition, I serve as an Expert Advisor to the UK's AI Security Institute . I joined TR through the acquisition of Safe Sign Technologies, for which I served as Co-Founder and Chief Scientific Officer (CSO).

Previously, I was also a Research Fellow at Harvard University and a Senior Research Scientist at Google DeepMind. I obtained my PhD from the joint DeepMind-University College London programme, advised by Yee Whye Teh and Peter Latham. My Thesis focused on sparse parameterisations and knowledge transfer for efficient Machine Learning. Before that, I spent two years at the Gatsby Computational Neuroscience Unit and graduated top-of-the class from The University of Edinburgh.

My research focuses on the objective of building (i) efficient, (ii) general and (iii) robust Machine Learning systems. A central paradigm in my approach is the design of algorithms that can effectively abstract knowledge and skills present in related problems, enabling their utilisation for efficient learning on future tasks. In this way, agents gradually build diverse repertoires of skills, allowing transfer to future tasks using only a fraction of the otherwise required learning time and/or data.

To that end, most of my existing work falls within one or more of the following categories:

Sparsity & Efficient Parameterizations: [ICLR'19] [ICLR'20] [NeurIPS'21] [TMLR'22] [JMLR'22] [ICML'23] [ICML'24] [ACL'25]
Large Language Models (LLMs): [TACL'18] [NeurIPS'24] [NeurIPS'24] [ICLR'25] [ACL'25] [EMNLP'25] [arXiv'25]
Data-centric ML: [NeurIPS'23] [ECCV'23] [NeurIPS'24] [TMLR'25] [arXiv'25]
Continual Learning: [ICML'18] [NeurIPS'19] [ICLR'20] [NeurIPS'21] [NeurIPS'24]
INRs / Neural Data Compression: [TMLR'22] [ICML'23] [ICLR WS'23] [NeurIPS'23] [CVPR'24]
Meta-Learning: [ICML WS'18] [NeurIPS WS'18] [ICLR'19] [arXiv'19] [TMLR'22] [ICML'23] [NeurIPS'23a] [NeurIPS'23b] [ICML'24]

Email / Google Scholar / Twitter / LinkedIn / Full CV: on request

News

Invited talks at The Allen Institute for AI and The University of Oxford (November 2025).
Scales++ (Efficient Evals through Data Pruning) released (October 2025).
Serving as Area Chair for ICLR'25 (October 2025).
ADMIRE-BayesOpt accepted to TMLR (November 2025).
New paper on efficient LLM evals with Data Pruning based on Cognitative Scales Embeddings: Scales++ (October 2025).
Invited panel discussion at The 2025 Global Summit on Open Problems for AI (October 2025).
Invited panel discussion at The Royal Society (September 2025).
New EMNLP paper on Automatic LLM evals calibrated with human judgements (May 2025).
Serving as Area Chair for EMNLP'25 (September 2025).
Our new paper on BayesOpt for data-mixture discovery during pre- and post-training is now available (August 2025).
New ACL'25 Oral paper on Automatic Expert Discovery in LLM Upcycling via Sparse Interpolated Mixture-of-Experts (May 2025).
Guest lecture at The University of Virginia on Data-Centric ML for LLM Training (February 2025).
Our paper on Composable Interventions for LLMs was accepted to ICLR 2025 (January 2025)!
Organising the NeurIPS'24 workshop on Compositional Learning: Perspectives, Methods, and Paths Forward (December 2024).
Serving as publicity chair at CoLLAs 2025 (December 2024)!
Our paper on AI agents for scientific discovery accepted at Cell (October 2024).
Invited talk at the AE: Global Summit on Open Problems for AI on Data-Centric ML (October 2024).
Invited talk at MICCAI 2024 on INRs (October 2024).
CoLoR-Filter accepted to NeurIPS'24 (September 2024).
MAC accepted to NeurIPS'24 (September 2024).
Safe Sign Technologies acquired by Thomson Reuters (August 2024)!
Bad Students make great teachers accepted to ECCV (June 2024)!
SMAT accepted to ICML (May 2024)!
Invited talk at the Hong Kong University of Science and Technology (HKUST) (April 2024).
Releasing our new perspective paper on Empowering Scientific Discovery with AI Agents (April 2024).
Releasing SMAT, our new state-of-the-art Meta-Learner (March 2024) !
Announcing the Harvard Efficient ML Seminar Series (March 2024) !
Announcing MAC, our new RAG/Online Adaptation method for LLMs (March 2024).
C3 accepted to CVPR (March 2024).
Invited talk at MIT on continual learning for LLMs (February 2024).
Invited talk at the Stanford Information Theory Forum on neural data compression with INRs (January 2024).
Invited talk at Mistral on Bad Students Make Great Teachers (January 2024).

Research

Selected papers are highlighted.

Scales++: Compute Efficient Evaluation Subset Selection with Cognitive Scales Embeddings
Andrew M. Bean, Nabeel Seedat, Shengzhuang Chen, Jonathan Richard Schwarz°

🔗 Project Website

arXiv 2025

ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization
Shengzhuang Chen*, Xu Ouyang*, Michael Arthur Leopold Pearce, Thomas Hartvigsen, Jonathan Richard Schwarz

🔗 Project Website
💻 Code
📊 Data

TMLR 2025

*: Joint first authorship

Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses
Fangyi Yu, Nabeel Seedat, Dasha Herrmannova, Frank Schilder, Jonathan Richard Schwarz

EMNLP 2025

Automatic Expert Discovery in LLM Upcycling via Sparse Interpolated Mixture-of-Experts

Shengzhuang Chen, Ying Wei°, Jonathan Richard Schwarz°

🔗 Project Website

Association for Computational Linguistics (ACL) 2025 (Oral)

° : Joint senior authorship

Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts
Shengzhuang Chen, Jihoon Tack, Yunqiao Yang, Yee Whye Teh, Ying Wei°, Jonathan Richard Schwarz°

💻 Code
🔗 Project Website

ICML 2024

° : Joint senior authorship

CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
David Brandfonbrener, Hanlin Zhang, Andreas Kirsch, Jonathan Richard Schwarz, Sham Kakade

NeurIPS 2024

Online Adaptation of Language Models with a Memory of Amortized Contexts
Jihoon Tack, Jaehyung Kim, Eric Mitchell, Jinwoo Shin, Yee Whye Teh, Jonathan Richard Schwarz

🗣️ Slides
🎞️ Video
💻 Code
🔗 Project Website

NeurIPS 2024

Empowering Biomedical Discovery with AI Agents
Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, Marinka Zitnik

Cell 2024

Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
Talfan Evans, Shreya Pathak, Hamza Merzic, Jonathan Richard Schwarz, Ryutaro Tanno, Olivier J. Henaff

ECCV 2024

C3: High-performance and low-complexity neural compression from a single image or video
Hyunjik Kim, Matthias Bauer, Lucas Theis, Jonathan Richard Schwarz, Emilien Dupont

💻 Code
🔗 Project Website

CVPR 2024

Efficient Meta-Learning via Error-based Context Pruning for Implicit Neural Representations
Jihoon Tack, Subin Kim, Sihyun Yu, Jaeho Lee, Jinwoo Shin, Jonathan Richard Schwarz

🗣️ Slides
💻 Code

NeurIPS 2023

Secure Out-of-Distribution Task Generalization with Energy-Based Models
Shengzhuang Chen, Long-Kai Huang, Jonathan Richard Schwarz, Yilun Du, Ying Wei

NeurIPS 2023

Modality-Agnostic Variational Compression of Implicit Neural Representations (VC-INR)
Jonathan Richard Schwarz*, Jihoon Tack*, Yee Whye Teh, Jaeho Lee, Jinwoo Shin

ICML 2023

* : Joint first authorship

Spatial Functa: Scaling Functa to ImageNet Classification and Generation
Matthias Bauer*, Emilien Dupont, Andy Brock, Dan Rosenbaum, Jonathan Richard Schwarz, Hyunjik Kim*

arXiv 2023

Meta-Learning Sparse Compression Networks (MSCN)
Jonathan Richard Schwarz, Yee Whye Teh

Transactions on Machine Learning Research (TMLR) 2022

Behavior Priors for Efficient Reinforcement Learning
Dhruva Tirumala, Alexandre Galashov, Hyeonwoo Noh, Leonard Hasenclever, Razvan Pascanu, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech Marian Czarnecki, Arun Ahuja, Yee Whye Teh, Nicolas Heess

Journal of Machine Learning Research (JMLR) 2022

Powerpropagation: A sparsity inducing weight reparameterisation
Jonathan Richard Schwarz, Siddhant M. Jayakumar, Razvan Pascanu, Peter E. Latham, Yee Whye Teh

Neural Information Processing Systems (NeurIPS) 2021

💻 Code

Functional Regularisation for Continual Learning using Gaussian Processes
Jonathan Richard Schwarz*, Michalis K. Titsias*, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh

International Conference on Learning Representations (ICLR) 2020

💻 Code

* : Joint first authorship

Multiplicative Interactions and Where to Find Them
Siddhant M. Jayakumar, Wojciech M. Czarnecki, Jacob Menick, Jonathan Richard Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, Razvan Pascanu

International Conference on Learning Representations (ICLR) 2020

Meta-Learning surrogate models for sequential decision making
Jonathan Richard Schwarz*, Alexandre Galashov*, Hyunjik Kim, Marta Garnelo, David Saxton, Pushmeet Kohli, SM Ali Eslami°, Yee Whye Teh°

ICLR 2019 Workshop on Structure & Priors in Reinforcement Learning

*, ° : Joint first/senior authorship

	Experience replay for continual learning David Rolnick, Arun Ahuja, Jonathan Richard Schwarz, Timothy P. Lillicrap, Greg Wayne Neural Information Processing Systems (NeurIPS) 2019
	Information asymmetry in KL-regularized RL Alexandre Galashov, Siddhant M Jayakumar, Leonard Hasenclever, Dhruva Tirumala, Jonathan Richard Schwarz, Guillaume Desjardins, Wojciech M Czarnecki, Yee Whye Teh, Razvan Pascanu, Nicolas Heess International Conference on Learning Representations (ICLR) 2019
	Empirical Evaluation of Neural Process Objectives Tuan Anh Le, Hyunjik Kim, Marta Garnelo, Dan Rosenbaum, Jonathan Richard Schwarz, Yee Whye Teh NeurIPS 2018 workshop on Bayesian Deep Learning
	Attentive Neural Processes Hyunjik Kim, Andriy Mnih, Jonathan Richard Schwarz, Marta Garnelo, SM Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh International Conference on Learning Representations (ICLR) 2019 💻 Code
	Neural Processes Marta Garnelo, Jonathan Richard Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, Yee Whye Teh ICML 2018 Workshop on Theoretical Foundations and Applications of Deep Generative Models (Spotlight talk) 🗣️ Talk (credit to Marta) 💻 Code
	Progress & Compress: A scalable framework for continual learning Jonathan Richard Schwarz, Jelena Luketina, Wojciech M. Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Raia Hadsell°, Razvan Pascanu° International Conference on Machine Learning (ICML) 2018 (Long oral) 🗣️ Talk 📊 Data (Sequential Omniglot) ° : Joint senior authorship
	The NarrativeQA Reading Comprehension Challenge Tomas Kocisky, Jonathan Richard Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, Edward Grefenstette 🗣️ Talk (credit to Tomas) 📊 Data Transactions of the Association for Computational Linguistics (TACL) 2018
	A Recurrent Variational Autoencoder for Human Motion Synthesis Ikhsanul Habibie, Daniel Holden, Jonathan Richard Schwarz, Joe Yearsley, Taku Komura 💻 Code 📊 Data British Machine Vision Conference (BMVC) 2017

Academic Workshops

NeurIPS 2024 Compositional Learning: Perspectives, Methods, and Paths Forward

Jonathan Richard Schwarz, Ying Wei, Yilun Du, Laurent Charlin, Mengye Ren, Matthias Bethge

ICLR 2023 Neural Fields across Fields: Methods and Applications of Implicit Neural Representations

Jonathan Richard Schwarz, Hyunjik Kim, Emilien Dupont, Thu Nguyen-Phuoc, Vincent Sitzman, Srinath Sridhar

NeurIPS 2021 Workshop on Meta Learning

Jonathan Richard Schwarz, Fábio Ferreira, Erin Grant, Frank Hutter, Joaquin Vanschoren, Huaxiu Yao

NeurIPS 2020 Workshop on Meta Learning

Jonathan Richard Schwarz, Roberto Calandra, Jeff Clune, Erin Grant, Joaquin Vanschoren, Francesco Visin, Jane Wang

ICML 2020 Workshop on Continual Learning

Jonathan Richard Schwarz, Rahaf Aljundi, Eugene Belilovsky, Arslan Chaudhry, Puneet Dokania, Sayna Ebrahimi, Haytham Fayek, David Lopez-Paz , Marc Pickett

Based on Jon Barron's website.