The key idea is in how the model is trained. Earlier looped models asked every round to produce the final answer straight away, and they tended to fall apart if you let them loop more often than they had practised. LiFT instead gives each round its own small task: move a set fraction of the way from the model’s first rough guess toward the correct answer. Because that “fraction of the way” is a continuous dial rather than a fixed count, a trained model can afterwards be run with far more loops than it ever saw during training, and it keeps getting better. One of our models, trained with just two loops, improved its image quality score (FID, where lower is better) from 17.1 to 9.3 when allowed sixteen loops, with no retraining at all.

The payoff is a small model that outperforms a big one. On the standard ImageNet benchmark, a mid-sized LiFT model produced better images than a conventional model more than twice its size, with roughly 60% fewer parameters, 32% less compute to train and 52% less compute to generate each image. That makes the approach attractive wherever memory is scarcer than processing time, such as drones, robots and other devices at the edge, and you can turn the number of loops up or down depending on how much time you have. There are honest caveats: at the smallest model size the conventional approach still wins, and so far we have tested only on 256×256 images of the ImageNet classes. Next we want to find out whether “think longer instead of growing bigger” also works for video and for models that simulate how the world moves.

Demo. Video.

Our method ranks every building block of a transformer, meaning each individual neuron and each attention head, on a single common scale of importance. Once you have that ranking, you can shrink the model to any budget by removing units from the bottom of the list. The challenge is finding a good ranking, since units work together in ways that simple importance scores miss. We solve this with “soft pruning”. Instead of deleting units outright during the search, we gradually dim them. This lets us use ordinary gradient descent to adjust the ranking until the model performs well across many budgets at once, from lightly trimmed to heavily pruned. The original weights are never touched: no retraining and no fine-tuning. Building an elastic model takes minutes, and the cost grows slower than the model’s size.

The results are striking. We tested 27 vision and text models ranging from 22 million to 8 billion parameters. Performance degrades smoothly up to 60% sparsity, where existing pruning methods tend to collapse. Pruning the large DINOv3 vision model by half removes 427 million parameters, yet its ImageNet accuracy drops by only 0.8%. The savings also show up on real hardware. A pruned ViT-B/16 runs up to 1.8× faster on an A100 GPU, and switching between budgets takes less than half a millisecond. Two further findings stand out. For search and retrieval, slimmed-down query encoders stay compatible with an index built by the full model, so you can scale queries up or down without rebuilding anything. And within one model family, a pruned large model can beat a smaller model trained at full size. Decoder-based text embedders remain harder to prune deeply, which is a puzzle we are still working on. The broader lesson is encouraging: much of the model you already have is spare capacity you can use when you need it.

EDMA first finds the photo showing the arch in St. Louis, then uses that to pick the right row from the airline’s destination table, relates St. Louis to its state, and answers ‘Missouri’. It also reports that the photo did most of the work, cutting the remaining uncertainty by more than half, so a user can see exactly why the answer is what it is.

The method answers complex questions by combining different kinds of information at once, such as images, tables and text reports, and, unlike current AI systems, it shows step by step which source contributed how much to the answer. It also comes with a built-in confidence signal: when the method is less certain, the answer is measurably more likely to be wrong, so the system can flag when a human should take a closer look. On two challenging benchmarks, EDMA gave clearly better answers than the best existing systems, such as chain-of-thought, by roughly 20 points on the standard score, and it works with a wide range of AI models, including open-source ones that can run on our own infrastructure.

To answer complex questions, it applies Multimodal LLMs (MLLMs) by iterative logical-entropy reduction. It extracts answer requirements, constructs modality-specific partitions, performs cross-modal conditioning, enumerates answering trajectories, and selects the trajectory with the lowest terminal entropy. Experiments on MultiModalQA and ManyModalQA report average gains of 12.8 F1 over the strongest prompting baseline and 19.8 F1 over prior systems. The core idea is interesting and reasonably interpretable: logical entropy provides a model-agnostic way to quantify how different modalities narrow the candidate space, without relying on calibrated model probabilities. The paper also goes beyond aggregate accuracy by examining entropy–error correlations and alternative trajectory choices.

EDMA: Entropy-Driven Multimodal Answering. Emanuele Mezzi, Gertjan Burghouts, Fabio Massacci, Mengyuan Zhang. NeurIPS 2026.

In our work on OASIC, we take a different perspective. Instead of forcing a single model to handle all cases, we first ask: how much of the object is actually visible? A system that sees almost everything should rely on fine details, while one that sees only fragments must fall back on coarse, robust cues. By estimating the severity of occlusion, filtering out distracting patterns, and selecting a classifier tailored to that situation, the system adapts its strategy.

This simple shift — matching perception to visibility — turns out to matter. In real-world settings like autonomous driving or robotics, objects are rarely fully visible. Teaching AI not just to see, but to understand how well it sees, is a small but important step toward making it work reliably in the messy, occluded world we actually live in.

We therefore developed a traversability system that explicitly takes the robot’s embodiment into account. Before deployment, an LLM reasons about the robot’s morphology and capabilities and translates this into relevant concepts for perception: what surfaces can it handle, what objects are obstacles, and what situations require caution.

We then combine three complementary foundation models. YOLO-E provides fast, open-vocabulary object detection, while SAM3 turns relevant concepts into precise pixel-level segmentation. A VLM adds an open-world safety net: when the scene contains something that was not anticipated by the prior knowledge—say, a cardboard box suddenly blocking the road—it can reason about its meaning and enrich the traversability analysis. The result is a robot-specific map of the scene, where green means “go ahead” and red means “better not”.

The interesting part is that there is no single “correct” traversability map. Traversability is a relationship between the world and the robot. The same staircase, branch, puddle or box can be perfectly manageable for one robot and a hard obstacle for another. By combining LLM reasoning, open-vocabulary perception, segmentation and VLM-based open-world reasoning, we are moving from robots that merely see their environment toward robots that understand what that environment means for them.

This new work shows why. In open-vocabulary detectors, language and vision are mixed early inside the model using attention. The words you provide don’t just guide the model — they leak everywhere, nudging it to find a match even when none exists. The model isn’t hallucinating at random; it’s being gently pushed into seeing patterns that aren’t there.

The fix turns out to be surprisingly simple. The authors introduce attention sinks: special tokens added to the prompt that act like drains for misplaced attention. When there’s no real visual match, attention flows into these sinks instead of being spread across the image. No retraining, no new data — just a smarter way to control where attention goes.

The result is a big drop in false positives and a model that’s much better at admitting when something isn’t there. Sometimes, making AI more reliable doesn’t mean teaching it more — it just means giving its attention somewhere safe to go.

A visualisation of the attention scores of each head (horizontal) and scale (vertical) from the first vision-language fusion layer of LLMDet, between the visual features of an image of a golden retriever and the prompt tokens “dog” and “tractor”, without using attention sinks (left) and with attention sinks (right). Both the positive and negative class have a much cleaner attention map after adding attention sinks, with most irrelevant information (in case of the tractor prompt) being routed away from the negative classes to the attention sinks.

Accepted at ICLR 2026, see Publications.

Although modern language models are trained purely on text, that text richly describes the visual world: objects, scenes, and how things relate. The question explored in this work is whether such models already contain visual structure that can be quickly unlocked with a small amount of image data.

The answer is yes. By adding a lightweight vision–language alignment step, researchers show that stronger language models require far less image–text training to perform well on visual recognition tasks. As language models improve, their representations align more naturally with visual concepts, making the final learning step remarkably efficient.

The technological breakthrough is clear: instead of training massive vision–language systems from scratch, we can reuse powerful language models and adapt them to vision with minimal extra supervision.

This points toward more data-efficient, scalable AI systems—and a future where language provides a surprisingly strong foundation for visual understanding.

The proposed method is about situation understanding. We reason within images, focusing on the relationships between objects. Relationships between objects are important for interpreting situations. For example: a van next to a boat, farther away from the highway, may indicate an illegal offloading.

We have found that modern multimodal AI models perform rather poorly when it comes to interpreting such relationships between objects—especially when the relationships become more complex and involve multiple entities; for example: A is close to B and C, while B and C are aligned one behind the other, and C is of the same type as D.

That is why we take a different approach: we use logic. With first-order logic (FOL), you can precisely define the relationships of interest. We interpret this logic through neurosymbolic AI, which we place on top of modern AI. This layer combines the information extracted from images about the objects with the logic describing the interesting relationships.

Here is the key part: the user asks questions to the system in natural language. This textual query is translated into FOL by an LLM that we have specialized using in-context learning. Each image can then be analyzed to determine whether a relational pattern matching the user’s query is present. You can see this in the following figure:

Our method, NeCo, improved the DINO patch-level features by self-supervised post-training. We enforced that the patches in two views of the same image (i.e. different augmentations) follow the same ordering, relative to an external memory set of representative image patches. This ordering proves to be a very strong learning signal: it is much stronger than just looking at the nearest neighbors only, or looking at clusters of patches in embedding space.

Proud that it was cited by DINO v3!

The question is how to use them optimally for the task at hand. One challenge is that today’s large AI models are trained on everyday web images with common objects (cars, skateboards, types of cats, etc.). Clients often want something different: other images (e.g. drones that have other viewpoints) and/or other objects. It requires some tricks to adopt large AI models for our capabilities, we call these adoption strategies.

We have identified four strategies: (1) include pre- or post-processing, e.g. to apply the model to pre-filtered image regions, or to check or clean-up the predictions afterwards. (2) change the model behavior from outside, by changing the textual prompt, or improve the question. (3) re-learn part of the model, by adding a few learnable elements (e.g. prompt tuning, LoRa), or finetuning all weights (e.g. few-shot learning). (4) change the model itself, i.e. its architecture, for example to modify the tokenizer which enables to learn without forgetting (e.g. Grounding DINO extension that it can learn new objects without forgetting known objects).

During this lecture, I have discussed these four strategies.

Course website.