Steffen Jung

Experiments in Fluids2026

STELLA: a modular framework for SpatioTemporal Event-based Lagrangian particLe trAcking

Sebastian Sachs, Steffen Jung, Max Kahl, Margret Keuper, Christian Willert, Christian Cierpka

DOI

Event cameras report brightness changes asynchronously, pixel by pixel, instead of whole frames, which makes them attractive for following thousands of tiny tracer particles through a turbulent flow. STELLA puts the two competing ways of handling that stream — accumulating events into dense representations, or clustering the raw events directly — inside one modular tracking pipeline, together with synthetic and experimental datasets whose ground truth makes them comparable for the first time. Neither approach wins outright: dense representations track high-frequency periodic motion best, while direct event processing follows more than 900 particles in a cylinder wake at uncertainties comparable to high-speed-camera PTV.

TMLR — Transactions on Machine Learning Research2025

Corner Cases: How Size and Position of Objects Challenge ImageNet-Trained Models

Mishal Fatima, Steffen Jung, Margret Keuper

OpenReviewarXivcode

Datasets inherit the habits of the people who photographed them: objects tend to sit near the center and to fill a good part of the frame, and how far a class departs from that varies. This paper builds Hard-Spurious-ImageNet, a set derived from ImageNet-1k in which object position and size are varied independently of the background, and finds that pretrained models fall back on background cues precisely when the object is small and far from the center. Methods meant to suppress spurious features account for neither factor, so their gains on worst-group accuracy largely evaporate under exactly those conditions.

ICLR — International Conference on Learning Representations2025

Can We Talk Models Into Seeing the World Differently?

Paul Gavrikov, Jovita Lukasik, Steffen Jung, Robert Geirhos, M. Jehanzeb Mirza, Margret Keuper, Janis Keuper

OpenReviewarXiv

Vision–language models inherit the texture-versus-shape bias of the vision encoder they are built on, but not straightforwardly: joint training with language shifts the balance by itself, so a VLM leans further towards shape than its own encoder does. The more useful question is whether a user can move that balance at will, and the answer turns out to be asymmetric — prompting a model towards texture-based decisions works reasonably well, while pushing it further towards shape barely helps.

WACV — IEEE/CVF Winter Conference on Applications of Computer Vision2025

FAIR-TAT: Improving Model Fairness Using Targeted Adversarial Training

Tejaswini Medi, Steffen Jung, Margret Keuper

proceedingsarXiv

Adversarial training buys robustness at the cost of fairness: the classes a model already found easy take most of the benefit, while the hard ones are left more exposed than before — which is exactly where an attacker will aim. FAIR-TAT builds the training perturbations with targeted attacks rather than untargeted ones, choosing which class each sample is pushed towards, so robustness is spread more evenly across classes. The trade also holds up better than standard adversarial training when the model later meets threats it was not trained on, common corruptions among them.

ICML — International Conference on Machine Learning2024

CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks

Shashank Agnihotri, Steffen Jung, Margret Keuper

proceedingsarXivcode

Adversarial attacks such as PGD were built around a single prediction per image, so applying them to dense tasks — segmentation, optical flow, disparity — tends to concentrate the perturbation on a few pixels rather than spread it across the image, and the fixes for that have cost optimization stability. CosPGD scales each pixel's loss by a cosine alignment score between its prediction and the target, which is smooth and fully differentiable, so the attack balances itself over the whole image domain without that trade-off. It outperforms the previous state of the art on semantic segmentation and, unlike earlier dedicated attacks, carries over to pixel-wise regression.

CVPR Workshop — CVPR 2023 Workshop on Generative Models for Computer Vision2023

Happy People — Image Synthesis as Black-Box Optimization Problem in the Discrete Latent Space of Deep Generative Models

Steffen Jung, Jan Christian Schwedhelm, Claudia Schillings, Margret Keuper

CVPR 2023 Workshop on Generative Models for Computer Vision

arXiv

Optimizing in the latent space of a generative model is a standard way to attack expensive black-box problems, but it is bounded by the training distribution — you can only ask for samples the model has already seen. This work treats the discrete latent space of a vector-quantized VAE as a mathematical program over tree-based ensembles, which can be solved globally, then retrains on those solutions to shift the distribution towards the property being maximized. The demonstration — coaxing a generator trained only on mildly happy faces into producing broadly smiling ones — is deliberately frivolous, but FID to the unseen target distribution improves from 50.5 to 41.7.

ICLR — International Conference on Learning Representations2023

Neural Architecture Design and Robustness: A Dataset

Steffen Jung, Jovita Lukasik, Margret Keuper

OpenReviewarXivproject page

Measuring whether an architecture is robust costs far more than measuring whether it is accurate, which is why design choices have mostly been studied for clean accuracy alone. This paper evaluates all 6,466 non-isomorphic networks in the NAS-Bench-201 search space against a range of adversarial attacks and corruption types and releases the result as a database, so questions about robustness and topology can be answered by lookup instead of by retraining. Topology matters more than one might expect: at equal parameter count, mean adversarial robust accuracy ranges from 20% to 41%.

ECCV — European Conference on Computer Vision2022

Learning Where To Look — Generative NAS is Surprisingly Efficient

Jovita Lukasik, Steffen Jung, Margret Keuper

proceedingsarXivposter

Architecture search splits into two camps — surrogate models that embed architectures and predict their performance, and generative models that let you optimize inside the latent space they draw from — each trading query efficiency against the quality of what gets proposed. This work pairs the two: a generator and a surrogate predictor that together learn to sample from progressively more promising regions, so fewer architectures have to be trained before a good one appears. It extends without modification to several objectives at once, accuracy together with hardware latency among them.

ECCV — European Conference on Computer Vision2022

FrequencyLowCut Pooling — Plug & Play against Catastrophic Overfitting

Julia Grabinski, Steffen Jung, Janis Keuper, Margret Keuper

proceedingsarXivposter

The pyramid down-sampling inside most CNNs quietly violates the sampling theorem, and the aliasing that follows correlates with how easily a model can be attacked. FrequencyLowCut pooling removes the high band before down-sampling so the operation is alias-free; it has no hyper-parameters and drops into any architecture. Paired with plain FGSM adversarial training it improves robustness substantially and, more usefully, prevents catastrophic overfitting — the collapse that normally makes cheap single-step adversarial training unusable.

ECML PKDD — European Conference on Machine Learning and Knowledge Discovery in Databases2022

Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks

Steffen Jung, Margret Keuper

proceedingsarXivposter

Partitioning a graph by minimum cost multicut is NP-hard, and the graph neural networks brought to combinatorial problems are almost all built for positive edge weights — the wrong shape for a problem whose costs are signed. This work adapts several GNN families to encode real-valued edge costs and reformulates the multicut ILP constraints as a polynomial program usable as a loss, which makes the multicut end-to-end trainable for the first time. The solutions are good rather than optimal, but they arrive far faster than LP solvers or tuned heuristics, and the gap widens as instances grow.

NeurIPS Workshop — NeurIPS 2021 Workshop on Distribution Shifts2021

Internalized Biases in Fréchet Inception Distance

Steffen Jung, Margret Keuper

NeurIPS 2021 Workshop on Distribution Shifts

OpenReviewposter

FID is the default way to rank generative models, but it inherits whatever the Inception v3 classifier it borrows its features from happens to be sensitive to. Measured across the ImageNet-C corruptions, that network barely moves for shifts in brightness, saturation or contrast, yet reacts sharply to anything touching high frequencies — added noise and blur alike — and it tolerates horizontal flips but not vertical ones, mirroring its own augmentation pipeline. So FID quietly prefers models that reproduce texture over models that reproduce color, and swapping Inception for a different classifier only exchanges one set of biases for another.

AAAI — AAAI Conference on Artificial Intelligence2021

Spectral Distribution Aware Image Generation

Steffen Jung, Margret Keuper

proceedingsarXivpostercode

Generated images can be separated from real ones almost trivially by their frequency spectra: common up-sampling schemes leave artifacts in the high band that the generator cannot correct on its own. Rather than penalizing deviation from an average spectrum, this paper adds a second discriminator that judges the frequency distribution directly — lightweight, modular, and stable across the usual GAN losses. The resulting models produce spectra much closer to real data, and are correspondingly harder to detect by that cue.