Vera Neplenbroek

Vera Neplenbroek

PhD candidate in Safety for Conversational AI

Publications

You can also find my publications on my Google Scholar profile.

Conference Papers

Actionable Interpretability for Churn Classification: A Text Bottleneck Model Case Study at a Major Telecom Provider

Published in ACL 2026 Industry Track, 2026

In this work, we explore the application of Text Bottleneck Models (TBMs) to customer service calls, showing that it is possible to combine accurate churn classification with actionable, human-interpretable explanations. Rather than relying solely on black-box predictions, our approach distills conversations into a sparse set of interpretable concepts and provides snippet-based evidence for each classification. In a case study at a large European telecommunications provider, we demonstrate that TBMs achieve competitive predictive performance while supporting practical business use cases such as automated call profiling and an interactive stakeholder dashboard.

Adrian Sauter, **Vera Neplenbroek**, Georgios Vlassopoulos, and Gianluigi Bardelloni. 2026. Actionable Interpretability for Churn Classification: A Text Bottleneck Model Case Study at a Major Telecom Provider. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 1005–1024, San Diego, California, USA. Association for Computational Linguistics. https://aclanthology.org/2026.acl-industry.70/

One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization

Published in ACL 2026 Main, 2026

We compare using different cues to convey the same user persona and find that while cues are overall highly correlated, they produce substantial variance in responses across personas.

Franziska Weeber, Vera Neplenbroek, Jan Batzner, and Sebastian Padó. 2026. One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 44892–44921, San Diego, California, United States. Association for Computational Linguistics. https://aclanthology.org/2026.acl-long.2079/

Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization

Published in EMNLP 2025 Main, 2025

In this work we show that LLMs infer the user’s demographic attributes based on stereotypical signals in the conversation, which for a number of groups even persists when the user explicitly identifies with a different demographic group. We effectively mitigate this form of stereotype-driven implicit personalization by intervening on the model’s internal representations using a trained linear probe to steer them toward the explicitly stated identity.

Vera Neplenbroek, Arianna Bisazza, and Raquel Fernández. 2025. Reading Between the Prompts: How Stereotypes Shape LLM’s Implicit Personalization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20378–20411, Suzhou, China. Association for Computational Linguistics. https://aclanthology.org/2025.emnlp-main.1029/

Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation

Published in ACL 2025 Findings, 2024

Finetuning on specialized datasets can mitigate harmful behavior, and doing this in English can transfer to other languages. In this work we also observe this transfer and show that the extent to which transfer takes place can be predicted by the amount of data in a given language present in the model’s pretraining data. However, this transfer of bias and toxicity mitigation often comes at the expense of decreased language generation ability in non-English languages.

Vera Neplenbroek, Arianna Bisazza, and Raquel Fernández. 2025. Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 2805–2830, Vienna, Austria. Association for Computational Linguistics. https://aclanthology.org/2025.findings-acl.145

MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs

Published in COLM 2024, 2024

MBBQ (Multilingual Bias Benchmark for Question-answering) is a carefully curated version of the English BBQ dataset extended to Dutch, Spanish, and Turkish, which measures stereotypes commonly held across these languages. Our results based on several open-source and proprietary LLMs confirm that some non-English languages suffer from bias more than English, and that there are significant cross-lingual differences in bias behaviour for all except the most accurate models.

Neplenbroek, V., Bisazza, A. and Fernández, R., 2024. MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs. In the first Conference on Language Modeling (COLM) 2024. https://openreview.net/pdf?id=X9yV4lFHt4

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

Published in ACL 2025 Main, 2024

We provide JUDGE-BENCH, a collection of 20 NLP datasets with human annotations, and comprehensively evaluate 11 current LLMs, covering both open-weight and proprietary models, for their ability to replicate the annotations. Our evaluations show that each LLM exhibits a large variance across datasets in its correlation to human judgments. We conclude that LLMs are not yet ready to systematically replace human judges in NLP.

Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, Andre Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, and Alberto Testoni. 2025. LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 238–255, Vienna, Austria. Association for Computational Linguistics. https://aclanthology.org/2025.acl-short.20

Pre-prints

Journal Articles