Programme

The workshop is in room A305
Poster session is in the big room on boards 137 – 152

Underline

9:00-9:30	Introduction
9:30-10:00	Shared Task Description – LeWiDi-2025 at NLPerspectives: The Third Edition of the Learning with Disagreements Shared Task Siyao Peng Paper
10:00-10:30	Remote presentations (Workshop and Shared Task) – McMaster at LeWiDi-2025: Demographic-Aware RoBERTa Aadi Sanghani, Sarvin Azadi, Virendra Jethra, Charles Welch Paper – Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learning Taylor Sorensen, Yejin Choi Paper – From Disagreement to Understanding: The Case for Ambiguity Detection in NLI Chathuri Jayaweera, Bonnie J. Dorr Paper
10:30-11:00	Break
11:00-12:30	Keynote talk Cultural Awareness in Multilingual Language Models: A Perspectivist Personal Perspective Jose Camacho-Collados
12:30-14:00	Lunch break
14:00-15:30	Poster session – Non-directive corpus annotation to reveal individual perspectives with underspecified guidelines: the case of mental workload Iuliia Arsenteva, Caroline Dubois, Philippe Le Goff, Sylvie Plantin and Ludovic Tanguy Paper – DeliChess: A Multi-party Dialogue Dataset for Deliberation in Chess Problem Solving Xiaochen Zhu, Georgi Karadzhov, Tom Stafford and Andreas Vlachos – Calibration as a Proxy for Fairness and Efficiency in a Perspectivist Ensemble Approach to Irony Detection Samuel Jesus, Guilherme Dal bianco, Wanderlei Soares, Valerio Basile and Marcos André Gonçalves Paper – SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentation Yizhe Zhang and Navdeep Jaitly Paper – Aligning NLP Models with Target Population Perspectives using PAIR: Population-Aligned Instance Replication Stephanie Eckman, Bolei Ma, Christoph Kern, Rob Chew, Barbara Plank and Frauke Kreuter Paper – Weak Ensemble Learning from Multiple Annotators for Subjective Text Classification Ziyi Huang, Nishanthi Rupika Abeynayake and Xia Cui Paper – Revisiting Active Learning under (Human) Label Variation Cornelia Gruber, Helen Alber, Bernd Bischl, Goeran Kauermann, Barbara Plank and Matthias Assenmacher Paper – Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement Gavin Abercrombie, Tanvi Dinkar, Amanda Cercas Curry, Dirk Hovy and Verena Rieser Paper – Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions Eve Fleisig, Matthias Orlikowski, Philipp Cimiano and Dan Klein Paper – Towards a Perspectivist Understanding of Irony through Rhetorical Figures Pier Felice Balestrucci, Michael Oliverio, Elisa Chierchiello, Eliana Di Palma, Luca Anselma, Valerio Basile, Cristina Bosco, Alessandro Mazzei and Viviana Patti Paper – CINEMETRIC: A Framework for Multi-Perspective Evaluation of Conversational Agents using Human-AI Collaboration Vahid Sadiri Javadi, Zain Ul Abedin and Lucie Flek Paper – A Disaggregated Dataset on English Offensiveness Containing Spans Pia Pachinger, Janis Marc Goldzycher, Anna Maria Planitzer, Julia Neidhardt and Allan Hanbury Paper – DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning Daniil Ignatev, Nan Li, Hugh Mee Wong, Anh Dang, Shane Kaszefski Yaschuk Paper – BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet) Tomas Ruiz, Siyao Peng, Barbara Plank, Carsten Schwemmer Paper – PromotionGo at LeWiDi-2025: Enhancing Multilingual Irony Detection with Data-Augmented Ensembles and L1 Loss Ziyi Huang, Nishanthi Rupika Abeynayake, Xia Cui Paper – Uncertain (Mis)Takes at LeWiDi-2025: Modeling Human Label Variation With Semantic Entropy Ieva Raminta Staliunaite, Andreas Vlachos Paper
15:30-16:00	Break
16:00-16:30	Remote presentations (Workshop and Shared Task) – Hypernetworks for Perspectivist Adaptation Daniil Ignatev, Denis Paperno and Massimo Poesio Paper – NLP-ResTeam at LeWiDi-2025:Performance Shifts in Perspective Aware Models based on Evaluation Metrics Olufunke O. Sarumi, Charles Welch, Daniel Braun Paper – LPI-RIT at LeWiDi-2025: Improving Distributional Predictions via Metadata and Loss Reweighting with DisCo Mandira Sawkar, Samay U. Shetty, Deepak Pandita, Tharindu Cyril Weerasooriya, Christopher Homan Paper
16:30-17:20	Panel discussion Jose Camacho-Collados, Eve Fleisig, Beiduo Chen
17:20-17:30	Closing remarks

Accepted papers

• Non-directive corpus annotation to reveal individual perspectives with underspecified guidelines: the case of mental workload
  Iuliia Arsenteva, Caroline Dubois, Philippe Le Goff, Sylvie Plantin and Ludovic Tanguy
• DeliChess: A Multi-party Dialogue Dataset for Deliberation in Chess Problem Solving
  Xiaochen Zhu, Georgi Karadzhov, Tom Stafford and Andreas Vlachos
• Calibration as a Proxy for Fairness and Efficiency in a Perspectivist Ensemble Approach to Irony Detection
  Samuel Jesus, Guilherme Dal bianco, Wanderlei Soares, Valerio Basile and Marcos André Gonçalves
• SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentation
  Yizhe Zhang and Navdeep Jaitly
• Hypernetworks for Perspectivist Adaptation
  Daniil Ignatev, Denis Paperno and Massimo Poesio
• Aligning NLP Models with Target Population Perspectives using PAIR: Population-Aligned Instance Replication
  Stephanie Eckman, Bolei Ma, Christoph Kern, Rob Chew, Barbara Plank and Frauke Kreuter
• Weak Ensemble Learning from Multiple Annotators for Subjective Text Classification
  Ziyi Huang, Nishanthi Rupika Abeynayake and Xia Cui
• Revisiting Active Learning under (Human) Label Variation
  Cornelia Gruber, Helen Alber, Bernd Bischl, Goeran Kauermann, Barbara Plank and Matthias Assenmacher
• Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
  Gavin Abercrombie, Tanvi Dinkar, Amanda Cercas Curry, Dirk Hovy and Verena Rieser
• Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions
  Eve Fleisig, Matthias Orlikowski, Philipp Cimiano and Dan Klein
• From Disagreement to Understanding: The Case for Ambiguity Detection in NLI
  Chathuri Jayaweera and Bonnie J. Dorr
• Towards a Perspectivist Understanding of Irony through Rhetorical Figures
  Pier Felice Balestrucci, Michael Oliverio, Elisa Chierchiello, Eliana Di Palma, Luca Anselma, Valerio Basile, Cristina Bosco, Alessandro Mazzei and Viviana Patti
• CINEMETRIC: A Framework for Multi-Perspective Evaluation of Conversational Agents using Human-AI Collaboration
  Vahid Sadiri Javadi, Zain Ul Abedin and Lucie Flek
• A Disaggregated Dataset on English Offensiveness Containing Spans
  Pia Pachinger, Janis Marc Goldzycher, Anna Maria Planitzer, Julia Neidhardt and Allan Hanbury

Invited talk

Jose Camacho-Collados

“Cultural Awareness in Multilingual Language Models: A Perspectivist Personal Perspective”

Abstract

Language models have become ubiquitous in NLP and beyond. In particular, the new wave of large language models (LLMs) are increasingly used to communicate and solve practical problems in many languages and countries, and by an increasingly diverse set of users. However, even though there is no doubt that these models open up plenty of opportunities, there are important issues and research questions that arise when it comes to LLMs and their application in different languages and cultures. For instance, the language coverage in language models drastically decreases for less-resourced languages and as such, their performance. And not only the general performance is affected, but general-purpose LLMs may be implicitly biased to specific cultures and languages depending on their underlying training data.

In this talk, I will discuss how language models reflect on cultural diversity, including potential shortcomings and how language coverage and cultural awareness may be intrinsically intertwined. I will also share some lessons learned based on recent research in this area – in particular, I will focus on the development of BLEnD, a large effort to develop a cultural benchmark of everyday knowledge for dozens of languages and countries.

---

Jose Camacho-Collados is a UKRI Future Leaders Fellow and Professor at the School of Computer Science of Cardiff University, where he co-founded the Cardiff Natural Language Processing group (Cardiff NLP). Before joining Cardiff University, he completed his PhD in Sapienza University of Rome and was a Google AI PhD Fellow.

Jose has worked in multiple NLP areas with a particular focus on semantics, multilinguality and computational social science with an interdisciplinary perspective. In this area, he has been developing specialised and efficient NLP models for social media applications, such as TweetNLP and related efforts. His work has received several recognitions, including awards at top NLP conferences, or the 2023 AIJ Prominent Paper Award. He is also the co-author of the “Embeddings in Natural Language Processing” book.