- 9:00-10:00 Invited talk: Przemysław Kazienko
- 10:00-10:15 Introduction
- 10:30-11:00 Coffee break
- 11:00-12:05 Short talks
- 12:30-13:30 Lunch
- 13:30-15:00 Poster session
- 15:00-15:30 Break
- 15:30-16:30 Panel Discussion
- 16:30-16:40 Closing
Invited talk:
Przemysław Kazienko
“Personalized NLP: context is all you need”
Some tasks in Natural Language Processing (NLP), like hate or offensive speech and emotional or funny text detection, are subjective by nature. Each human may perceive some content individually. The existing reasoning methods commonly rely on agreed output values, the same for all recipients. We propose fundamentally different — personalized solutions applicable to any subjective NLP task. Our deep learning models take into account not only the textual content but also the opinions and beliefs of a given person. They differ in their approaches to learning Human Bias (HuBi) and fusion with text representation. The experiments were carried out on dozens of tasks related to offensive, emotional, and humorous texts. The personalized HuBi methods radically outperformed the generalized ones for all NLP problems. Personalization also has a greater impact on reasoning quality than commonly explored pre-trained and fine-tuned language models. We discovered a high correlation between human bias calculated using our dedicated formula and that learned by the model. Multi-task solutions achieved better outcomes than single-task architectures. Analysis of human and word embeddings also provided additional insights. Further research on personalization of ChatGPT using prompt engineering with various few-shot approaches also proved that personalization appears to be obligatory while solving subjective problems in a human-centred manner. Simultaneously, personalization is only one component of the broader concept of contextual text processing.
Przemysław Kazienko, Ph.D. is a full professor and leader of ENGINE – the European Centre for Data Science and two research groups: HumaNLP and Emognition at The Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland. He authored over 300 research papers, including over 50 in journals with impact factor related to personalization and subjective tasks in NLP, ChatGPT, affective computing and emotion recognition, social/complex network analysis, spread of influence, collective classification, deep machine learning, sentiment analysis, DSS in medicine, finances and telecommunication, knowledge management, collaborative systems, recommender systems, information retrieval, and data security. He also intialized and led over 50 projects, including large European ones, chiefly in cooperation with companies with total local budget over €10M. He gave 20+ keynote/invited talks for international audience and served as a co-chair of over 20 international scientific conferences and workshops. He is an IEEE Senior Member, on the board of Network Science Society and a member of the Editorial Board of several scientific journals.
Accepted Papers
- Estimating p-values from a Single Test Set with Item and Response Variance
Shira Wein, Christopher Homan, Lora Aroyo and Chris Welty - Capturing Human Perspectives in NLP: Questionnaires, Annotations, and Biases
Wiktoria Mieleszczenko-Kowszewicz, Kamil Kanclerz, Julita Bielaniewicz, Marcin Oleksy, Marcin Gruza, Stanisław Woźniak, Ewa Dzięcioł, Przemyslaw Kazienko and Jan Kocon - Social or Individual Disagreement? Perspectivism in the Annotation of Sexist Jokes
Berta Chulvi, Lara Fontanella, Roberto Labadie and Paolo Rosso - Hierarchical Clustering of Label-based Annotator Representations for Mining Perspectives
Soda Marem Lo and Valerio Basile - Does Anyone See the Irony Here? Analysis of Perspective-aware Model Predictions in Irony Detection
Simona Frenda, Soda Marem Lo, Silvia Casola, Bianca Scarlini, Cristina Marco, Valerio Basile and Davide Bernardi - Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions
London Lowmanstone, Ruyuan Wan, Risako Owan, Jaehyung Kim and Dongyeop Kang - No Two Sides: Challenging the Subjectivity of Isms in Online Abuse and Harms Research
Amanda Cercas Curry, Gavin Abercrombie and Zeerak Talat - Whose Taste? Perspectivism and the Evaluation of Image Aesthetics Assessment Models in Computer Vision
Samuel Goree, Weslie Khoo and David J. Crandall - Uni-Coun: Unimodalities Count as Perspectives in Multimodal Emotion Annotation
Quanqi Du, Sofie Labat, Thomas Demeester and Veronique Hoste
Proceedings
The proceedings of the workshop are online on CEUR-ws at this link:
https://ceur-ws.org/Vol-3494/