| Vol-3681 urn:nbn:de:0074-3681-7 Copyright © 2023 for the individual papers by the papers' authors. Copyright © 2023 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# FIRE-WN 2023
FIRE 2023 Working Notes

### Working Notes of FIRE 2023 - Forum for Information Retrieval Evaluation (FIRE-WN 2023)

### Goa, India, December 15-18, 2023.

Edited by

### Kripabandhu Ghosh****
Thomas Mandl **
Prasenjit Majumder *
Mandar Mitra ***

**** Indian Institute of Science Education and Research, Kolkata, India
* Dhirubhai Ambani Institute of Information and Communication Technology, Gandhinagar, India
** University of Hildesheim, Hildesheim, Germany
*** Indian Statistical Institute, Kolkata, India

## Table of Contents

- Preface
  Kripabandhu Ghosh, Thomas Mandl, Prasenjit Majumder, Mandar Mitra

### Artificial Intelligence on Social Media (AISoMe)

- Overview of the FIRE 2023 Track: Artificial Intelligence on Social Media (AISoMe) 1-5
  Moumita Basu, Soham Poddar, Moumita Basu, Saptarshi Ghosh, Kripabandhu Ghosh
- Enhancing Multilabel Classification of Anti-Vaccine Tweets with the COVID-Twitter-BERT 6-11
  Aritra Mandal
- Vaccine Vision: A deep learning approach towards identifying societal concerns regarding vaccines 12-17
  Kaustav Das
- Multi-Label Classification of COVID-Tweets Using Large Language Models 18-23
  Aniket Deroy, Subhankar Maity
- Tweet Classifier: Advancements in Multi-Label Analysis 24-29
  Swastik Anupam
- Multi-label Classification of Covid-19 Vaccine Tweet 30-43
  Sumit Das, Palvika Bansal, Vikas Rai, Shalini Kumari
- VaxTweetClassifier: BERT for Dealing with Vaccination Tweets 44-51
  Shankha S Das, Sohan Choudhury, Priyam Saha, Dipankar Das
- VaxVerdict: a RoBERTa Based Multilabel Tweet Classifier 52-58
  Aditya O Patil, Raj Awate, Shivakumar Ranade, Sheetal Sonawane
- VaxiBERT: A BERT-Based Classifier for Vaccine Tweets with Multi-Label Annotations 59-66
  Rajat Singh, Shivangi Bithel, Samidha Verma, Prachi
- Deciphering Vaccine Sentiments: Transformer Models in Social Media Analysis 67-74
  Shriram M S
- BERT-Powered Multi-label Classifier: Analyzing Public COVID Vaccination Discourse 75-81
  Ranjit Patro, Asutosh Mishra
- Analysing Crowd-Sourced Vaccine Data Using Machine Learning: Uncovering Concerns and Insights 82-90
  Lakshmi Gopal
- Unveiling Diverse Vaccine Sentiments: Multi-Label Text Classification 91-98
  K Shanmukha Naveen, S Sharon Roshini, S Karthika
- Decoding Concerns: Multi-label Classification of Vaccine Sentiments in Social Media 99-111
  Somsubhra De, Shaurya Vats
- TweetClass: COVID-19 Vaccine Tweet Classification with scikit-learn 112-117
  Baivab Chakraborty, Subhajit Srimani, Souvit Biswas

### Cross-lingual Information Retrieval for African Languages (CIRAL)

- Overview of the CIRAL Track at FIRE 2023: Cross-lingual Information Retrieval for African Languages 118-136
  Mofetoluwa Adeyemi, Akintunde Oladipo, Xinyu Crystina Zhang, David Alfonso-Hermelo, Mehdi Rezagholizadeh, Boxing Chen, Jimmy Lin
- Extending Translate-Train for ColBERT-X to African Language CLIR 137-146
  Eugene Yang, Dawn J. Lawrie, Paul McNamee, James Mayfield

### Uncovering Truth in Social Media through Claim Detection and Identification of Claim Spans (CLAIMSCAN)

- Overview of the CLAIMSCAN-2023: Uncovering Truth in Social Media through Claim Detection and Identification of Claim Spans 147-158
  Megha Sundriyal, Md Shad Akhtar, Tanmoy Chakraborty
- Current language models' poor performance on pragmatic aspects of natural language 159-169
  Albert Pritzkau, Julia Waldmüller, Olivier Blanc, Michaela Geierhos, Ulrich Schade
- Positional Transformers for Claim Span Identification 170-178
  Michael Sullivan, Navid Madani, Sougata Saha, Rohini Srihari

### Word-level Language Identification in Code-mixed Tulu Texts (CoLI-Tunglish)

- Overview of CoLI-Tunglish: Word-level Language Identification in Code-mixed Tulu Text at FIRE 2023 179-190
  Asha Hegde, F. Balouchzahi, Sharal Coelho, H.L. Shashirekha, Hamada A. Nayel, Sabur Butt
- Using Character Ngrams for Word-Level Language Identification in Trilingual Code-Mixed Data (and Even More) 191-197
  Yves Bestgen
- Word-Level Language Identification of Code-Mixed Tulu-English Data 198-204
  Poorvi Shetty
- BFCAI at CoLI-Tunglish@FIRE 2023: Machine Learning Based Model for Word-level Language Identification in Code-mixed Tulu Texts 205-212
  Ahmed M. Fetouh, Hamada Nayel
- Word-level Language Identification in Code-mixed Tulu Texts 213-222
  Sushma N, Asha Hegde, Hosahalli Lakshmaiah Shashirekha
- Advancing Language Identification in Code-Mixed Tulu Texts: Harnessing Deep Learning Techniques 223-230
  Supriya Chanda, Anshika Mishra, Sukomal Pal

### Scarcasm Identification of Dravidian Languages (Malayalam and Tamil) in DravidianCodeMix 2023 (DravidianCodeMix)

- Overview of Sarcasm Identification of Dravidian Languages in DravidianCodeMix@FIRE-2023 231-239
  Bharathi Raja Chakravarthi, Sripriya N, Bharathi B, Nandhini K, Subalalitha Chinnaudayar Navaneethakrishnan, Thenmozhi Durairaj, Rahul Ponnusamy, Prasanna Kumar Kumaresan, Kishore Kumar Ponnusamy, Charmathi Rajkumar
- Sarcasm Identification in Dravidian Languages Tamil and Malayalam 240-248
  Poorvi Shetty
- Sarcasm Detection in Dravidian Code-Mixed Text Using Transformer-Based Models 249-258
  Anik Basu Bhaumik, Mithun Das
- Cross-Linguistic Sarcasm Detection in Tamil and Malayalam: A Multilingual Approach 259-269
  Dhanya Krishnan, Krithika Dharanikota, B Bharathi
- Unmasking Sarcasm: Sarcastic Language Detection with BiLSTMs 270-277
  Anusha M D, Parameshwar R. Hegde
- Identifying the Type of Sarcasm in Dravidian Languages using Deep-Learning Models 278-286
  Ramya Sivakumar, C.Jerin Mahibha, B.Monica Jenefer
- Learning Models with Text Augmentation for Sarcasm Detection in Malayalam and Tamil Code-mixed Texts 287-298
  Navya N, Vanitha V, Asha Hegde, H L Shashirekha
- A Few Shot Learning to Detect Sarcasm in Tamil and Malayalam Code Mixed Data 299-305
  Shanmitha Thirumoorthy, Manavh N R, Durairaj Thenmozhi, Ratnavel Rajalakshmi
- Sarcasm Detection in Dravidian Languages using Transformer Models 306-318
  Madhumitha.M, Kunguma Akshatra.M, Tejashri.J, C.Jerin Mahibha, Durairaj Thenmozhi
- Sarcasm Identification in Codemix Dravidian Languages 319-326
  Prabhu Ram. N, Meera Devi. T, Kanisha. V, Meharnath. S, Manoji. B
- Sarcasm Identification Of Dravidian Languages (Malayalam and Tamil) 327-335
  V Indirakanth, Dharunkumar Udayakumar, Thenmozhi Durairaj, B. Bharathi
- Sarcasm Detection in Tamil and Malayalam Dravidian Code-Mixed Text 336-343
  Supriya Chanda, Anshika Mishra, Sukomal Pal

### Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages (HASOC)

- Overview of the HASOC Subtrack at FIRE 2023: Hate-Speech Identification in Sinhala and Gujarati 344-350
  Shrey Satapara, Hiren Madhu, Tharindu Ranasinghe, Alphaeus Eric Dmonte, Marcos Zampieri, Pavan Pandya, Nisarg Shah, Sandip Modha, Prasenjit Majumder, Thomas Mandl
- Overview of the HASOC Subtrack at FIRE 2023: Identification of Conversational Hate-Speech 351-359
  Hiren Madhu, Shrey Satapara, Pavan Pandya, Nisarg Shah, Thomas Mandl, Sandip Modha
- Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection 360-367
  Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar, Tanmoy Chakraborty
- Annihilate Hates (Task 4 HASOC 2023): Hate Speech Detection in Assamese Bengali and Bodo languages 368-382
  Koyel Ghosh, Apurbalal Senapati, Aditya Shankar Pal
- Hate and Offensive Content Identification in Indo-Aryan Languages using Transformer-based Models 383-392
  Olumide Ebenezer Ojo, Olaronke Oluwayemisi Adebanji, Hiram Calvo, Alexander Gelbukh, Anna Feldman, Grigori Sidorov
- Addressing Hate Speech: ATLANTIS for Efficient Hate Span Detection 393-402
  Niyar R Barman, Krish Sharma, Yashraj Poddar, Advaitha Vetagiri, Partha Pakray
- Detecting Hate Speech and Offensive Content in English and Indo-Aryan Texts 403-410
  Mohammadmostafa Rostamkhani, Sauleh Eetemadi
- Using Only Character Ngrams for Hate Speech and Offensive Content Identification in Five Low-Ressource Languages 411-417
  Yves Bestgen
- Detecting Offensive Language in Bengali Bodo and Assamese using Word Unigrams Char N-grams Classical Machine Learning and Deep Learning Methods 418-426
  Avigail Stekel, Avital Prives, Yaakov HaCohen-Kerner
- Harnessing Pre-Trained Sentence Transformers for Offensive Language Detection in Indian Languages 427-434
  Ananya Joshi, Raviraj Joshi
- Enhancing Hate Speech Detection in Sinhala and Gujarati: Leveraging BERT Models and Linguistic Constraints 435-444
  G Gnana Sai, Aswath Venkatesh, Kishore N, Olirva M, Balaji V A, Prabavathy Balasundaram
- Multilingual Hate Speech and Offensive Language Detection of Low Resource Languages 445-458
  Abhinav Reddy Gutha, Nidamanuri Sai Adarsh, Ananya Alekar, Dinesh Reddy
- Breaking Barriers: Multilingual Toxicity Analysis for Hate Speech and Offensive Language in Low-Resource Indo-Aryan Languages 459-473
  Ch Muhammad Awais, Jayveersinh Raj
- Examining Hate Speech Detection Across Multiple Indo-Aryan Languages in Tasks 1 & 4 474-485
  Gyandeep Kalita, Eisha Halder, Chetna Taparia, Advaitha Vetagiri, Dr. Partha Pakray
- Crossing Borders: Multilingual Hate Speech Detection 486-500
  Supriya Chanda, Abhishek Dhaka, Sukomal Pal
- Hate speech classification for Sinhalese and Gujarati 501-515
  Muhammad Deedahwar Mazhar Qureshi, Madhuri Sawant, M. Atif Qureshi, Wael Rashwan, Arjumand Younus, Simon Caton
- Hate Speech Detection in Low Resource Indo-Aryan Languages 516-520
  Sougata Saha, Michael Sullivan, Rohini Srihari
- Sinhala and Gujarati Hate Speech Detection 521-531
  M. Krithik Sathya, K.H. Gopalakrishnan, Manickam PA, Prabavathy Balasundaram
- Cross-Linguistic Offensive Language Detection: BERT-Based Analysis of Bengali Assamese & Bodo Conversational Hateful Content from Social Media 532-543
  Jhuma Kabir Mim, Mourad Oussalah, Akash Singhal
- Improving Detection of Hate Speech, Offensive Language and Profanity in Short Texts with SVM Classifier 544-552
  Surya Agustian, Zaky Idhafi, Agit Fadillah Rihardi
- Bengali Hate Speech Detection Using Deep Learning Technique 553-562
  Chandan Senapati, Utpal Roy
- Taming Toxicity: Learning Models for Hate Speech and Offensive Language Detection in Social Media Text 563-573
  Prajnashree M, Rachana K, Asha Hegde, Kavya G, Sharal Coelho, H L Shashirekha
- Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers 574-587
  Nikhil Narayan, Mrutyunjay Biswal, Pramod Goyal, Abhranta Panigrahi
- Multilingual Hate Speech Detection Using Ensemble of Transformer Models 588-597
  Md Saroar Jahan, Fadi Hassan, Walid Aransa, Abdessalam Bouchekif

### Information Retrieval in Software Engineering (IRSE)

- Generative AI for Software Metadata: Overview of the Information Retrieval in Software Engineering Track at FIRE 2023 598-604
  Srijoni Majumdar, Soumen Paul, Bhargav Dave, Debjyoti Paul, Ayan Bandyopadhyay, Samiran Chattopadhyay, Partha Pratim Das, Paul D Clough, Prasenjit Majumder
- A ML-LLM pairing for better code comment classification 605-614
  Hanna Abi Akl
- Software Metadata Classification based on Generative Artificial Intelligence 615-623
  Seetharam Killivalavan, Durairaj Thenmozhi
- Leveraging Generative AI: Improving Software Metadata Classification with Generated Code-Comment Pairs 624-632
  Samah Syed, Angel Deborah
- Source Code Comment Classification using machine learning algorithms 633-641
  Trisha Datta
- Enhancing Binary Code Comment Quality Classification: Integrating Generative AI for Improved Accuracy 642-651
  Rohith Arumugam S, Angel Deborah S
- Binary Classification of Source Code Comments using Machine Learning Models 652-660
  Lisa Sarkar
- A study of the impact of generative AI-based data augmentation on software metadata classification 661-669
  Tripti Kumari, Chakali Sai Charan, Ayan Das
- Leveraging Language Models for Code Comment Classification 670-678
  Jagrat T Patel
- Enhancing Code Comment Classification Using Language Models 679-688
  Jaivin Barot
- Assessing the Utility of C Comments with SVM and Naïve Bayes Classifier 689-695
  Aritra Mitra
- Source Code Comment Classification using Naive Bayes and Support Vector Machine 696-703
  Raj Jitendra Shah
- On the Impact of Synthetic Data on Code Comment Usefulness Prediction 704-710
  Vishesh Agarwal
- Exploring LLM-based Data Augmentation Techniques for Code Comment Quality Classification 711-717
  Priyam Dalmia
- Exploring Large Language Models for Code Explanation 718-723
  Paheli Bhattacharya, Manojit Chakraborty, Kartheek Palepu, Vikas Pandey, Ishan Dindorkar, Rakesh Rajpurohit, Rishabh Gupta

### Indian Language Summarization (ILSUM)

- Key Takeaways from the Second Shared Task on Indian Language Summarization (ILSUM 2023) 724-733
  Shrey Satapara, Parth Mehta, Sandip Modha, Debasis Ganguly
- Prompted Zero-Shot Multi-label Classification of Factual Incorrectness in Machine-Generated Summaries 734-746
  Aniket Deroy, Subhankar Maity, Saptarshi Ghosh
- Advancing Human-Like Summarization: Approaches to Text Summarization 747-754
  Saliq Gowhar, Bhavya Sharma, Ashutosh K Gupta, Anand Kumar Madasamy
- Named Entity-Aware Abstractive Text Summarization for Hindi Language 755-765
  Saumay Gupta, Sukomal Pal
- Text Summarization for Indian Languages: Finetuned Transformer Model Application 766-774
  V Ilanchezhiyan, R Darshan, E M Milin Dhitshithaa, B Bharathi

### Machine Translation for Indian Languages (MTIL)

- Overview of MTIL Track at FIRE 2023: Machine Translation for Indian Languages 775-782
  Surupendu Gangopadhyay
- Hindi-Odia Machine Translation System 783-789
  Rakesh Chandra Balabantaray
- Fine tuning based Domain Adaptation for Machine Translation of Low Resource Indic Languages 790-795
  Amulya Ratna Dash
- Bidirectional Hindi-Punjabi Machine Translation 796-801
  Mukund K Roy

2024-03-06: submitted by Kripabandhu Ghosh, metadata incl. bibliographic data published under Creative Commons CC0
2024-05-10: published on CEUR-WS.org |valid HTML5|