| Vol-4054 urn:nbn:de:0074-4054-X Copyright © 2024 for the individual papers by the papers' authors. Copyright © 2024 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).

# FIRE 2024
FIRE 2024 Working Notes

### Working Notes of FIRE 2024 - Forum for Information Retrieval Evaluation

### Gandhinagar, India, December 12-15, 2024.

Edited by

### Kripabandhu Ghosh*
Thomas Mandl **
Prasenjit Majumder ***
Debasis Ganguly ****

* IISER Kolkata, Mohanpur, India
** University of Hildesheim, Hildesheim, Germany
*** Dhirubhai Ambani Institute of Information and Communication Technology, Gandhinagar, India
**** University of Glasgow, Glasgow, UK

## Table of Contents

- Preface
  Kripabandhu Ghosh, Thomas Mandl, Prasenjit Majumder, Debasis Ganguly

### Indian Language Summarization (ILSUM)

- Overview of the Third Shared Task on Indian Language Summarization (ILSUM 2024) 1-12
  Shrey Satapara, Parth Mehta, Sandip Modha, Asha Hegde, H. L. Shashirekha, Debasis Ganguly
- Text Summarization for Indian Languages: Extractive summarization using graph based technique 13-21
  Ansh Vora, Aastha Shah, Rinit Jain, Sheetal Sonawane
- CrossFactual: A Novel Approach for Detecting Factual Inaccuracies in Machine-Generated Summaries 22-31
  Aniket Deroy, Subhankar Maity
- Enhancing Accuracy in Indian Language Summarization 32-38
  Rishika Jha, Rakshit Ahuja, Sainik Kumar Mahata, Monalisa Dey, Dipankar Das
- Enhancing Extractive Summarization for Low Resource Indian Languages using TF-IDF and SVD 39-47
  Sangita Singh, Jyoti Prakash Singh, Akshay Deepak, Supriya
- Overcoming Code-Mixing and Script-Mixing in Indian Language Summarization with Transformer Models 48-57
  Pulkit Chatwal, Amit Agarwal, Ankush Mittal
- Indian Language Summarization and Factual Error Detection using Pretrained Sequence-to-Sequence Models and Large Language Models 58-68
  Abhijith Balan, Chittoor Karthik, Oswald Christopher
- Text Summarization using Pre-Trained Models on Tamil English Gujarati and Bengali 69-81
  Tanisha Sriram, Ananya Raman, Sowmya Anand, Durairaj Thenmozhi
- Abstractive Summarization of large articles in Hindi Language using IndicBART 82-87
  Chaitanya Subhedar, Sheetal Sonawane
- Text Summarization and Detection of Factual Incorrectness for Indian Languages 88-97
  Durairaj Thenmozhi, Rohan R, Niranjana A, Madussree Ravi, Padmashri R
- Summarizing English News Articles: Leveraging T5 and Google Gemini 1.0 Pro 98-106
  M Saipranav, Murari Sreekumar, Durairaj Thenmozhi, Shreyas Karthik, Rahul VS
- Enhancing News Article Summarization Through Post-Processing Techniques 107-115
  Fathima Firose A, Reshma Sheik, Kathijaafrose A, S. Jaya Nirmala
- Assessing Factual Accuracy in Machine Generated Cross Lingual Summaries using Logistic Regression and BERT 116-123
  Kushaal Shyam Potta, Jaswanth Sridharan, Mahadev Ramesh Ramya, Shriram Gopalakrishnan, Durairaj Thenmozhi

### Code-Mixed Information Retrieval

- Overview of the shared task on code-mixed information retrieval from social media data 124-128
  Supriya Chanda, Sukomal Pal
- RetrieveGPT: Merging Prompts and Mathematical Models for Enhanced Code-Mixed Information Retrieval 129-139
  Aniket Deroy, Subhankar Maity
- InfoTextCM: Addressing Code-Mixed Data Retrieval Challenges via Text Classification 140-147
  Aditya Sharma

### CoLI-Dravidian: Word-level Code-Mixed Language Identification in Dravidian Languages

- Overview of CoLI-Dravidian: Word-level Code-Mixed Language Identification in Dravidian Languages 148-157
  Asha Hegde, Fazlourrahman Balouchzahi, Sabur Butt, Sharal Coelho, Kavya G, Harshitha S Kumar, Sonith D, Shashirekha Hosahalli Lakshmaiah, Ameeta Agrawal
- Prompt Engineering Using GPT for Word-Level Code-Mixed Language Identification in Low-Resource Dravidian Languages 158-168
  Aniket Deroy, Subhankar Maity
- Finding the Right Language Model Goes a Long Way Towards Accurate Language Identification 169-181
  Rob van der Goot
- Cracking the Code: Machine Learning Approaches for Dravidian Code-Mixed Texts 182-190
  Nikhil Narayan, Sachin Mohanty
- Leveraging MuRIL for Word-Level Language Identification in Code-Mixed Dravidian Text 191-198
  Akhil Bhogal, Soumyadeep Sar, Dwaipayan Roy, Kripabandhu Ghosh
- Machine Learning Based Model for Word-level Language Identification in Code-mixed Kannada Texts 199-206
  Abdollah Abadian
- Word-Level Language Identification in Dravidian Code-Mixed Text Using Machine Learning: A Comparative Analysis of Models and Vectorization Techniques 207-215
  Ponsubash Raj R, Bharathi B
- Word Level Language Identification in Code-mixed Dravidian Languages 216-223
  Harshitha S Kumar, Sharal Coelho, Asha Hegde, Kavya G, Hosahalli Lakshmaiah Shashirekha
- Enhancing Word-Level Language Identification in Code-Mixed Dravidian Languages 224-232
  Sonith D, Kavya G, Asha Hegde, Hosahalli Lakshmaiah Shashirekha

### Sarcasm Identification of Dravidian Languages Tamil & Malayalam (DravidianCodeMix)

- Overview of Sarcasm Identification of Dravidian Languages in DravidianCodeMix@FIRE-2024 233-243
  Sripriya N, Bharathi B, Thenmozhi Durairaj, Nandhini K, Rahul Ponnusamy, Prasanna Kumar Kumaresan, Kishore Kumar Ponnusamy, Charmathi Rajkumar, Bharathi Raja Chakravarthi
- YouTube Comments Decoded: Leveraging LLMs for Low Resource Language Classification 244-254
  Aniket Deroy, Subhankar Maity
- Text Classification Based on Ensembles of Pre-Trained Models for Sarcasm Identification in Dravidian Languages 255-265
  Jaime Cerda-Flores, Daniel Castro-Pineda, Miguel G. Juarez, Rodrigo I Hernandez-Mazariegos, Jaime Cerda-Jacobo, Mario Graff, José Ortiz-Bejar
- Sarcasm Detection and Identification of Dravidian Language Using Machine Learning Approach 266-273
  Moogambigai A, Kamesh S, Paruvatha Priya B, Bharathi B
- A Hybrid Approach to Sarcasm Detection in Dravidian Code-Mixed Texts 274-278
  Prasun Maity, Dipanjan Saha, Sayan Das, Sainik Kumar Mahata
- Sarcasm Detection in Dravidian Languages Using Bi-directional LSTM 279-288
  Sulaksha B K, Shruthi Priyaa G K, Amritha P, C. Jerin Mahibha
- Unmasking Sarcasm: Exploring mBERT+CNN and LSTM Models for Sarcasm Identification in Code-Mixed Tamil and Malayalam Texts 289-298
  Sonith D, Kavya G, H L Shashirekha
- A Transformer-based Model for Detecting Multilingual Sarcasm in Social Media Posts 299-309
  Shraddha Chauhan, Abhinav Kumar
- Sarcasm Identification Of Dravidian Languages Malayalam and Tamil 310-317
  Shreyas Karthik, Murari Sreekumar, Kushaal Shyam Potta, Durairaj Thenmozhi
- Sarcasm Detection in Dravidian Languages 318-324
  Aioshi Chowdhury, Soumyajit Paul, Souvik Kundu, Anurag Kumar Thakur, Anuska Sarkar, Anusmita Ray Chaudhuri, Anirban Ray, Debasree Mitra
- MSD: Multilingual Sarcasm Detection using Deep Learning-Based Model 325-332
  Ranjeet Kumar, Abhinav Kumar
- Investigation of Machine Learning and Transformer Models for Sarcasm Detection in Dravidian Languages 333-339
  Malliga Subramanian, Ananthakumar S, Deepiga P, Dharshini S, Kogilavani S V
- Sarcasm Detection from Dravidian Language Text 340-347
  C.Jerin Mahibha, Gersome Shimi, Durairaj Thenmozhi
- Leveraging ChatGPT and XLM-RoBERTa for Sarcasm Detection in Dravidian Code-Mixed Languages 348-356
  Supriya Chanda, Krishna Tewari, Arjun Mukherjee, Sukomal Pal
- Automated Sarcasm Identification in Code-Mixed Social Media Text using Machine Learning Techniques 357-364
  Kogilavani S V, Palani Murugan V, Pooja Sree M, Pavul Chinnappan D
- Detecting Sarcasm in Social Media Text Using Indic Transliteration and Machine Learning Techniques 365-372
  Kogilavani S V, Malliga Subramnanian, Prenitha S P, Varshini S H
- Enhanced Sarcasm Detection in Code-Mixed Tamil-English Text Using GRU and LSTM with SMOTE and Padding Techniques 373-383
  Kogilavani S V, Navbila K, Sridhar S
- Leveraging Transfer Learning and Deep Recurrent Networks for Sarcasm Detection in Tamil Language Text 384-389
  Kogilavani S V, Subhadevi K, Sowbharanika Janani J S, Rahul K
- Sarcasm Unveiled: Advanced Detection Techniques for Tamil and Malayalam Using Multi modal Approaches 390-403
  Sumathi S, Jayaseelan S, Kevin Jeyaraj P
- BI-LSTM and Hybrid Model Based Approaches for Accurate Sarcasm Identification in Tamil and Malayalam Languages 404-410
  Kogilavani S V, Malliga Subramnanian, Sanjai R, Mohammed Sameer B, Motheeswaran K
- Leveraging Machine Learning and BERT for Sarcasm Detection in Text 411-417
  Kogilavani S V, Priyadharshini C, Varshini L, Sathyaa S
- Sarcasm Detection in Dravidian Languages Using Machine Learning and Transformer Models 418-422
  Malliga Subramnanian, Aruna A, Anbarasan T, Amudhavan M, Kogilavani S V

### Personalized Information Retrieval (PIR)

- Overview of the PIR Track at FIRE 2024: Evaluation of Personalised Information Retrieval 423-426
  Pranav Kasela, Marco Braga, Effrosyni Sokli, Gian Carlo Milanese, Georgios Peikos, Sandip Modha, Alessandro Raganato, Marco Viviani, Gabriella Pasi
- Fusing Retrieval Techniques for Enhanced Personalized Community Question Answering 427-431
  Vraj Patel, Palak Vanpariya, Kandarp Gajjar, Himani Trivedi

### CryptOQA: Opinion Extraction and Question Answering from CryptoCurrency-Related Tweets and Reddit posts

- Overview of CryptOQA: Opinion Extraction and Question Answering from CryptoCurrency-Related Tweets and Reddit posts 432-438
  Gourav Sen, Sougata Sarkar, Subham Kumar, Koustav Rudra, Kripabandhu Ghosh
- CryptoLLM: Unleashing the Power of Prompted LLMs for SmartQnA and Classification of Crypto Posts 439-447
  Aniket Deroy, Subhankar Maity
- Navigating Crypto Conversations: A Multi-Level Approach to Classify Cryptocurrency Opinions on Social Media 448-455
  Kavya G, Sonith D, H L Shashirekha
- CryptoOpinionMining: A Comparative Analysis of Hierarchical and Flat Classification Models for Social Media Content 456-463
  Rudra Roy, Pritam Pal, Rishika Jha, Dipankar Das

### Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages (HASOC)

- Overview of the HASOC Track at FIRE 2024: Hate-Speech Identification in English and Bengali 464-474
  Nishat Raihan, Koyel Ghosh, Sandip Modha, Shrey Satapara, Tanishka Gaur, Yaashu Dave, Marcos Zampieri, Sylvia Jaki, Thomas Mandl
- Hate Speech and Offensive Content Identification in English Language Based on BERT Model 475-481
  Jing Li, Xutao Yang
- Detecting Hate Speech and Offensive Content in English: A Model Using DistilBERT and A Model Based on FastText Strategies 482-492
  Kongqiang Wang, Xiaobing Zhou
- Detection of Hate Speech using Universal Sentence Encoding and Bidirectional Long Short-Term Memory Models 493-502
  Pedro Alonso, György Kovács, Rajkumar Saini, Marcus Liwicki
- HateGPT: Unleashing GPT-3.5 Turbo to Combat Hate Speech on X 503-510
  Aniket Deroy, Subhankar Maity
- Zero-Shot and Multitask Learning Synergy for Robust Hate Speech Detection across English and Bangla 511-521
  Kavya G, Asha Hegde, Sonith D, Subrahmanya, H L Shashirekha
- Detecting Hate Speech in Bangla: A Hybrid Model Using Machine Learning and Lexicon-Based Strategies 522-528
  Vinayak Vijay, Avishikta Bhattacharjee, Kirti Kumari, Upkar Kumar Kedia
- Detecting Hate Speech in Hinglish: A BiLSTM Neural Network Approach 529-534
  Kirti Kumari, Avishikta Bhattacharjee, Vinayak Vijay

### Spoken-Query Cross-Lingual Information Retrieval for the Indic Languages (SqCLIR)

- Overview of the FIRE 2024 SqCLIR Track: Spoken Query Cross-Lingual Information Retrieval for the Indic Languages 535-540
  Bhargav Dave, Prasenjit Majumder, Debasis Ganguly, Evangelos Kanoulas
- Spoken Query Retrieval in Monolingual Contexts with Whisper and SBERT Models 541-548
  Prabhu Ram Nagarajan, Livin Nector Dhasan, Meera Devi Thiagarajan

### Information Retrieval in Software Engineering (IRSE)

- LLMs for Code: Overview of the information retrieval in software engineering track at fire 2024 549-555
  Soumen Paul, Srijoni Majumdar, Raj Shah, Susmita Das, Madhusudan Ghosh, Debasis Ganguly, Gul Calikli, Debarshi Sanyal, Partha Pratim Das, Paul D. Clough, Ayan Bandhyopadhyay, Samiran Chattopadhyay
- Categorizing Code Comments based on its Relevance for Code Readability 556-561
  Om Joshi
- Can we predict Comment Quality using Machine Learning? 562-567
  Ashwin Gaikwad
- Code and Comment Classification in C using Machine Learning 568-573
  Sahil Walunj
- Code and Comments Categorization in terms of Program Comprehension 574-579
  Abhinav Daggubelli
- Assessing the Quality of Source Code Comments with Binary Classification Models 580-586
  Krishna K
- Improving Code Readability: A Machine Learning Approach to Classifying Valuable Source Code Comments 587-593
  Subhash Agrawal
- Source Code Comment Classification in C 594-599
  Devyani Remulkar
- Classification of Code and Comments in a Machine Learning Framework 600-605
  Adwita Deshpande
- Classifying useful and not useful code comments in C 606-610
  Atharva Keny
- Enhancing Code Annotation Reliability: Generative AI’s Role in Comment Quality Assessment Models 611-619
  Seetharam Killivalavan, Durairaj Thenmozhi
- Improving Binary Code Comment Quality Classification with Augmented Code-Comment Pairs Using Generative AI 620-629
  Rohith Arumugam, Angel Deborah
- Assessing the Utility of C Comments with SVM and Naïve Bayes Classifier 630-634
  Anamitra Mukhopadhyay
- Exploring Usefulness of C Comments with Silver-Standard Machine Learning Models 635-640
  Aritra Mitra
- Coding Smarter Not Harder: Leveraging GPT-3.5 Turbo for Automated Code Solution Scoring 641-646
  Aniket Deroy, Subhankar Maity

### Conversational System for Differential Diagnosis of GI Cancer

- Conversational System for Differential Diagnosis of GI Cancer: Track Overview 647-650
  Manjira Sinha, Rajat Pal, Tirthankar Dasgupta
- Cancer-Answer: Empowering Cancer Care with Advanced Large Language Models 651-659
  Aniket Deroy, Subhankar Maity
- A Retrieval-Augmented Generation (RAG) Pipeline for GI-Cancer Prediction and Classification Using Quantized Large Language Models 660-665
  Arti Jha, Nidhi Shah, Vikas Kumawat, Yashvardhan Sharma
- Gastrointestinal Cancer Related Question Answering Using BERT 666-671
  S ArunaDevi, J Abirami, B Bharathi

2025-10-02: submitted by Kripabandhu Ghosh, metadata incl. bibliographic data published under Creative Commons CC0
2025-10-05: published on CEUR-WS.org |valid HTML5|