Report on the FIRE 2020 evaluation initiative

Abstract

This report gives an overview on the Forum for Information Retrieval Evaluation (FIRE) initiative for South-Asian languages1. The FIRE conference was conducted online in December 2020. The event combined a conference including keynotes, peer reviewed paper session with an Evaluation Forum. This report will present an overview of the conference and provide insights into the evaluation tracks. Current domains include legal information access, mixed script information retrieval, semantic analysis and social media posts classification. The tasks are discussed and connections to other evaluation initiatives are shown.