CONCORD: Enhancing COVID-19 Research with Weak-Supervision based Numerical Claim Extraction

Abstract

The COVID-19 Numerical Claims Open Research Dataset (CONCORD) is a comprehensive, open-source dataset that extracts numerical claims from academic papers on COVID-19 research. To extract numerical claims, a weak-supervision based model is employed, leveraging its white-box, explainable nature and advantages over transformer-based models in terms of computational and manual annotation costs. Labelling functions are used to programmatically generate labels, incorporating techniques like pattern matching, external knowledge bases, phrase matching, and third-party models. An aggregator function reconciles overlapping or contradictory labels. The weak-supervision model is evaluated against established baselines and transformer based models, achieving a weighted F1-score of 0.932 and micro F1-score of 0.930 in extracting numerical claims.While the weak-supervision model showcases superior performance compared to baseline models, it is observed that transformer-based models achieve comparable results.CONCORD, comprising around 200,000 numerical claims extracted from over 57,000 COVID-19 research articles, serves as a valuable tool for knowledge discovery and understanding the chronological developments in various research areas associated with COVID-19. In conclusion, CONCORD, alongside the weak-supervision methodology, offers researchers a valuable resource, enhancing advancements in COVID-19 research while highlighting the significant potential of weak-supervision models within the broader biomedical domain.