Deep Learning based approach for Handwritten Gujarati Word Image Matching

Abstract

Information retrieval from scanned handwritten digital copies is a very challenging task especially in Indian scripts like Gujarati due to the presence of joint and conjuct characters as well as matras, cursive nature and varying size of the characters. There are two methods namely recognition-based and recognition-free for document image retrieval. OCR is one of the techniques from the recognitionbased approach and Word Matching is a technique from the recognition-free approach. OCR is a technique that converts scanned text into an editable format. Good OCR models are not available in most Indian Scripts. Word Matching is the task of locating specific words in a collection of document images. The difference in both approaches lies in the level of segmentation. There are two levels of segmentation namely Fine and Coarse Grain. In Fine Grain segmentation, the base character and the matras are considered as separate symbols and are two different units of segmentation. In Coarse Grain segmentation, the base character and matras are considered as a single unit of segmentation. Fine Grain segmentation is suitable for recognition type of works while Coarse Grain is suitable for word matching kind of work. Segmentation is the most crucial step in both approaches. The accuracy of the segmentation highly affects the result of information retrieval. The research here heads towards addressing these issues and improving the retrieval results using deep learning. In recent times, deep learning has been very effective in many domains. But it has not been used much in this domain. Moreover, we find very few works that use deep learning for the Gujarati script. In this thesis, we propose a Coarse Grain segmentation method using the object detection model Faster RCNN and a Fine Grain segmentation method using a combination of Connected Component Analysis and Faster RCNN. The annotation of the dataset for training these models has been carried out manually using LabelImg tool. For the retrieval of words from the dataset, an incremental matching model using Siamese Network is proposed.