A CNN-based Handwritten Script to Digital Text Predictor for Real-Time Character Recognition and Text Digitization

Abstract

The task of Handwritten Text Recognition (HTR) still persists in computer vision and machine learning because there are numerous intra-class and inter-class differences in handwritten characters. In other words, differences in character shapes, stroke widths, styles, slants, spacings, and pen pressures result in great challenges related to automatic recognition. The inability to use HTR technology hinders the access to the information in different forms, e.g., in medical records, legal documents, educational tests, historic archive materials, etc. It is obvious that overcoming this challenge will facilitate more opportunities regarding information availability and automated processing of documents. The goal of this paper is to develop an end-to-end Handwritten Script to Digital Text Predictor (HSDTP) that will be able to convert images of handwritten characters to machine-readable text. In order to implement this goal, the CNN model trained on the MNIST database was developed and optimized for CPU-based hardware implementation without any need in special GPU. Thus, six successive steps should be implemented: image capture, pre-processing, character segmentation, character classification, reconstruction, and deployment of HSDTP as web service. Pre-processing stage involves such operations as grayscale transformation, noise filtering with Gaussian blur, binarization based on the Otsu method and normalization with resizing to 28 × 28 pixels. Character segmentation is done by contour detection; characters are recognized by CNN with prediction scores.