S2TR: A Squeeze-Excited 2D Transformer Recognizer for Indic Scene Text Recognition

Abstract

Recognizing text in natural scene images taken from Indian road environments is a much more challenging task than recognizing Latin script images, as the recognition problem involves recognizing Indic written text, which are two-dimensional configurations of glyphs like Devanagari, Telugu and Malayalam. These scripts have modifiers on top and bottom of the vowel body, multiple consonant components stacked on top of each other and written vertically, and complex consonant shapes, which cannot be meaningfully described as a flat left-to-right character string. To overcome this complexity, most current scene text recognition systems integrate a visual encoder with an autoregressive language decoder that enhances accuracy for words but results in significant architectural overhead, longer inference times and a language model dependency that restricts scalability to the numerous Indian languages that lack large annotated corpora. This paper proposes a Single Visual Transformer for Text Recognition (SVTR), which focuses on placing all the power of the model in the visual encoder and employs a simple Connectionist Temporal Classification (CTC) head for text decoding. The encoder is based on three elements. First, a Residual Convolutional Backbone, equipped with Squeeze and Excitation (Squeeze-Excite) channel attention to learn to highlight the stroke channels relevant to the script and silence the background channels. Secondly, a patch tokenization module that transforms the feature map generated above into a sequence of embeddings that explicitly encode 2D positional information while maintaining the vertical structure of the components of Brahmic sub-glyphs. Third, to improve the representation of each token in the sequence by mixing with the global context, a multi-head self-attention transformer is applied before the last CTC projection. S2TR is evaluated on the WIRIndic benchmark with a small, purely visual, decoder-free architecture, without any synthetic pretraining and external language models, achieving word recognition rates of 73, 65 and 76 percent for Hindi, Telugu and Malayalam respectively, that is competitive with character-level performance on challenging Indic scene text.