Skip to content
#small language model Open access

Character-Level Visual Guidance for Arbitrarily Shaped Scene Text Detection and Recognition

Sep 2026 · Electronics · 0 citations · 15 references

TL;DR

Study of explicit character-level guidance for scene text detection and recognition indicates that character semantics for detection and local visual evidence for decoding are an effective way to improve robustness in difficult scene text.

Abstract

Curved, blurred, low-contrast, and cluttered scene text remains difficult to localize and transcribe because visual boundaries and character identities are often ambiguous. Most detectors refine spatial regions from visual and positional features, while many recognizers depend on implicit attention between global image tokens and language context. This separation weakens character-level evidence in both localization and text decoding. To address this issue, this paper studies explicit character-level guidance for scene text detection and recognition. For arbitrary-shaped text detection, the character-guided adaptive detector (CADet) combines a text-enhancement network (TENet), a character information adaptive guidance module (CIA), and a position and classification compensation module (COMP). Then, character semantics can participate in boundary-query construction before final detection. For scene text recognition (STR), serialized image embeddings for text recognition (SIETR) introduce local visual embeddings into autoregressive decoding via the character local image embedding module (CLIE) and permutation language modeling (PLM). On ArT, Total-Text, and CTW1500, CADet obtains F-measures of 79.5%, 89.4%, and 89.2%, respectively, outperforming representative Transformer-based detectors with only a small increase in computation. For recognition, SIETR reaches 95.6% sample-size-weighted average accuracy with 23.8 M parameters and 3.2 G FLOPs and improves most irregular-text benchmarks over PARSeq with fewer FLOPs. The results indicate that character semantics for detection and local visual evidence for decoding are an effective way to improve robustness in difficult scene text.

Read PDF

Similar papers

Preprint Sep 2026

MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting

Scene text spotting remains challenging for arbitrarily shaped text instances such as curved signs and dense multi-oriented characters in natural images, where tightly coupled architectures propagate localization errors directly into recognition failures. We present a two-stage pipeline that combines multi-encoder segm...

Aman Anand, Partha Pratim Roy, Shivakumara Palaiahnakote · 0 citations
Conference Open access Sep 2026

Auxiliary text-guided image restoration for image-text matching

A new ITM framework that improves the model's discriminative performance by focusing on localized core attributes and has a better robustness in handling highly similar hard negatives, which provides a new way to address cross-modal hard sample discrimination.

Ya-Ting Liu, Kuang-Rong Hao · 0 citations
#computer vision Preprint Sep 2026

On the Design Fundamentals of Pixel Text Representation Learning

This work investigates the fundamental design principles required for robust visual text representation learning and trains Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples.

Chaohao Yuan, Rui-Feng Yuan, Zhuo-Xu Huang et al. · 2 citations
Sep 2026

BPE-Level Visual-Textual Alignment for Multi-Scene Text Retrieval

Scene Text Retrieval (STR) aims to search images containing a given textual query within large-scale image collections. However, existing approaches are fundamentally constrained in two ways: 1) they are evaluated on narrow benchmarks that focus primarily on natural scenes; and 2) they rely on either error-prone multi-...

Tong-Kun Guan, Yu-Tong Cai, Hao-Cheng Wang et al. · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.