Aug 2026· Proceedings of the 2026 ACM Symposium on Document Engineering· pp. 1-4· 0 citations· 10 references
TL;DR
A systematic evaluation of several proprietary and open-source Large Language Models for sensitive entity extraction from documents and a form-filling pipeline that uses vision-capable LLMs to label form fields, generate realistic synthetic personas, and fill real blank forms, enabling reproducible evaluation across diverse layouts.
Abstract
Detecting and extracting sensitive information from documents is essential for privacy and regulatory compliance. Existing approaches either require training on large labeled datasets or rely on brittle, costly-to-curate pattern matching, while Large Language Models (LLMs) offer a promising alternative. We present a systematic evaluation of several proprietary and open-source LLMs for sensitive entity extraction from documents. Because large datasets of completed forms containing personal information are unavailable, we also introduce a form-filling pipeline that uses vision-capable LLMs to label form fields, generate realistic synthetic personas, and fill real blank forms, enabling reproducible evaluation across diverse layouts. Evaluating on these forms and the public RVL-CDIP dataset, we find performance is uneven across entity types—and that LLMs fall short of a simple pattern-based baseline on Social Security numbers.
With the rapid growth of digital information, the risk of sensitive information leakage in textual data, including personally identifiable information, medical privacy, financial data, and corporate confidential information, has become increasingly prominent. Traditional sensitive information detection methods, which mainly rely on rule matching, supervised learning, and manual annotation, struggle to meet the requirements of identifying diverse, open-domain, and dynamically evolving sensitive information. In recent years, Large Language Models (LLMs) have provided a new technical paradigm for zero-shot sensitive information detection without annotated data, owing to their powerful semantic understanding, contextual reasoning, and knowledge transfer capabilities. Through prompt learning, in-context learning, and instruction-driven information extraction approaches, LLMs can achieve flexible sensitive information identification in scenarios involving unknown sensitive categories and cross-domain applications. However, LLMs themselves introduce new security risks, including training data leakage, privacy memorization, and prompt injection attacks, posing significant challenges to sensitive information detection technologies. This paper presents a systematic survey of the development of LLM-based zero-shot sensitive information detection techniques. First, it introduces the development path of sensitive data detection technology, including rule-based methods, older machine-learning techniques, and currently popular pre-trained language models. Then it introduces the research content of LLM-driven zero-shot named entity recognition, open information extraction and privacy detection. List the privacy-leakage risks and corresponding defence measures for current applications of LLMs. Finally, this paper presents some future research directions for the above work and provides a path for the development of efficient, secure and trustworthy intelligent sensitive information detection systems.
Unknown authors· Scientific Journal of Intell...· 0 citations
The massive amount of publicly available data has necessitated an increase in public and organizational awareness of the potential risks of leaking private data, whether intentionally or unintentionally. The damage caused by leaking these data depends on their degree of sensitivity. Disclosing a person’s or an organization’s private data via different social media platforms might threaten people’s lives or the organization’s reputation or finances. Handling big data, especially unstructured data, is challenging. Consequentially, many solutions have been proposed to detect sensitive data in structured containers. However, detecting sensitive data in unstructured containers is still challenging, especially with context-dependent and high-performance measurement results. In this study, experiments on certain machine learning models and two transformers—DistilRoberta and ALBERT—were conducted to detect unstructured, textual, context-dependent sensitive data. The results show that DistilRoberta demonstrated higher accuracy and recall, and was faster and lighter than ALBERT.
Hala Qawara, Hanan Alhindi· Information· 0 citations
This survey provides the first comprehensive and systematic review of text anonymization methods published between 2020 and 2025, covering 48 primary studies identified through a structured search and rigorous screening procedure and reveals a growing shift from identifier‐centric de‐identification toward context‐aware anonymization.
Marina Litvak, A. Jorge· WIREs Data Mining and Knowle...· 0 citations
ProxyDrift is presented, a framework that identifies and measures drift between production traffic and offline evaluation sets, and constructs and refreshes those evaluation sets accordingly; all without access to raw user data.
Michael Levit, Josh Ledgard, Haoyu Dong et al.· 0 citations
The core of VaryBalance is that, compared to LLM-generated texts, there is a greater difference between human texts and their rewritten version via LLMs, and quantifies this through Mean Squared Deviation and distinguishes human texts and LLM-generated texts.
Xuecong Li, Xiaohong Li, Qiang Hu et al.· 0 citations
Real estate property listings expose structured metadata through the API. Still, the richest property-level information (i.e., legal status, structural condition, utility supplies, heating systems) sits in attached questionnaire documents that no automated system currently processes at scale. These documents are heterogeneous. Some are digitally generated with selectable text, others are scanned physical forms. There are even more complex layouts that contain checkbox annotations that defeat conventional text extraction. In this paper, we present an end-to-end pipeline for acquiring, classifying, and extracting structured data from selectable text documents. The pipeline was applied to 3965 questionnaire documents collected from a live property platform via reverse-engineered REST APIs. First, we classified each document into one of three structural categories (text_only, scanned, and special_char), then extracted 35 predefined property attributes from eligible documents using DeepSeek R1 as the Large Language Model, prompted to return a structured JSON object. All 2781 submitted documents were processed successfully, producing a final dataset of 2766 unique property records. Downstream validation confirmed the data quality. Cosine similarity matching achieves a Jaccard consistency score of 0.82, and K-Means clustering produces interpretable market segments with a silhouette score of 0.2088. Results show that the proposed extraction from each property document is both feasible and reliable at this scale.
Muhammad Shehbaz, Carlos Francisco Moreno-García· 0 citations