Skip to content

REALIS: A Curated Dataset for Studying the Challenges of AI Image Detection

Sep 2026 · arXiv.org · Vol abs/2609.32734 · 0 citations · 100 references
Computer Science

TL;DR

REALIS provides a unified framework for measuring and improving the reliability of AI-image detectors under conditions that better reflect real-world use, and proposes a robustness protocol covering 35 transformations at five severity levels to analyze detector behavior under image processing.

Abstract

AI-generated image detectors are often evaluated on benchmarks where real and synthetic images differ in content, quality, or generation artifacts, allowing models to rely on dataset-specific cues and fail on unfamiliar generators or processed images. Existing datasets provide limited support for evaluating these challenges jointly across diverse visual content. We introduce REALIS, a dataset of 1.43 million real and synthetic images generated by 42 modern text-to-image models, including the latest proprietary systems such as Nano Banana 2. REALIS combines prompts derived from real images, quality filtering, and stratified sampling to reduce class-specific shortcuts while preserving content diversity. We further introduce REALIS-Expert, a stress-test subset for high-quality synthetic images, where real and generated samples are selected with closely matched semantic and visual characteristics. We also propose a robustness protocol covering 35 transformations at five severity levels to analyze detector behavior under image processing. Based on REALIS, our benchmark evaluates pretrained detectors, fine-tuned models, and zero-shot vision-language models under generator and post-processing shifts. On the hardest processed split, the best pretrained conventional detector achieves 0.550 ROC-AUC, compared with 0.752 for the best REALIS-trained detector. REALIS provides a unified framework for measuring and improving the reliability of AI-image detectors under conditions that better reflect real-world use.

View source

Similar papers

Preprint Sep 2026

Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents

AI-generated image detection has attracted increasing attention, but existing evaluations mainly focus on natural images, leaving AI-generated document images largely underexplored. This omission is concerning because documents often appear in sensitive real-world scenarios, such as invoices, expense reports, certifica...

Zhang-Jie Fu, Jia-Zhen Yan, Yuan-Wen Chen et al. · 0 citations
Sep 2026

Learning Continuous Source Responses For Generalizable AI-Generated Image Detection

Advances in image generation have made synthetic images increasingly difficult to distinguish from real photographs, raising concerns about the trustworthiness of visual media. Existing AI-generated image detectors often perform well on in-domain data, but their robustness and cross-generator generalization remain limi...

Man-Ni Cui, Rui-Qi Liu, Zi-Jian Yu et al. · 0 citations
#generative ai Open access Sep 2026

Transfer Learning-Based Detection of AI-Generated Image

This study investigates the automatic classification of real and AI-generated flower images using fine-tuned transfer learning models and shows that Swin Transformer-Tiny achieved the best overall performance, reaching an F1-score of 88.64% and outperforming the other architectures.

Mehtap Ülker · 0 citations
Sep 2026

A Generalizable and Explainable Framework for Synthetic Video Detection Using First-Digit Gradient Statistics

AI video generators have not only become harder to detect but are used to generate a diverse set of scenarios from landscapes to street views to animal videos. This creates a problem where CNN-based detectors are effective but offer no insight into their inner workings, while forensics-based detectors are often pretrai...

S. Shanu, Gautam Kumar, Tej Singh · 0 citations
Open access Sep 2026

Curating Synthetic Data for Task-Specific Visual Perception

Synthetic data are most valuable where general-purpose datasets cannot provide the domain-specific priors a task requires, and where manual annotation is expensive, imprecise, or infeasible. In this article we argue that the central question for specialized vision systems is not how to generate more data, but which dat...

Saptarshi Neil Sinha, P. Kühn, M. Weinmann · 0 citations
Preprint Aug 2026

Understanding Why Foundation Models Work for Diffusion-Generated Image Detection

This work investigates what cues are exploited by foundation-model-based detectors to distinguish real images from diffusion-generated ones and suggests that foundation-model-based detectors succeed by capturing non-semantic low-to-mid frequency distributional discrepancies between real and diffusion-generated images.

D. Cozzolino, G. Poggi, L. Verdoliva · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.