Skip to content
#generative ai Open access

E089 Quantified-Chunk New-Post Cohort: Pre-registration with In-Flight Amendments (v1.2). Primary analysis re-scoped to P3; primary contribution a query-validity and namespace-drift taxonomy.

Sep 2026 · Zenodo (CERN European Organization for Nuclear Research)

Abstract

E089 is a pre-registered field study testing how generative AI search engines cite cold, newly published content. The study used 12 posts arranged as six topic-matched A/B pairs. One page in each pair opened with a quantified-chunk template: a front-loaded definition, named entities and at least one hard number already deposited in a public source. The other used the standard template. I checked citation and chunk reconstruction at D14 and D28 across Google Search, AI Overviews, AI Mode and Perplexity Sonar. This version finishes the analysis re-scope that began in amendment v1.1 and changes what the study can reasonably claim. V1.1 had already narrowed the confirmatory analysis from all six pairs to the two coined-term pillar pairs, P3 and P4. The informational half of the cohort failed query validity because the target queries hadn't been screened for search demand or namespace ownership, and every pair was compromised in a different way. P4 later collapsed too. Both arms were namespace-invalid, although for different reasons. That left P3 as the only primary pair. P3 produced a measurable negative result on the registered dependent variable: reconstruction of the engineered opening chunk. None of the four answer surfaces reproduced the treatment page's front-loaded statistic. They did, however, reproduce the plain declarative definition almost word for word on both arms. So the number being placed at the front wasn't what got the content selected. That result comes from one pair at one timepoint, with no D14 baseline and a sample below the minimum-N standard. I'm therefore reporting it as a directional negative, not as a confirmatory finding about the template. The study's primary contribution is now a taxonomy of query-validity failure and namespace drift. Five of the six pairs were compromised through different mechanisms: foreign-entity namespace capture, product-brand ownership, dense-corpus collision, authority saturation, topic-invalid queries and within-pair namespace asymmetry. This extends my earlier work on namespace drift. Two incidental extraction positives remain in the record. Two cold control pages, published without amplification, earned citations in AI Overviews and Perplexity with their engineered openings reproduced. One was still being cited in an off-protocol observation roughly three weeks after its last confirmed read. Nothing about the experiment itself has changed. Treatment assignments, the randomisation seed, publishing schedule, hygiene baseline, measurement offsets and coding rules remain as registered in v1.1. This version changes only the analysis and interpretation plan. Deposited under CC BY 4.0.

View source

Similar papers

#artificial intelligence Conference Open access Apr 2020

ECCOLA - a Method for Implementing Ethically Aligned AI Systems

The method, ECCOLA, is presented, which aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.

Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson · 64 citations · ⚡6
#computer vision Review Apr 2024

AI-powered Code Review with LLMs: Early Results

The goal is to not only refine the accuracy of the LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.

Z. Rasheed, Malik Abdul Sami, Muhammad Waseem et al. · 62 citations · ⚡3
#computer vision Open access Mar 2024

LLM-based agents for automating the enhancement of user story quality: An early report

The use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams is explored, with a reference model for an Autonomous LLM-based Agent System developed and implemented at the company.

Zheying Zhang, M. Rayhan, Tomas Herda et al. · 48 citations · ⚡4
#computer vision Review Mar 2024

System for systematic literature review using multiple AI agents: Concept and an empirical evaluation

This paper introduces a novel multi-AI-agent system designed to fully automate SLRs, and demonstrates how it substantially reduces the time and effort traditionally required for SLRs while maintaining comprehensiveness and precision.

Abdul Malik Sami, Z. Rasheed, Kai-Kristian Kemell et al. · 44 citations · ⚡2
#computer vision Feb 2024

Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis

The proposed LLM-based multi-agent system automates qualitative data analysis process, creating opportunities for researchers and practitioners, and future improvements focus on enhancing multilingual performance and integrating continuous expert feedback.

Z. Rasheed, Muhammad Waseem, Aakash Ahmad et al. · 41 citations

Related blog posts

Microsoft Research Blog Oct 7, 2026

Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses

Training AI agents with reinforcement learning can be challenging because their tools, context, and decision-making are managed by complex frameworks. Agent Lightning connects existing agents to RL training, making it easier to improve them without rebuilding them. The post Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.