Fact-Checking Comments Precede Community Notes to Misleading Posts on X

Abstract

Community-based fact-checking systems, such as X's Community Notes program, offer a potentially scalable approach against online misinformation diffusion. However, their efficacy is substantially undermined by the delay of fact-check delivery. To address this challenge, we investigated the promise of organic user comments as rapid corrective signals by analyzing a large-scale dataset of over 2.2 million comments directed to 1,841 community fact-checked misleading posts. We developed a high-performance language model pipeline to identify ``fact-checking (FC) comments'', i.e., comments that correct source posts with reasoning or evidence (91% accuracy). Using this pipeline, we find that 99.4% of misleading posts receive their first FC comments before the official community note is created. Notably, the median time to the initial FC comment is only 0.1 hours, while the creation of community notes has a median latency of 9.7 hours since the publication of misleading posts. Additionally, compared to random posts, those misleading posts with displayed notes have significantly more FC comments. This suggests that FC comments are a unique pattern for misleading posts. We further identified characteristics of FC comments' intensity and speed: (i) FC comments' volume is significantly associated with content richness and misinformation type, favoring multimedia content and missing-context scenarios; (ii) the speed of FC comments remains robust across diverse topics and emotional intensities. These results suggest that platforms could use FC comments as early-warning signals, and potential resource for complementing community notes or automated synthesis.