Skip to content
Review Open access

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

Aug 2026 · ACM Computing Surveys · 0 citations · 204 references
Computer Science

TL;DR

This comprehensive review examines the dual role of LLMs in both facilitating and mitigating various information integrity challenges, including misinformation, disinformation, fake news, social bots, and privacy concerns, and demonstrates critical gaps in current approaches.

Abstract

Large Language Models (LLMs) have emerged as powerful tools that impact information integrity on social media platforms. This comprehensive review examines the dual role of LLMs in both facilitating and mitigating various information integrity challenges, including misinformation, disinformation, fake news, social bots, and privacy concerns. We conduct a comprehensive review of the literature from 2019 to 2024, screening 1048 studies and performing an in-depth analysis of 215 representative papers. This systematic approach allows us to identify key patterns in how LLMs influence the information security in social media ecosystems. Through a systematic analysis of papers from multiple databases, our findings reveal that while LLMs can enhance detection capabilities for malicious content and enable sophisticated defense mechanisms, they simultaneously pose risks by enabling the generation of highly convincing, deceptive content. We categorize and analyze the potential and challenges across different dimensions of information integrity, examining technical capabilities, ethical implications, and privacy concerns. The study demonstrates critical gaps in current approaches, particularly in cross-lingual detection, real-time monitoring, and privacy-preserving implementations. We conclude by proposing future research directions and recommendations for stakeholders to leverage LLMs while mitigating risks in social media information integrity.

Read PDF

Similar papers

Preprint Jul 2026

Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

A role-layer framework is introduced to unify LLM risks and defenses, and identifies three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense.

Lingwei Wei, Dou Hu, Wei Zhou et al. · 0 citations
Review Open access Jul 2026

Small Language Models for Phishing Website Detection: A Review of Cost, Performance, and Privacy Trade-Offs

This review investigates Goldenits et al.'s (2025) empirical comparison of fifteen open small language models (SLMs) for phishing website detection, which aimed to evaluate whether locally hosted models can achieve the same accuracy as proprietary large language models (LLMs), without the prohibitive costs or privacy concerns. The authors test each model with a stratified sample of 1,000 labelled websites from a pool of 10,395 websites and measure the accuracy, precision, recall and F1 score of the results. The best local model, llama3.3:70b, achieves an F1 score of 0.893 and recall of 0.948, which is close to, but still lower than, the F1 scores above 0.95 achieved by the largest proprietary systems (Goldenits et al. 2025). This review restates the three research questions posed in this paper, assesses the evidence provided for each of the questions and situates the evidence in the context of the existing literature on cost-aware deployment (Irugalbandara et al. 2024; Kavya & Sumathi, 2024) and LLM-based phishing detection (Koide et al. 2024). It concludes that, besides the number of parameters, the deployability of an SLM depends on its architecture and on the reliability of its output format, which does not depend only on its classification skill.

K. Mani · 0 citations
Review Open access Aug 2026

Recent Advances in Text Anonymization: A Systematic Review

This survey provides the first comprehensive and systematic review of text anonymization methods published between 2020 and 2025, covering 48 primary studies identified through a structured search and rigorous screening procedure and reveals a growing shift from identifier‐centric de‐identification toward context‐aware anonymization.

Marina Litvak, A. Jorge · 0 citations
Review 2025

The Effectiveness of Social Media Privacy Settings in Protecting Users' Privacy: A Comprehensive Study

This research paper provides a comprehensive examination of the effectiveness of privacy settings on social media platforms in safeguarding user information and privacy rights. Through an extensive literature review and analysis of current scholarly research, this paper investigates the multifaceted challenges surrounding social media privacy protection mechanisms, user behaviour, and the gap between technical capabilities and practical implementation. The study reveals that while privacy settings offer theoretical protection mechanisms, their effectiveness is severely limited by default permissive settings, poor user awareness, complex interface designs, inadequate privacy literacy, and persistent data collection practices by platforms and third parties. The paper analyses how regulatory frameworks, such as GDPR and CCPA, address these concerns while identifying emerging threats, including algorithmic manipulation, facial recognition technologies, deepfakes, and breaches of personal data. Furthermore, this paper examines the "privacy paradox" phenomenon, where users express strong concerns about privacy yet fail to adopt protective measures. Based on a comprehensive analysis of current research findings, the paper recommends a multi-stakeholder approach incorporating privacy-by-design principles, enhanced user education, regulatory enforcement, and platform accountability to improve the effectiveness of social media privacy protection mechanisms.

M. Tarun, Gyana R. Panda · 0 citations
Review Open access Aug 2026

Machine Learning for the Detection of Fake News on Social Media: A Critical Narrative Review of Methods, Evaluation Validity and Deployment Readiness

Automated detection of false and misleading news circulating on social media has become one of the most heavily researched applications of machine learning in the computational social sciences, yet the practical value of the resulting systems remains contested. Reported classification accuracies on standard benchmarks frequently approach ceiling, while independent assessments of the same model families under temporal, topical and adversarial shift record substantially weaker performance. This review examines the disjunction between benchmark success and operational capability, and asks what the accumulated evidence genuinely supports. The literature was identified through structured searching of openly accessible scholarly indexes and metadata registries, supplemented by backward and forward citation tracking from recent reviews and by targeted retrieval of methodological and institutional sources. Evidence was appraised for construct validity of labels, evaluation design, transparency, replication and external validity, and was synthesised thematically rather than catalogued study by study. Four findings emerge with reasonable confidence. Label provenance, rather than model architecture, is the dominant determinant of what a classifier learns, and source-level labelling propagates publisher-specific stylistic signals that inflate apparent accuracy. Social-context and propagation models achieve stronger discrimination than content-only models but forfeit the early-detection window and depend on platform data whose availability has narrowed. Multimodal and evidence-retrieval systems address genuine failure modes of text-only classification, although gains are reported on heterogeneous benchmarks that resist direct comparison. Large language models function simultaneously as a generative threat that decouples writing style from veracity and as a source of reasoning and rationale that improves small detectors, with the second role better evidenced than autonomous zero-shot verification. Evidence remains concentrated in English-language, politically framed, text-dominant corpora drawn from a small number of platforms, and almost no study measures downstream effects on audiences or on fact-checking workflows. Progress now depends less on architectural novelty than on label construction, temporally honest evaluation and outcome measurement beyond the confusion matrix.

Mary Oluwakemi Abioye · 0 citations
Open access 2026

Leveraging Large Language Models for Rumours Detection in Social Media

A robust and scalable framework for real-time rumor detection powered by Large Language Models, like BERT, RoBERTa, and GPTs-4, which combines Natural Language Processing techniques with sentiment analysis, stance detection, and automated fact-checking to enhance contextual understanding and assess credibility more effectively.

Priyanshi Borase, S. Kolhe · 0 citations