Using LLMs to Mimic the Conversational Dynamics of Reddit Communities
TL;DR
It is found that LLMs outperform expectations at replicating comment structure and formality, but struggle to accurately capture nuanced emotions, e.g. understating joy and overstating anger.
Abstract
Online communities face a constant battle against toxicity and misinformation. While human moderators struggle to keep pace with the volume of content, LLMs offer a promising solution for automatically generating constructive responses and shaping online interactions. This paper preliminarily investigates if LLMs can mimic the communication styles of Reddit users using their comment history as context. We evaluate two prompting approaches: predicting a target comment and filling in masked comments. We find that LLMs outperform expectations at replicating comment structure and formality, but struggle to accurately capture nuanced emotions, e.g. understating joy and overstating anger. These findings highlight a promising direction for LLMs in guiding online conversations towards prosociality influencing emergent communication patterns and norms within the community. The results of our study inspire future work with more rigorous methods of evaluation to explore the LLMs'effectiveness across diverse online communities to better understand their broader societal impact.
