Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate (arXiv)

Though there are still issues with automated counter-narratives’ (CN) emotive tone, accessibility, and ethical implications, CN presents a potential approach to combating hate speech online. The researchers provide a paradigm for assessing CNs produced by Large Language Models (LLMs) in four areas: ethical soundness, emotive tone, verbosity and readability, and persona framing. On the MT-Conan and HatEval datasets, the investigators evaluate three prompting techniques using GPT-4o-Mini, Cohere’s CommandR-7B, and Meta’s LLaMA 3.1-70B. According to the research presented, LLM-generated CNs are frequently lengthy and tailored for college-level readers, which restricts their use. Although replies to emotionally led prompts are more sympathetic and readable, questions about their efficacy and safety still exist.

https://arxiv.org/abs/2506.04043