Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets (arXiv)

There is still much to learn about how the hatred target’s traits interact with the annotator’s. To close this gap, we make use of a large dataset that includes comprehensive socio-demographic data on both annotators and targets. This allows us to see how human biases relate to the characteristics of the target. Widespread biases are exposed by our investigation, which we objectively classify and explain according to their frequency and strength, displaying notable variations. Additionally, we contrast persona-based LLM biases with human biases. Our results show that persona-based LLMs do have biases, but they are very different from those of human annotators.

https://arxiv.org/abs/2410.07991