Measuring Online Debaters’ Persuasive Skill from Text over Time

Abstract

Online debates allow people to express their persuasive abilities and provide exciting opportunities for understanding persuasion.
Prior studies have focused on studying persuasion based on current debate content, without accounting for each debater's history and exploring the progression of a debater's persuasive ability.
We study debater skill by modeling how participants progress over time in a collection of debates from Debate.org. We build on a widely-used model of skill in two-player games and augment it with linguistic features of a debater's content.
We show that online debaters' skill levels do tend to improve over time.
Incorporating linguistic profiles leads to more robust skill estimation than winning records alone. Notably, we find that an interaction feature combining uncertainty cues (hedging) with terms strongly associated with either side of a particular debate (fightin' words) is more predictive than either feature on its own, indicating the importance of fine-grained linguistic features.