To truly target hate speech, moderation must extend beyond civility

Many Americans decry the decline of civility online and platforms typically prohibit profane speech. Tech critics say the emphasis on “civility” alone is dangerous and that such thinking helps fuel the white supremacist movement, particularly on social media. 

They’re right. 

Big Tech errs by treating content moderation as merely about content-matching. Polite speech diverts attention from the substance of what white supremacists say and redirects it to tone. When content moderation is too reliant on detecting profanity, it ignores how hate speech targets people who have been historically discriminated against. Content moderation overlooks the underlying purpose of hate speech — to punish, humiliate and control marginalized groups. 

Prioritizing civility online has not only allowed civil but hateful speech to thrive and it normalizes white supremacy. Most platforms analyze large bodies of speech with small quantities of hate rather than known samples of extremist speech — a technological limitation. But platforms don’t recognize that white supremacist speech, even when not directly used to harass, is hate speech — a policy problem. 

My team at the University of Michigan used machine learning to identify patterns in white supremacist speech that can be used to improve platforms’ detection and moderation systems. We set out to teach algorithms to distinguish white supremacist speech from general speech on social media. 

Our study, published by ADL (the Anti-Defamation League), reveals that white supremacists avoid using profane language to spread hate and weaponize civility against marginalized groups (especially Jews, immigrants and people of color). Automated moderation systems miss most white supremacist speech when they correlate hate with vulgar, toxic language. Instead, we analyzed how extremists differentiate and exclude racial, religious and sexual minorities.

White supremacists, for example, frequently center their whiteness by appending “white” to many terms (white children, white women, the white race). Keyword searches and automated detection don’t surface these linguistic patterns. By analyzing known samples of white supremacist speech specifically, we were able to detect such speech — sentiments such as “we should protect white children” or accusing others, especially Jews, of being “anti-white.”

Extremists are active on multiple social media platforms and quickly recreate their networks after being caught and banned. White supremacy, sociologist Jessie Daniels says, is “algorithmically amplified, sped up and circulated through networks to other White ethnonationalism movements around the world, ignored all the while by a tech industry that ‘doesn’t see race’ in the tools it creates.”

Our team developed computational tools to detect white supremacist speech across three platforms from 2016-2020. Despite its outsized harm, hate speech is a small proportion of the vast quantity of speech online. It’s difficult for machine learning systems to recognize hate speech based on large language models, systems trained on large samples of general online speech. We turned to a known source of explicit white supremacist speech: the far-right, white nationalist website Stormfront. We collected 275,000 posts from Stormfront and compared them to two other samples: tweets from users in a census of “alt-right” accounts and typical social media speech from Reddit’s r/all (a compendium of discussions on Reddit). We trained algorithms to study the sentence structure of posts, identify specific phrases and spot broad, recurring themes and topics.

White supremacists come across surprisingly polite across platforms and contexts. Along with adding “white” to many words, they often referred to racial or ethnic groups with plural nouns (Blacks, whites, Jews, gays). They also racialized Jews through their speech patterns, framing them as racially inferior and appropriate targets of violence and erasure. Their conversations about race and Jews overlapped, but their conversations about church, religion and Jews did not. 

White supremacists talked frequently about white decline, conspiracy theories about Jews and Jewish power and pro-Trump messaging. The specific topics they discussed changed, but these broader grievances did not. Automated detection systems should look for these themes rather than specific terms. 

White supremacist speech doesn’t always involve explicit attacks against others. On the contrary, white supremacists in our study were just as likely to use distinctive speech to signal their identity to others, to recruit and radicalize and to build in-group solidarity. Marking one’s speech as a white supremacist, for example, may be necessary for inclusion into these online spaces and extremist communities. 

Platforms claim content moderation at scale is too difficult and expensive, but our team detected white supremacist speech with affordable tools available to most researchers–much less expensive than those available to platforms. By “affordable” we mean the laptops and central computing resources provided by our university and open source Python code that’s freely available. 

Once white supremacists enter online spaces — as with offline ones — they threaten the safety of already marginalized groups and their ability to participate in public life. Content moderation should focus on proportionality: the impact it has on people already structurally disadvantaged, compounding the harm. Treating all offensive language as equal ignores the inequalities under girding American society.

Ultimately, research shows that social media platforms would do well to focus less on politeness and more on justice and equity. Civility be damned.

Libby Hemphill is an associate professor at the University of Michigan’s School of Information and the Institute for Social Research.