Many observers and commentators were shocked by the violence that erupted at a white supremacist rally in Charlottesville, Virginia, last weekend. But “incidents of hate are actually all too common” in the U.S., according to the journalism nonprofit ProPublica, which has launched a tool to help Americans better understand that reality.
The Documenting Hate News Index, built in partnership with Google’s News Lab and the data visualization studio Pitch Interactive, collects news reports on hate incidents and makes them searchable by name, topic, and date.
The site is certainly eye-opening, and grim.
More than just a list, the site allows hate-related stories to be browsed by date, and shows fluctuations in overall reports of hate crimes over time. Astonishingly, while the violence in Charlottesville captured headlines, last weekend was not a peak for U.S. Hate crimes – a much broader wave crested in late May, when crimes included two fatalities in an anti-Muslim attack in Portland, a teacher ripping off a young student’s hijab, and the killing of a young black Army lieutenant by a white supremacist.
Get Data Sheet, Coins2Day’s technology newsletter.
According to Google News Lab’s announcement, the site uses machine learning – specifically, Google’s Natural Language API – to understand both the content of news reports about hate crimes, and subtler things like intent and sentiment. That means it can detect stories about events “suggestive of hate crime, bias, or abuse” and track the frequency of particular names, places, and more general keywords like “businessman” and “nationalists.”
Currently, “Donald Trump” is the highest-ranking keyword associated with incidents of hate.
The project is important because, according to ProPublica’s larger Documenting Hate project, there is no reliable national database of hate crimes. The Index is primarily intended to help journalists, researchers, and civil rights organizations get a broader view on the national situation.