Automating Online Hate Speech Detection: A Survey of Deep Learning Approaches

Motivation

Hateful content on social media

Contributes to real-world violence
Recruitment to and propaganda for terrorist individuals/groups
Makes other users feel less safe and secure on social platforms
Triggers increased levels of toxicity in the network

Contributions

Survey of deep learning model architectures, embedding choices, and feature inputs
Experiment with user behavior metrics in a multiple input model architectures
We find that Google’s pretrained Bert embeddings provide enough semantic meaning. User behavior metrics do not improve upon Bert

Research Question:

How can we improve the performance of automated systems on identifying hate speech when they must learn from very few hateful samples?

Methodology

Our dataset:

64,149 tweets total
4% hateful
20% abusive
62% normal
14% spam

Dataset Analytics

Source: 80k annotated tweets

Embedding choices

TF-IDF
Pretrained Twitter
Pretrained Bert

Architectures

Logistic Regression baseline
Multilayer Perceptron
CNN
LSTM
DenseNet

Experiment Design

Phase 1: Tweet Embeddings
Phase 2: Tweet embeddings + Reply-pairing embeddings + reply network metrics as embedding coefficients (favorite count & retweet count)
Phase 3: Tweet embeddings + Dominant LDA Topic words from user timeline tweets

To run

Use Tweepy API to collect data or email me for 64k tweets dataset + embeddings
Install dependencies
Configure cloud computing environment
Sample run: python deep_learning_experiments.py --num_epochs 100 --model CNN --name test --seed 28 --embedding_key twitter --embedding_level word --experiment_flag 2

Full paper: Coming Soon

Contact: ashe.magalhaes@gmail.com

Copyright 2019 Ashe Magalhaes

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.