Skip to main content

Detection of Abusive Language over social media

We all use online platforms like facebook, twitter or youtube for sharing our views on a post, video or a person.  We do this by commenting on their post or by sharing information on their wall or page. Many of you might have noted that some person makes use of abusive language in the comment which may lead the person to depression. There are some instances in the past which led to severe problems. One of the cases was when Daughter of Robin William, Zelda, was bullied on Twitter and Instagram for posting a memoriam for her late father, which caused her to delete all her online accounts. There was one more significant highlight in 2013 when Facebook hosted the pages which were hateful against women.
Many types of research have been done in this field using different technologies like NLP, Machine Learning, Artificial Intelligence, Web Science etc.

What makes it a difficult task?

Some factors make it difficult to analyse like:
  1. One cannot use the simple word to spot Abusive language. Obfuscations such as ni9 9er etc., make it difficult to detect.
  2. There may also be a possibility that some insults that are offensive in one group may be acceptable by some other group.
  3. Some offensive sentence might be grammatically correct and fluent, which can be detected automatically but noisy sentences are difficult to process.
  4. Consider a sentence: "Chuck Hagel will shield Americans from the desert animals bickering. Let them kill each other, good riddance!", In this, the second sentence has more bitterness. And to decide whether the sentence is abusive or not we may need to account more than one sentence to classify.
  5. Apart from all these, the most problematic task is to separate sarcasm from abusive language as one might use the same tone of sarcastic comment as used by the person for offensive language.

Different approaches to detect Abusive language:

Some of the techniques that can be used:
  1. One of the fundamental approaches is to use offensive or Abusive keywords and Bag-of words for detecting the sentences. But the problem with these is that the number of false positive will be more.
  2. The accuracy of the previous technique can be increased by using n-gram models to train the model.
  3. The different feature can be extracted, like lexical, syntactic or semantic, from the sentences and use the word embedding model for classification of posts or sentences.
  4. One can also train different models like Naive Bayes classifier,  Support Vector Machine (SVM) etc. with pre-classified posts or tweets for detecting abusive sentences.
  5. Multi-dimensional analysis (MDA) can also be done to analyse different posts and sentences. But the problem with MDA is that it performs better with longer text. So, one can combine more than one tweets or posts to form larger text.
  6. One approach is to use convolutional neural network (CNN), model. Further, the person can use the character-level convolutional network, word-level convolutional network or a combination of both.

To perform Abusive Language Detection with better performance and accuracy, one can combine different techniques discussed as all of these methods have some positive points as well as some negative points for classification or identification of tweets or posts.

References:

Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We ...
4 comments
Read more

Word embeddings and an application in SMT

We all are aware of (not so) recent advancements in word representation, such as Word2Vec, GloVe etc. for various NLP tasks. Let's try to dig a little deeper of how they work, and why they are so helpful! The basics, what is a Word vector? We need a mathematical way of representing words so as to process them. We call this representation, a word vector. This representation can be as simple as a one-hot encoded vector having the size of the vocabulary.  For ex, if we had 3 words in our vocabulary {man, woman, child}, we can generate word vectors in the following manner Man : {0, 0, 1} Woman : {0, 1, 0} Child : {1, 0, 0} Such an encoding cannot be used to for any meaningful comparisons, other than checking for equality. In vectors such as Word2Vec, a word is represented as a distribution over some dimensions. Each word is assigned some particular weight for each of the dimensions. Picking up the previous example, this time the vectors can be as following (assuming a 2 dime...
1 comment
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE...
5 comments
Read more