Skip to main content

HEADLINE GENERATION

Natural Language Processing is gaining importance gradually in almost all walks of life ranging from industries like financial, health care to day to day activities like searching on search engines, chatting with automated chat bots and so on. Another major application of NLP is in text summarization which is still a hot topic and being worked upon vibrantly.
Source: http://bit.ly/2y0HFMt
Many techniques have been worked upon for summarization but to retrieve a summary of less than a sentence (heading probably) has not been much worked upon and is gaining more popularity.
There can be many applications of finding short summaries of texts of less than a sentence. Some of them are:
·     Generating Headlines automatically for newspaper stories.
·     Generating table of contents for a document.
Some of the techniques touched upon to achieve this are as follows:

1.  Context Free Grammars(CFGs):
        Generating headlines through CFG involves two steps:
·      Extracting sentence from the content-This involves various other steps like normalizing the text followed by feature extraction and sentence ranking.
·      Headline generation- After the sentences have been extracted based on their scores, content words from them are extracted which represent the entire text. Finally the headings are generated using rules of CFG.

2.    Recurrent Neural Networks(RNN): Neural Nets have been used in this field earlier as well but RNN's have increased the efficiency manifold. The RNN encoder-decoder technique is used with LSTM (Long short-term memory)elements. 

Source: http://bit.ly/2yycPeE

A huge dataset is required for RNNs so that they give a new correct output for an input which wasn’t present in the training set.
Attention is a technique which is used to understand which word is to be given more attention during headline formation.
Two types of attention techniques may be used: Simple Attention and Complex Attention. Although Simple Attention tends to give a better results in this.
3. Statistical models: Statistical methods involve two steps:-
·      Content selection-By this the model selects the content to be present in the summary. One of the basic models is “zero-level” model .In this model, the probability of a sentence to be in a summary is calculated by multiplying the probabilities of its individual terms(bag of words assumption).
·      Surface realization- The probability of deciding sentence ordering using models like bigram or trigram etc.
    4. Generative Model: Using HMMs for Story Generation from Headlines: 

 Source- http://www.cs.northwestern.edu/~akm175/docs/btp.pdf
HMM (Hidden Markov Model) is used in this technique. Other techniques used here are tagging, normalisation, segmentation, stemming stop word filtering and merging similar content words.
Source: http://www.cs.northwestern.edu/~akm175/docs/btp.pdf

Hence various techniques as mentioned above have been used to generate headlines out of a given text document. It has been observed that these techniques work fairly well for technical or historical documents but not much for poetic or artistic works as in such works , most of the times, the meaning of the context is much different from the words mentioned there.
Some of the most efficient techniques mentioned above are HMM model which around 20% of the times gives efficient headings other times it’s a little less matching with the context. RNNs also give good efficiency due to the techniques of simple attention used to give attention to candidate heading words.
Although all these techniques have been used but there is still much scope to make headline generation much better.

  
References:




Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We can use NLP to impr
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE
5 comments
Read more

Coreference Resolution and Applications in NLP

In computational linguistics and natural language processing coreference resolution (CR) is an avidly studies problem in discourse which has managed to be only partially solved by the state of the art and consequently remain one of the most exciting open problems in this field. Introduction and Definition The process of linking together mentions of a particular entity in a speech or text excerpt that related to real world entities is termed as coreference resolution. This process identifies the dependence between a phrase with the rest of the sentence or other sentences in the text.  This is an integral part of natural languages to avoid repetition, demonstrate possession/relation etc. A basic example to illustrate the above definition is given below : Another example which uses elements from popular fiction literature : Harry  wouldn’t bother to read “ Hogwarts: A History ” as long as  Hermione  is around.  He  knows  she  knows  the book  by heart. The different type
Post a Comment
Read more