Skip to main content

NLU analysis for question generation

How do we communicate? Not a minute passes without asking or answering any question. (In IIITD, not even a second. :P)

As machines have become part of our day today communications, they also need to ask us questions. The question generation uses the Natural language processing ideas as a backbone. This question generation has many applications in various fields such as the IVR systems, Tutoring systems, requirement elicitation activities before development of systems. It can be used to enhance the learning of students using dialogue-based systems, which help in deeper learning and understanding. Even used to generate question for our quizzes.

Techniques
1.       Syntax analysis or phase structure analysis
The most common approach would be to convert the complex sentences into simple sentences, identifying parts of speech and the entities in the sentence using syntactic parser. Depending on the relationship of the parts of speech and the entities, grammatical rules can be formed which help in clustering the sentences into different types of questions and then framing the questions.

2.       Semantic Analysis
Identifies each predicate in a sentence, its associated arguments and modifiers, and specifies their semantic roles. Semantic analysis gives better results as human sentences have words with different meaning and contexts.
Sentence: Delhi is in India.
Default question: Where is Delhi?
India is a country(has a semantic meaning). We can frame a different question.
In which country is Delhi?

3.       NLU Analysis
There is a very new way to do the same using the NLU analysis. The central idea behind the NLU analysis “what the sentence is communicating”. For every new sentence, a corresponding matching sentence pattern is identified from the template, as opposed to generating questions on every possible sentence constituent. The matched pattern determines the question that would be generated. The template generation is the most important task. This uses dependency parsing and semantic role labelling. The dependency parse provides a representation of the grammatical relations between individual words in a sentence. Also, the templates should be such to frame a question on the major theme of the sentence.

Pattern Distribution

Assigning Ranks for better questions
To generate better questions, the questions are given ranks depending on the popularity, usage and aptness of the question. This ranking is based on human inputs to the best possible question for a given sentence. For a given sentence, the generated questions are better as per human natural language instincts.

Ranking Question

Conclusion

The closer the algorithm gets to the natural language by considering meaning, context and natural ways of use, the better are the results.  















References:
http://aclweb.org/anthology/W16-6609
http://www.iro.umontreal.ca/~felipe/TALN2010/Xml/Papers/all/taln2010_submission_172.pdf

Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We can use NLP to impr
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE
5 comments
Read more

Semantic Similarity using Word Embeddings and Wordnet

Measuring semantic similarity between documents has varied applications in NLP and  Artificial sentences such as in chatbots, voicebots, communication in different languages etc. . It refers to quantifying similarity of sentences based on their literal meaning rather than only syntactic structure.  A semantic net such as WordNet and Word Embeddings such as Google’s Word2Vec, DocToVec can be used to compute semantic similarity. Let us see how. Word Embeddings Word embeddings are vector representations of words. A word embedding tries to map a word to a numerical vector representation using a dictionary of words, i.e. words and phrases from the vocabulary are mapped to the vector space and represented using real numbers. The closeness of vector representations of 2 words in the real space is a measure of similarity between them. Word embeddings can be broadly classified into frequency based (eg: count vector, tfidf, co occurrence etc) and prediction based (eg: Continuous bag
Post a Comment
Read more