Skip to main content

NLU analysis for question generation

How do we communicate? Not a minute passes without asking or answering any question. (In IIITD, not even a second. :P)

As machines have become part of our day today communications, they also need to ask us questions. The question generation uses the Natural language processing ideas as a backbone. This question generation has many applications in various fields such as the IVR systems, Tutoring systems, requirement elicitation activities before development of systems. It can be used to enhance the learning of students using dialogue-based systems, which help in deeper learning and understanding. Even used to generate question for our quizzes.

Techniques
1.       Syntax analysis or phase structure analysis
The most common approach would be to convert the complex sentences into simple sentences, identifying parts of speech and the entities in the sentence using syntactic parser. Depending on the relationship of the parts of speech and the entities, grammatical rules can be formed which help in clustering the sentences into different types of questions and then framing the questions.

2.       Semantic Analysis
Identifies each predicate in a sentence, its associated arguments and modifiers, and specifies their semantic roles. Semantic analysis gives better results as human sentences have words with different meaning and contexts.
Sentence: Delhi is in India.
Default question: Where is Delhi?
India is a country(has a semantic meaning). We can frame a different question.
In which country is Delhi?

3.       NLU Analysis
There is a very new way to do the same using the NLU analysis. The central idea behind the NLU analysis “what the sentence is communicating”. For every new sentence, a corresponding matching sentence pattern is identified from the template, as opposed to generating questions on every possible sentence constituent. The matched pattern determines the question that would be generated. The template generation is the most important task. This uses dependency parsing and semantic role labelling. The dependency parse provides a representation of the grammatical relations between individual words in a sentence. Also, the templates should be such to frame a question on the major theme of the sentence.

Pattern Distribution

Assigning Ranks for better questions
To generate better questions, the questions are given ranks depending on the popularity, usage and aptness of the question. This ranking is based on human inputs to the best possible question for a given sentence. For a given sentence, the generated questions are better as per human natural language instincts.

Ranking Question

Conclusion

The closer the algorithm gets to the natural language by considering meaning, context and natural ways of use, the better are the results.  















References:
http://aclweb.org/anthology/W16-6609
http://www.iro.umontreal.ca/~felipe/TALN2010/Xml/Papers/all/taln2010_submission_172.pdf

Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We ...
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE...
5 comments
Read more

Dbpedia Datasets

WHAT IS Dbpedia? It is a project idea aiming to extract structured content from the information created in the wikipedia project. This structured information is made available on the World Wide Web. DBpedia allows users to semantically query relationships and properties of Wikipedia resources, including links to other related datsets. BUT? But why i am talking about Dbpedia ? How it is related to natural language processing? The DBpedia data set contains 4.58 million entities, out of which 4.22 million are classified in a consistent ontology, including 1,445,000 persons, 735,000 places, 123,000 music albums, 87,000 films, 19,000 video games, 241,000 organizations, 251,000 species and 6,000 diseases. The data set features labels and abstracts for these entities in up to 125 languages; 25.2 million links to images and 29.8 million links to external web pages. In addition, it contains around 50 million links...
4 comments
Read more