Skip to main content

Story Generation

We all grow up reading stories, some of which even form the basis of our etiquette. They help us enjoy and relieve stress. 
The automation of stories' generation has been a long researched problem. Major problem arises because neither the inputs nor the features of the output are defined clearly and thus evaluation of the output story becomes problematic. How do you say your algorithm performs well or not if you can't judge the output. How do you make an algorithm for generating the story when the result desired is not clearly defined?
Many techniques have been used to make such a system like planning approach where start state of characters, world, etc. and goals - character goals or author goals were given and story was generated by creating a plan to reach end state from start state to reach the goal, case based approach where a database had all the previous stories and new story was generated using preexisting ontology, recurrent neural network approach by training the model on stories of a similar kind and using the model to predict the characters, words and sentences that follow by assigning probabilities.

We focus on the open story generation which requires to tell a story given any domain without any retraining of the model. The latest technique used to generate story is via Markov Chain Monte Carlo (MCMC) and Deep Neural Networks. The problem of open story generation is broken down into two sub-problems:

  1. Generating a successor event.
  2. Translating events back into human readable natural language.
As distribution is unknown, we iterate over distributions and evaluate them and according to an acceptance criteria take the best. First, a bi-gram distribution is taken over the words(this serves as initial sampling distribution X) thus greatly limiting the number of stories from previously all possible stories. Now, to represent each sentence an event is defined of the form <subject, verb, object, modifier>. Sentences with more than one verb or subject are broken into multiple events. Now it is difficult to combine both the word distribution and the event representation because simple conditional distribution can't encode position information for a 4 tuple. 
Example: A noun can be subject or object depending on its position and thus next word can verb or sentence end depending on it. So while deciding the word we see both the location and the type of the previous word while deciding the next word. 
For the example given, if we know noun at position 0, we prefer verb at position 1.
Two acceptance criteria are taken which are multiplied to score the distribution:
  • Event Succession: Events sequence is important as certain events are more likely to follow a particular event than other events. A sequence to sequence neural network is created to learn model of event succession. This contains two neural networks- encoder and decoder. They combined give probabilities of event that succeed the given event. For the story this all is summed to generate a score for the story.
  • Long Range Event Relationships: The event succession model helps in capturing which sentence should come after which sentence but it misses the long distance relationship between various parts in the story. For this, k verbs are taken and the story is scored by looking into whether the verbs come in order. The k verbs can be any and are given according to human intuition. If the verbs are not in order, the score is 0 and hence multiplied score is also 0 else score is 1. In place of using human intuition, we learn the sequence of words using skipping Recurrent Neural networks on a corpus of stories to automate the process to get k most important events in the story. 
These when combined give a score for the story and the story with best score is taken into consideration. This is a good method to generate stories and performs well. The problem is still under research and possibly better solution will be obtained in the near future as the techniques we have are improving very quickly.

References


Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We can use NLP to impr
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE
5 comments
Read more

Semantic Similarity using Word Embeddings and Wordnet

Measuring semantic similarity between documents has varied applications in NLP and  Artificial sentences such as in chatbots, voicebots, communication in different languages etc. . It refers to quantifying similarity of sentences based on their literal meaning rather than only syntactic structure.  A semantic net such as WordNet and Word Embeddings such as Google’s Word2Vec, DocToVec can be used to compute semantic similarity. Let us see how. Word Embeddings Word embeddings are vector representations of words. A word embedding tries to map a word to a numerical vector representation using a dictionary of words, i.e. words and phrases from the vocabulary are mapped to the vector space and represented using real numbers. The closeness of vector representations of 2 words in the real space is a measure of similarity between them. Word embeddings can be broadly classified into frequency based (eg: count vector, tfidf, co occurrence etc) and prediction based (eg: Continuous bag
Post a Comment
Read more