Skip to main content

A Must Combo: NLP & Computer Vision & their Applications



INTRODUCTION

Natural Language processing has been used to solve many problems such as machine translation, sentiment analysis, parts of speech tagging and many more. On the other hand Image analysis is also shifting its gear up in the era of upcoming technology. Facebook’s face tagging, object recognition are some of the applications that come under image analysis. Both domains are a part of Artificial Intelligence but are much independent of each other.


Now what if we could combine the powers of these two to solve many more interesting problems. Let’s discuss some real life problems that can be solved by these two techniques for better understanding of their working extents:

APPLICATIONS
One of the most significant recent advances in health information systems has been the shift from paper to electronic documents. Now as we are aware that medical stuff involves much image components in form of x-rays scans, MRI scans, etc. accompanied by a short text description that usually describes those captured images. Here computer vision’s image acquisition & analysis could help better to capture all the necessary components that might be missed by a human and after capturing all these components they can be converted into human understandable language by NLP technique.
Next application is Automatic Image Caption Generation. Since social media is now a days bursting with lots of images, how good will it be if the system automatically suggests a great caption for your image as soon as you upload one. This application would require immense image analysis ability and rich NLP data set to express most suitable caption.
Another application where the duo can be used is in studying historical languages and symbols that ancient people have left behind usually on the walls of monuments and other rock structures. This would definitely require a very deep knowledge and excellently trained NLP system. The computer vision part would help to achieve the objective with much more ease in this case.

Another Important Application might be Prediction of Terrorist/Criminal Activity from socially posted images. Since everyone is now under the claw of social media, often criminals and other terrorist by mistake or intentionally leave clue or messages that might be used to get alert from upcoming disaster. Hence there is a need to make our systems intelligent by use of our combo of NLP and computer vision to predetermine some of those terrorist attacks.



REFERENCES
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2714909/
https://pdfs.semanticscholar.org/8ef9/75a35f4d96e9883929168f8aac71b625ee84.pdf
https://vnsgu.ac.in/dept/publication/vnsgujst41july2015/24.pdf
https://www.cs.washington.edu/mssi/2013/mooney-msr-uw-2013.pdf

Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We ...
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE...
5 comments
Read more

Word embeddings and an application in SMT

We all are aware of (not so) recent advancements in word representation, such as Word2Vec, GloVe etc. for various NLP tasks. Let's try to dig a little deeper of how they work, and why they are so helpful! The basics, what is a Word vector? We need a mathematical way of representing words so as to process them. We call this representation, a word vector. This representation can be as simple as a one-hot encoded vector having the size of the vocabulary.  For ex, if we had 3 words in our vocabulary {man, woman, child}, we can generate word vectors in the following manner Man : {0, 0, 1} Woman : {0, 1, 0} Child : {1, 0, 0} Such an encoding cannot be used to for any meaningful comparisons, other than checking for equality. In vectors such as Word2Vec, a word is represented as a distribution over some dimensions. Each word is assigned some particular weight for each of the dimensions. Picking up the previous example, this time the vectors can be as following (assuming a 2 dime...
1 comment
Read more