Skip to main content

Empowering Lives with Artificial Intelligence

Can you imagine, a life without eyesight? Can you feel the complete blackness surrounding you? I hope not.



Computer Vision

Vision comes very naturally to us, but we do not understand the power of vision. Its not even much appreciated by the individuals having the gift of vision. We could see everything around us, and wonder about the sophisticated sense organ provided to us.


This is what a blind person sees.


This is what we see.

Even a visual description about this picture could be of a huge help to a person with visual impairment. That's what happens when we read a book, we build up visual elements in our brain, based on the description provided to us. "A beautiful lotus shaped architecture at night." can be a useful description for the picture above.

A real world example:

Automatically Captioned: a group of items on a table

Using this field of Artificial Intelligence, we can generate a textual description of the scene. A Convolutional Neural Network (CNN) can be used to do this task. It can achieve accuracy as high as 93.9% on ImageNet classification task.

source: https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html


Natural Language Processing

Now, we have a description about a scene, but what is the use of a textual description for a person with visual impairment? We need to provide this information to them in the form they can avail and find this information useful, i.e. by sound waves in the form of a language.

We can pass this information to a Natural Language Processing unit, to process this information and generate a speech for the user to understand. It is essential that the voice generated should not be a computer like sound, rather it should be of natural form, probably in native language of the user. This ensures that it doesn't sound too fabricated, it should be as good as a real person.

But one description of an image cannot define it all. How can we decide the, that the description contains the information relevant to what the user desires? An assistant along with this model can be of great help in this matter. Person can ask question from the assistant and the Computer Vision can find out more information about the attribute asked by the person, which again goes through the same process of text to speech and further more questions by the person.

Automatically captioned: a man standing in front of a mountain


Resources and References:


Labels:

Comments

Post a Comment

Popular posts from this blog

NLP in Video Games

From the last few decades, NLP (Natural Language Processing) has obtained a high level of success in the field  of Computer Science, Artificial Intelligence and Computational Logistics. NLP can also be used in video games, in fact, it is very interesting to use NLP in video games, as we can see games like Serious Games includes Communication aspects. In video games, the communication includes linguistic information that is passed either through spoken content or written content. Now the question is why and where can we use NLP in video games?  There are some games that are related to pedagogy or teaching (Serious Games). So, NLP can be used in these games to achieve these objectives in the real sense. In other games, one can use the speech control using NLP so that the player can play the game by concentrating only on visuals rather on I/O. These things at last increases the realism of the game. Hence, this is the reason for using NLP in games.  We ...
4 comments
Read more

Discourse Analysis

NLP makes machine to understand human language but we are facing issues like word ambiguity, sarcastic sentiments analysis and many more. One of the issue is to predict correctly relation between words like " Patrick went to the club on last Friday. He met Richard ." Here, ' He' refers to 'Patrick'. This kind of issue makes Discourse analysis one of the important applications of Natural Language Processing. What is Discourse Analysis ? The word discourse in linguistic terms means language in use. Discourse analysis may be defined as the process of performing text or language analysis, which involves text interpretation and knowing the social interactions. Discourse analysis may involve dealing with morphemes, n-grams, tenses, verbal aspects, page layouts, and so on. It is often used to refer to the analysis of conversations or verbal discourse. It is useful for performing tasks, like A naphora Resolution (AR) , Named Entity Recognition (NE...
5 comments
Read more

Cross Language Plagiarism Detection

Plagiarism: the practice of taking someone else work and presenting that as of their own. So its reverse Plagiarism detection is detecting weather a given piece of document is plagiarised or not. Detection of plagiarism was so naive in earlier times ,People used to just compare strings across 2 documents , then as the technology grew people also grew smarter , they started coming with ideas by which they can copy a document while go undetected from detection. In this post i will talk about a particular type of plagiarism detection which is essentially one of hardest in this domain i.e Cross-Language plagiarism detection(CLPD) . People take a piece of document and convert it into some other language and then post it, now our task is to detect weather it is copied from some other language or not ? In this post i will keep my limits to documents only not programming code, though there are many ways as well by which it can be detected that weather a piece of code in converted into some ...
4 comments
Read more