After my last blog about the IBM Watson Tone Analyzer, I have been reviewing some interesting stories about IBM Watson, the answering machine. It came to me that IBM Watson per se is, in fact, worth of another posting, especially with the fascinating Q/A system, which distinguishes it from general search engines, like Google. I want to share a very brief, if not a superficial analysis of the whole system.
Before all that, we can take a look at Watson’s real experience at Jeopardy! first.
Watson makes mistakes, for sure, but it has already been amazing that Watson can answer that many questions, particularly in the form of those that people usually ask every day. The Q/A system, or the question answering system is what makes Watson stand out (before Google included Q/A in some inquiries of course). The Q/A system is based on the conception that the system should be able to retrieve answers to questions formed in natural language when it is given a variety of documents (such as the World Wide Web or simply local documents). This is tempting as traditional search engines normally require you to transform natural language into short keywords queries for more relevant answers.
So obviously, using natural language is one of the greatest features that Watson has. As a matter of fact, it is called natural language processing, or NLP, which has question-type analysis and answer patterns, uses semantic processing, and syntactic processing and parsing. I think it is roughly fine to imagine a syntax tree here, which we saw when we learned the grammar a long time ago. NLP is harder than general search engines as the latter often use exact and explicit inputs to do better calculations while the former is more implicit and highly contextual with the focus on the real meaning of the input or questions.
Other than NLP, machine learning is also essential when Watson works, as it must learn to interact with human and get training from large data. The combination of NLP and machine learning during the answering process is, therefore, the key that makes Watson a history.
As Watson’s algorithm is still being developed, we can notice that it did make mistakes in the competition, but just envision how it can help mass research in the future to create unexpected values to our lives.