The hottest man-machine war will be fought again.

The man-machine war will be fought again, and Watson will push artificial intelligence to a new stage.

the aftermath of deep blue is not over, and Watson heat wave is rising again. In the environment of constant load and temperature, the supercomputer system once again duels with human beings. Who can win? Watson, the IBM supercomputer system, defeated the human champion in the man-machine battle of the just concluded American quiz show "danger edge"! Watson is the crystallization of more than 20 IBM researchers' four-year efforts. It is their breakthrough to give Watson the ability to understand natural language and accurately answer questions that has pushed AI to a new stage. Therefore, although Watson won the game, mankind is the ultimate wise man

20 years ago, everyone probably thought that it was impossible for machines to defeat humans in intellectual Q & A. Edward Feigenbaum, a Turing prize winner and AI expert at Stanford University, once made such an exclamation. However, today it has become a reality

jeopardy, a well-known TV quiz show in the United States In the morning of February 17, Beijing time, Watson, an IBM supercomputer system, defeated Ken and Brad, the two best human champions in the program's history, successfully ending the three-day man-machine war

compared with dark blue, the protagonist in the first battle of the man-machine war in 1997, Watson is another landmark supercomputer system. Compared with its predecessors, Watson's computing power is not comparable, and the greater difference lies in the algorithm challenges they deal with. In the first man-machine war, dark blue's proficiency in chess once shocked the world, because chess was clearly defined, mainly involving mathematical processing capabilities, and computers could easily calculate each game state and corresponding steps. However, the intelligence Q & A in the "danger edge" program requires computers to understand human language. Unlike chess, human language is completely open, often ambiguous, and requires context to understand its meaning. Although IBM researchers can easily understand human language, developing supercomputer systems that understand human language is extremely challenging

although a large number of encyclopedias and other information are stored, Watson will not easily find the answer to the question of "danger edge", because finding the answer has never been the strength of computers. Search engines can't answer questions, and can only give thousands of plausible possible answers that match the search keywords. Watson needs to obtain more evidence support for all candidate answers through various algorithms, and then give confidence to each candidate answer according to the strength of the evidence. Finally, it decides whether to provide users with the only answer with the highest confidence according to the confidence. This process is extremely complex, so it requires a supercomputer with thousands of processors to deal with a problem

Watson needs to master a lot of knowledge and find clues in relevant and irrelevant information. This is a huge challenge for computers. Because human beings can distinguish the connections between things in an instant, but computers must consider everything in parallel to draw conclusions

Watson won the man-machine war. This means that IBM has mastered the technical ability to respond more accurately to human information needs and problems, and foresees huge business opportunities in this field. This achievement will also be widely used in many fields, such as faster and more accurate medical diagnosis, research on potential drug interactions, help lawyers and judges find cases, realize scenario analysis and regulatory compliance in the financial field, help companies cultivate smarter salespeople, Watson's emergence subverts the previous simple human-computer relationship, and will bring a new era of human-computer cooperation

Watson must first understand the problem. A question may have multiple understandings, and Watson will look for possible answers in the stored information for different understandings; And this will get multiple answers. For each answer, Watson needs to study the corresponding evidence; Because the amount of evidence is huge, Watson needs to compare and exclude all answers according to the correlation strength of the evidence; Finally, the confidence level of their answers determines whether to provide answers to the outside world

the most remarkable thing is that Watson is a supercomputer system that can match the ability of human beings to answer questions. It has sufficient speed, accuracy and reliability, and can answer questions in natural language. The understanding of human natural language is also the core problem to be solved by supercomputer systems, especially how to use all kinds of unstructured and structured knowledge to help them understand natural language faster. This involves semantic analysis and processing, computer self-learning ability, large-scale parallel computing and other fields. IBM integrates these technologies into an architecture to help Watson meet the great challenges of natural language understanding

understanding natural language

Watson's emergence is inseparable from the progress in three fields: the progress of computer natural language, huge computing power, and massive digital global information

Watson's first breakthrough is his great success in answering questions in natural language for all areas of knowledge. Natural language is the language actually used by human beings, including puns, slang, jargon, abbreviations, and even words used in the wrong context. Computers are very good at computing, but natural language has the characteristics of fuzziness, high correlation with context, ambiguity, and even imprecision. In particular, the design of the "danger edge" program poses a greater challenge to IBM researchers. The title of this competition involves various fields of knowledge. It needs to analyze the subtle meanings, sarcasm, riddles, etc. in human language. These are usually the aspects that human beings are good at, but computers have no advantage over them. Watson's deepqa (Deep Open Domain Question Answering System) adopts breakthrough analysis technology, which can understand the content of the problem, analyze massive amounts of information, and then give the best answer according to the evidence it finds

answer questions accurately

Watson's second breakthrough is that it gives more accurate responses to information needs and problems through advanced analysis technology. In the competition of the edge of danger, there is a problem that a colorful plague appeared in the 14th century, which was later rewritten into a famous drama by Arthur Miller. The correct answer should be the death of a salesman

when Watson is asked a question, hundreds of algorithms will analyze the question in different ways, and give possible answers and evidence to choose them, and these analyses are carried out synchronously. For each candidate answer, Watson will find evidence for and against this answer. Therefore, each of these hundreds of answers will have hundreds of evidences, and then hundreds of algorithms will evaluate the degree to which these evidences support the answer. The better the result of evidence evaluation, the higher the confidence. The answer with the highest confidence will eventually become the answer selected by Watson. In the competition, if the answer with the highest confidence does not reach or exceed the threshold, it may decide not to rush to answer according to the situation, so as not to lose the bonus. All these calculations, choices and decisions must be completed within three seconds

analytical power of the Chinese team

it is worth mentioning that in the global team developing Watson, the team of IBM China Research Institute is also an important force. The research team of IBM headquarters in the United States mainly studies how to use unstructured knowledge sources to carry out statistics and analysis, and solve the problem of understanding human language. However, it is often difficult for researchers to grasp the accuracy and reliability of knowledge obtained from unstructured knowledge sources, while structured knowledge sources can provide a complementary help. One of the tasks of the Chinese team is to use structured knowledge as much as possible to help Watson answer questions and assess the reliability of answers more accurately

however, this type of exclusion and overlap is not absolute. For example, ask: which student of Professor Dumbledore defeated Voldemort? The type of question is student. This requires assessing the reliability of Harry Potter as a student type. This is easy for people, but for computers, it needs to match in all kinds of structured knowledge. For example, in the movie database, you will find that this is the movie name and the character name; In the novel database, you will find that this is the name of the novel; However, it is difficult to find a database listing the names of all students in the world. Therefore, in structured knowledge, the type of Harry Potter does not include students. In this difficult situation, on the one hand, the algorithm should avoid giving Watson the wrong signal. On the other hand, it should even tell Watson that structured knowledge does not rule out the possibility that Harry Potter is a student, because the role type and student type overlap

mysterious self-learning ability

for humans, the most important thing is the knowledge stored in the brain, while for computers, it is the background database. If Watson can have the ability of self-learning like human beings, and also acquire new knowledge through reading books, reading newspapers, browsing the Internet, then computers will become more intelligent

the research on computer self-learning ability is an important topic in the field of computer science and artificial intelligence. What is surprising is that Watson now has a certain learning ability. Fans of jeopardy created and maintained a large amount of data about the program, including questions and answers of all previous programs. How to make Watson learn and improve from the questions and answers of previous programs is a key in the research and development process

according to Pan Yue, every time Watson analyzes and answers the problem of increasing production in the past, it will produce a large amount of data. Sometimes, an experiment will produce hundreds of GB of data. The Chinese team has tested and studied various mathematical models and methods, and applied them to actual data. Among them, the Chinese team's two-stage learning method has proved to be very effective, and the global team has also expanded this method into complex multi-stage learning, which is applied to Watson project