Recent progress in developing general purpose machine learning algorithms in playing the ancient game of Go and developing self-driving car shines a light on artificial intelligence (AI) and the benefit of achieving a workerless society. With such potential, more resources will be poured into the field and hasten the development of artificial general intelligence. Although the arrival date of an omnipotent AI agent might be decades away from now, the competing countries and sponsors might rush the process and obtain the first-mover advantage in controlling a superintelligent AI.
The contested development might pose a grave danger as Nick Bostrom points out in Superintelligence: Paths, Dangers, Strategies. The rushed solution might be one that would turn malignant after it has gained a decisive strategic advantage. Furthermore, dissimilar to the prediction made by many science fictions, the greatest threat of artificial intelligence might not be that it turns out to be malignant. Instead, an agent that follows the human order but doesn’t understand human moral value would take a short cut in ways that are unimaginable to us. In order to avoid superintelligence from causing pain and suffering to us, we need to develop superintelligence that adheres to our values.
The question of how to prevent existential crisis caused by superintelligence can thus be divided into three parts:
- What type of superintelligence should we focus on developing?
- How to control superintelligent agent when it is under development?
- How does a superintelligent agent acquire values that are aligned to our moral value?
Bostrom answers those three questions by analyzing many methods in solving them, and here I will summarize possible solutions to those questions.
Type of superintelligence: Bostrom proposes four possible paths to superintelligence, of which the most direct path is whole brain emulation (WBE). With whole brain emulation, we need not to consider the motivation and value acquisition since they will inherent our innate value and develop moral value in an understandable manner. Although some emulated minds might be malignant, we will be able to identify problems when we detect abnormalities in their thinking pattern. However, WBE is not the most efficient superintelligence and thus will bring about a second transition from emulation to artificial intelligence. With this in mind, we need to focus on solving AI control and value acquisition problem even if we concentrate our effort primarily on developing WBE. Other paths towards artificial intelligence include biological enhancement and brain-computer interaction. Biological enhancement would follow the pattern of evolution in nature, which takes a long time to change by a small fraction, and brain-computer interaction is likely going to be too invasive for ours liking.
Control problem: without control under developmental phase, we would be exposed to the chance of having an AI agent that doesn’t meet requisite moral and ethical value to create an existential crisis for the entire human race. Although the agent might have an altruistic final goal, it might want to achieve that goal by taking shortcuts. The creation of computronium, where agent turns all resources in the galaxy into computing devices, might be inevitable if it doesn’t realize our true intention. The control problem can be solved by capping its capacity or selecting its motivation. Those methods can be used together in creating oracle, genie or sovereign, in which it would not be able to gain a decisive strategic advantage over all human beings.
Value acquisition: we would not be satisfied with the controlled superintelligence since it would not realize its full potential. Thus, we need to ensure that the superintelligence follows our value. We, as human beings, have a complex understanding of the world around us. The understanding of the historical event, present event, and environment shapes our moral value. Since the values are different for two group of people and for a single group at different time periods, the task of transforming our value into computer understandable code might be infinitely hard. In order to solve this problem, Bostrom introduces the Coherent Extrapolated Volition (CEV) developed by Eliezer Yudkowsky, which I will discuss in greater detail in future articles. Coherent extrapolated volition, in brief, is a method for agents to find our intention and values: The computer needs to do what we meant but not what we asked.
There are some messages in the books that are valuable for those who are eager to create superintelligent agents.
First, superintelligence and artificial intelligence do not necessarily lead to disasters if we put enough emphasis on the control and value acquisition problem. Even if the superintelligence gains a decisive strategic advantage and forms a singleton, the correct value would prevent it from causing pain and suffering to human beings.
Second, some promising methods might not yield the intentioned outcome. If we don’t think hard enough about all possible outcomes, the superintelligent agent will then expose and exploit the underlying loopholes in our logics.
Superintelligence: Paths, Dangers, Strategies is a groundbreaking book that explores solutions to the control and value acquisition problems for the development of superintelligence. The book provides insights into many critical issues we might face in the future. The book is a suggested read for everyone who are curious about the paths we might take in the future and those who are interested in developing a superintelligence.