In a 2015 TED Talk, philosopher and Founding Director of the Oxford Future of Humanity Institute Nick Bostrom discusses the prospect of machine superintelligence: AI that would supersede human-level general intelligence. He begins by noting that with the advent of machine-learning models, we have shifted into a new paradigm of algorithms that learn—often from raw data, similar to the human infant (Bostrom, “What Happens” 3:26 – 3:49).
We are, of course, still in the era of narrow AI: the human brain possesses many capabilities beyond those of the most powerful AI. However, Bostrom notes that artificial general intelligence (AGI)—AI that can perform any intellectual task a human can—has been projected by many experts to arrive around mid- to late-century (Müller and Bostrom, 1) and that the period in between the development of AGI and whatever comes next may not be long at all.
Of course, Bostrom notes, the ultimate limits to information processing in the machine substrate lie far outside the limits of biological tissue due to factors such as size and speed difference (“What Happens” 5:05 – 5:43). So, Bostrom says, the potential for superintelligence lies dormant for now, but in this century, scientists may unlock a new path forward in AI. We might then see an intelligence explosion constituting a new shift in the knowledge substrate, and resulting in superintelligence (6:00 – 6:09).
What we should worry about, Bostrom explains, are the consequences (which reach as far as existential risk) of creating an immensely powerful intelligence guided wholly by processes of optimization. Bostrom imagines that a superintelligent AI tasked with, for example, solving a highly complex mathematical problem, might view human morals as threats to a strictly mathematical approach. In this scenario, our future would be shaped by the preferences of the AI, for better or for worse (Bostrom, “What Happens” 10:02 – 10:28).
For Bostrom, then, the answer is to figure out how to create AI that uses its intelligence to learn what we value and is motivated to perform actions that it would predict we will approve of. We would thus leverage this intelligence as much as possible to solve the control problem: “the initial conditions for the intelligence explosion might need to be set up in just the right way, if we are to have a controlled detonation,” Bostrom says (“What Happens” 14:33 – 14:41).
Thinking too far ahead?
Many academics think that concerns about superintelligence are too indefinite and too far in the future to merit much discussion. These thinkers usually also argue that our energies are better spent focused on short-term AI concerns, given that AI is already reshaping our lives in profound and not always positive ways. In a 2015 article, Oxford Internet Institute professor Luciano Floridi called discussions about a possible intelligence explosion “irresponsibly distracting,” arguing that we need to take care of the “serious and pressing problems” of present-day digital technologies (“Singularitarians” 9-10).
Beneficence versus non-maleficence
In conversations about how we can design AI systems that will better serve the interests of humanity and promote the common good, a distinction is often made between the negative principle (“do no harm”) and the positive principle (“do good”). Put another way, approaches toward developing principled AI can be either about ensuring that those systems are beneficent or ensuring they are non-maleficent. In the news, as one article points out, the two mindsets can mean the difference between headlines like “Using AI to eliminate bias from hiring” and “AI-assisted hiring is biased. Here’s how to make it more fair” (Bodnari, 2020).
Thinkers, like Bostrom, concerned with long-term AI worries such as superintelligence tend to structure their arguments more around the negative principle of non-maleficence. Though Bostrom does present a “common good principle” (312) in his 2014 book, Superintelligence: Paths, Dangers, Strategies, suggestions like this one are made more alongside the broader consideration that we need to be very careful with AI development in order to avoid the wide-ranging harm possible with general machine intelligence.
In an article from last year, Floridi once again accuses those concerned with superintelligence of alarmism and irresponsibility, arguing that their worries mislead public opinion to be fearful of AI progress rather than knowledgeable about the potential and much-needed solutions AI could bring about. Echoing the beneficence principle, Floridi writes, “we need all the good technology that we can design, develop, and deploy to cope with these challenges, and all human intelligence we can exercise to put this technology in the service of a better future” (“New Winter” 2).
In his afterword, Bostrom echoes the non-maleficence principle when he writes, “I just happen to think that, at this point in history, whereas we might get by with a vague sense that there are (astronomically) great things to hope for if the machine intelligence transition goes well, it seems more urgent that we develop a precise detailed understanding of what specific things could go wrong—so that we can make sure to avoid them” (Superintelligence 324).
Considerations regarding the two principles within the field of bioethics (where they originated), can be transferred to conversations about AI. In taking the beneficence approach (do good = help the patient), one worry in the medical community is that doctors risk negatively interfering in their patients’ lives or overstepping boundaries such as privacy. Similarly, with the superintelligence debate, perhaps the short-term, “do good now” camp risks sidelining, for example, preventative AI safety mechanisms in the pursuit of other more pressing beneficent outcomes such as problem-solving or human rights compliance.
There are many other complications involved. If we take the beneficence approach, the loaded questions of “whose common good?” and of who is making the decisions are paramount. On the other hand, taking an approach that centers doing good arguably also centers humanity and compassion, whereas non-maleficence may lead to more mathematical or impersonal calculations of how best to avoid specific risks or outcomes.
Bridging the gap
The different perspectives around hopes for AI and possible connections between them are outlined in a 2019 paper by Stephen Cave and Seán S. ÓhÉigeartaigh from the Leverhulme Centre for the Future of Intelligence at the University of Cambridge called “Bridging near- and long-term concerns about AI.”
The authors explain that researchers focused on the near-term prioritize immediate or imminent challenges such as privacy, accountability, algorithmic bias, and the safety of systems that are close to deployment. On the other hand, those working on the long-term examine concerns that are less certain, such as wide-scale job loss, superintelligence, and “fundamental questions about humanity’s place in a world with intelligent machines” (Cave and ÓhÉigeartaigh, 5).
Ultimately, Cave and ÓhÉigeartaigh argue that the disconnect between the two groups is a mistake, and that thinkers focused on one set of issues have good reasons to take seriously work done on the other.
The authors point to many possible benefits available to long-term research with insight from the present. For example, they write that immediate AI concerns will grow in importance as increasingly powerful systems are deployed. Technical safety research done now, they explain, could provide fundamental frameworks for future systems (5).
In considering what the long-term conversation has to offer us today, the authors write that “perhaps the most important point is that the medium to long term has a way of becoming the present. And it can do so unpredictably” (6). They emphasize that the impacts of both current and future AI systems might depend more on tipping points than even progressions, writing, “what the mainstream perceives to be distant-future speculation could therefore become reality sooner than expected” (6).
Regardless of the controversies over whether we should take the prospect of superintelligence seriously, support for investments in AI safety research unites many experts across the board. At the least, simply joining the conversation means asking one question which we might all agree is important: What does it mean to be human in a world increasingly shaped by the internet, digital technologies, algorithms, and machine-learning?
Bodnari, Andreea. “AI Ethics: First Do No Harm.” Towards Data Science, Sep 7, 2020, https://towardsdatascience.com/ai-ethics-first-do-no-harm-23fbff93017a
Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies, 2014, Oxford University Press.
Bostrom, Nick. “What happens when our computers get smarter than we are?” Ted, March 2015, video, https://www.ted.com/talks/nick_bostrom_what_happens_when_our_computers_get_smarter_than_we_are
Cave, Stephen and ÓhÉigeartaigh, Seán S. “Bridging near- and long-term concerns about AI,” Nature Machine Intelligence, vol. 1, 2019, pp. 5-6. https://www.nature.com/articles/s42256-018-0003-2
Floridi, Luciano. “AI and Its New Winter: from Myths to Realities,” Philosophy & Technology, vol. 33, 2020, pp. 1-3, SpringerLink. https://link.springer.com/article/10.1007/s13347-020-00396-6
Floridi, Luciano. “Singularitarians, AItheists, and Why the Problem with Artificial Intelligence Is H.A.L. (Humanity At Large), Not HAL.” APA Newsletter on Philosophy and Computers, vol. 14, no. 2, Spring 2015, pp. 8-10. https://www.academia.edu/15037984/
Müller, Vincent C. and Bostrom, Nick. ‘Future progress in artificial intelligence: A Survey of Expert Opinion.” Fundamental Issues of Artificial Intelligence. Synthese Library; Berlin: Springer, 2014, www.nickbostrom.com