{{Short description|Artificial intelligence scenario}} An '''AI takeover''' is a theorized future event, often depicted in fiction, in which autonomous artificial intelligence systems acquire the capability to supersede human decisions. This could occur through economic manipulation, infrastructure control, or direct intervention, leading to de facto governance. Scenarios range from gradual economic dominance, as automation supplants the human workforce, up to a sudden or aggressive global takeover by a '''robot uprising''' or other forms of '''rogue AI'''.
Stories of AI takeovers have been popular throughout science fiction. Commentators argue that recent advancements in the field have heightened concern about such scenarios. In public debate, prominent figures such as Stephen Hawking have advocated research into precautionary measures to ensure future superintelligent machines remain under human control.<ref>{{Cite web |last=Lewis |first=Tanya |date=2015-01-12 |title=''Don't Let Artificial Intelligence Take Over, Top Scientists Warn'' |url=http://www.livescience.com/49419-artificial-intelligence-dangers-letter.html |access-date=October 20, 2015 |website=LiveScience |publisher=Purch |quote=Stephen Hawking and dozens of other top scientists and technology leaders have signed a letter warning of the potential dangers of developing artificial intelligence (AI). |archive-date=2018-03-08 |archive-url=https://web.archive.org/web/20180308100411/https://www.livescience.com/49419-artificial-intelligence-dangers-letter.html |url-status=live }}</ref>
== Types == === Automation of the economy === {{Main|Technological unemployment}}
The traditional consensus among economists has been that technological progress does not cause long-term unemployment. However, recent innovation in the fields of robotics and artificial intelligence has raised worries that human labor will become obsolete, leaving workers in some sectors without employment.<ref>{{Cite web |last=Lee |first=Kai-Fu |date=2017-06-24 |title=The Real Threat of Artificial Intelligence |url=https://www.nytimes.com/2017/06/24/opinion/sunday/artificial-intelligence-economic-inequality.html |access-date=2017-08-15 |website=The New York Times |quote=These tools can outperform human beings at a given task. This kind of A.I. is spreading to thousands of domains, and as it does, it will eliminate many jobs. |archive-date=2020-04-17 |archive-url=https://web.archive.org/web/20200417183307/https://www.nytimes.com/2017/06/24/opinion/sunday/artificial-intelligence-economic-inequality.html |url-status=live }}</ref><ref>{{Cite web |last=Larson |first=Nina |date=2017-06-08 |title=AI 'good for the world'... says ultra-lifelike robot |url=https://phys.org/news/2017-06-ai-good-world-ultra-lifelike-robot.html |access-date=2017-08-15 |website=Phys.org |quote=Among the feared consequences of the rise of the robots is the growing impact they will have on human jobs and economies. |archive-date=2020-03-06 |archive-url=https://web.archive.org/web/20200306021915/https://phys.org/news/2017-06-ai-good-world-ultra-lifelike-robot.html |url-status=live }}</ref><ref>{{Cite web |last=Santini |first=Jean-Louis |date=2016-02-14 |title=Intelligent robots threaten millions of jobs |url=https://phys.org/news/2016-02-intelligent-robots-threaten-millions-jobs.html#nRlv |access-date=2017-08-15 |website=Phys.org |quote="We are approaching a time when machines will be able to outperform humans at almost any task," said Moshe Vardi, director of the Institute for Information Technology at Rice University in Texas. |archive-date=2019-01-01 |archive-url=https://web.archive.org/web/20190101014340/https://phys.org/news/2016-02-intelligent-robots-threaten-millions-jobs.html#nRlv |url-status=live }}</ref><ref>{{Cite web |last=Williams-Grut |first=Oscar |date=2016-02-15 |title=Robots will steal your job: How AI could increase unemployment and inequality |url=http://www.businessinsider.com/robots-will-steal-your-job-citi-ai-increase-unemployment-inequality-2016-2?r=UK&IR=T |access-date=2017-08-15 |website=Businessinsider.com |publisher=Business Insider |quote=Top computer scientists in the US warned that the rise of artificial intelligence (AI) and robots in the workplace could cause mass unemployment and dislocated economies, rather than simply unlocking productivity gains and freeing us all up to watch TV and play sports. |archive-date=2017-08-16 |archive-url=https://web.archive.org/web/20170816061548/http://www.businessinsider.com/robots-will-steal-your-job-citi-ai-increase-unemployment-inequality-2016-2?r=UK&IR=T |url-status=live }}</ref> Many small and medium-sized firms may also be forced to close if they cannot afford or license the latest robotic and AI technology, and may need to focus on areas or services that cannot easily be replaced for continued viability in the face of such technology.<ref>{{Cite news |date=2017-10-17 |title=How can SMEs prepare for the rise of the robots? |language=en-US |work=LeanStaff |url=http://www.leanstaff.co.uk/robot-apocalypse/ |url-status=dead |access-date=2017-10-17 |archive-url=https://web.archive.org/web/20171018073852/http://www.leanstaff.co.uk/robot-apocalypse/ |archive-date=2017-10-18}}</ref>
==== Technologies that may displace workers ==== While these technologies have replaced some traditional workers, they also create new opportunities. Industries that are most susceptible to AI-driven automation include transportation, retail, and the military. AI military technologies, for example, can reduce risk by enabling remote operation. A study in 2024 highlights AI's ability to perform routine and repetitive tasks poses significant risks of job displacement, especially in sectors like manufacturing and administrative support.<ref>{{Cite journal |last1=Hassan Soueidan |first1=Mohamad |last2=Shoghari |first2=Rodwan |date=2024-05-09 |title=The Impact of Artificial Intelligence on Job Loss: Risks for Governments |url=https://techniumscience.com/index.php/socialsciences/article/view/10917 |journal=Technium Social Sciences Journal |volume=57 |pages=206–223 |language=en-US |doi=10.47577/tssj.v57i1.10917|doi-access=free }}</ref> Author Dave Bond argues that as AI technologies continue to develop and expand, the relationship between humans and robots will change; they will become closely integrated in several aspects of life. AI will likely displace some workers while creating opportunities for new jobs in other sectors, especially in fields where tasks are repeatable.<ref>{{Cite journal|last=Frank|first=Morgan|date=2019-03-25|title=Toward understanding the impact of artificial intelligence on labor|journal=Proceedings of the National Academy of Sciences of the United States of America|volume=116|issue=14|pages=6531–6539|doi=10.1073/pnas.1900949116|pmid=30910965|pmc=6452673|bibcode=2019PNAS..116.6531F |doi-access=free}}</ref><ref>{{Cite book|last=Bond|first=Dave|title=Artificial Intelligence|year=2017|pages=67–69}}</ref>
Researchers from Stanford's Digital Economy Lab reported in 2025 that since the widespread adoption of generative AI in late 2022, early-career workers (ages 22–25) in the most AI-exposed occupations have experienced a 13 percent relative decline in employment—even after controlling for firm-level shocks—while overall employment has continued to grow robustly.<ref>{{cite web|last=Brynjolfsson|first=Erik|title=Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence|url=https://digitaleconomy.stanford.edu/wp-content/uploads/2025/08/Canaries_BrynjolfssonChandarChen.pdf|work=Stanford Digital Economy Lab|date=2025-08-26|access-date=2025-08-26}}</ref> The study further finds that job losses are concentrated in roles where AI automates routine tasks, whereas occupations that leverage AI to augment human work have seen stable or increasing employment.<ref>{{cite news|title=AI is already taking jobs away from entry-level workers|url=https://www.axios.com/2025/08/26/ai-entry-level-jobs|work=Axios|date=2025-08-26|access-date=2025-08-26}}</ref>
==== Computer-integrated manufacturing ==== {{See also|Artificial intelligence in industry}}
Computer-integrated manufacturing uses computers to control the production process. This allows individual processes to exchange information with each other and initiate actions. Although manufacturing can be faster and less error-prone through the integration of computers, the main advantage is the ability to create automated manufacturing processes. Computer-integrated manufacturing is used in automotive, aviation, space, and shipbuilding industries.
==== White-collar machines ==== {{See also|White-collar worker}}
The 21st century has seen a variety of skilled tasks partially taken over by machines, including translation, legal research, and journalism. Care work, entertainment, and other tasks requiring empathy, previously thought safe from automation, are increasingly performed by robots and AI systems.<ref>{{Cite news |last=Skidelsky |first=Robert |author-link=Robert Skidelsky, Baron Skidelsky |date=2013-02-19 |title=Rise of the robots: what will the future of work look like? |url=https://www.theguardian.com/business/2013/feb/19/rise-of-robots-future-of-work |url-status=live |archive-url=https://web.archive.org/web/20190403203821/https://www.theguardian.com/business/2013/feb/19/rise-of-robots-future-of-work |archive-date=2019-04-03 |access-date=14 July 2015 |work=The Guardian |location=London, England}}</ref><ref>{{Cite web |last=Bria |first=Francesca |date=February 2016 |title=The robot economy may already have arrived |url=https://www.opendemocracy.net/can-europe-make-it/francesca-bria/robot-economy-full-automation-work-future |access-date=20 May 2016 |publisher=openDemocracy |archive-date=17 May 2016 |archive-url=https://web.archive.org/web/20160517215840/https://www.opendemocracy.net/can-europe-make-it/francesca-bria/robot-economy-full-automation-work-future |url-status=dead }}</ref><ref>{{Cite web |last=Srnicek |first=Nick |author-link=Nick Srnicek |date=March 2016 |title=4 Reasons Why Technological Unemployment Might Really Be Different This Time |url=http://wire.novaramedia.com/2015/03/4-reasons-why-technological-unemployment-might-really-be-different-this-time/ |url-status=dead |archive-url=https://web.archive.org/web/20160625161447/http://wire.novaramedia.com/2015/03/4-reasons-why-technological-unemployment-might-really-be-different-this-time/ |archive-date=25 June 2016 |access-date=20 May 2016 |publisher=novara wire}}</ref><ref>{{Cite book |last1=Brynjolfsson |first1=Erik |title=The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies |last2=McAfee |first2=Andrew |publisher=W. W. Norton & Company |year=2014 |isbn=978-0393239355 |chapter=''passim'', see esp Chpt. 9}}</ref>
==== Autonomous cars ====
An autonomous car is a vehicle that is capable of sensing its environment and navigating without human input. Many such vehicles are operational and others are being developed, with legislation rapidly expanding to allow their use. Obstacles to widespread adoption of autonomous vehicles have included concerns about the resulting loss of driving-related jobs in the road transport industry, and safety concerns. On March 18, 2018, a pedestrian was struck and killed in Tempe, Arizona by an Uber self-driving car.<ref>{{Cite news |last=Wakabayashi |first=Daisuke |date=March 19, 2018 |title=Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam |url=https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html |url-status=live |archive-url=https://web.archive.org/web/20200421221918/https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html |archive-date=April 21, 2020 |access-date=March 23, 2018 |work=New York Times |location=New York, New York}}</ref>
==== AI-generated content ==== {{See also|Artificial intelligence art}} In the 2020s, automated content became more relevant due to technological advancements in AI models, such as ChatGPT, DALL-E, and Stable Diffusion. In most cases, AI-generated content such as imagery, literature, and music are produced through text prompts. These AI models are sometimes integrated into creative programs.
AI-generated art may sample and conglomerate existing creative works, producing results that appear similar to human-made content. Low-quality AI-generated visual artwork can be informally referred to as AI slop. Some artists use a tool called Nightshade that alters images to make them detrimental to the training of text-to-image models if scraped without permission, while still looking normal to humans.<ref>{{Cite web |last= |first= |date=2023-10-23 |title=This new data poisoning tool lets artists fight back against generative AI |url=https://www.technologyreview.com/2023/10/23/1082189/data-poisoning-artists-fight-generative-ai/ |access-date=2025-08-03 |website=MIT Technology Review |language=en}}</ref> AI-generated images are a potential tool for scammers and those looking to gain followers on social media, either to impersonate a famous individual or group or to monetize their audience.<ref>{{Cite journal |last1=DiResta |first1=Renée |last2=Goldstein |first2=Josh A. |date=2024-08-15 |title=How spammers and scammers leverage AI-generated images on Facebook for audience growth |url=https://misinforeview.hks.harvard.edu/article/how-spammers-and-scammers-leverage-ai-generated-images-on-facebook-for-audience-growth/ |journal=Harvard Kennedy School Misinformation Review |language=en-US |doi=10.37016/mr-2020-151|doi-access=free }}</ref>
The New York Times has sued OpenAI, alleging copyright infringement related to the training and outputs of its AI models.<ref>{{cite book | last1 = Jiang | first1 = Harry H. | last2 = Brown | first2 = Lauren | last3 = Cheng | first3 = Jessica | last4 = Khan | first4 = Mehtab | last5 = Gupta | first5 = Abhishek | last6 = Workman | first6 = Deja | last7 = Hanna | first7 = Alex | last8 = Flowers | first8 = Johnathan | last9 = Gebru | first9 = Timnit | chapter = AI Art and its Impact on Artists | title = Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society | pages = 363–374 | publisher = Association for Computing Machinery | date = 29 August 2023 | doi = 10.1145/3600211.3604681 | doi-access = free | isbn = 979-8-4007-0231-0 }}</ref><ref>{{cite arXiv | title = Can There be Art Without an Artist? | last1 = Ghosh | first1 = Avijit | last2 = Fossas | first2 = Genoveva | date = 19 November 2022 | class = cs.AI | eprint = 2209.07667 }}</ref><ref>{{cite arXiv | title = Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models | last1 = Shan | first1 = Shawn | last2 = Cryan | first2 = Jenna | last3 = Wenger | first3 = Emily | last4 = Zheng | first4 = Haitao | last5 = Hanocka | first5 = Rana | last6 = Zhao | first6 = Ben Y. | date = 3 August 2023 | class = cs.CR | eprint = 2302.04222 }}</ref><ref>{{cite news | last = Brooks | first = Libby | title = Glasgow Willy Wonka experience called a 'farce' as tickets refunded | work = The Guardian | date = 27 February 2024 | url = https://www.theguardian.com/uk-news/2024/feb/27/glasgow-willy-wonka-experience-slammed-as-farce-as-tickets-refunded | access-date = 2 April 2024}}</ref><ref>{{cite news | last1 = Metz | first1 = Cade | last2 = Robertson | first2 = Katie | title = OpenAI Seeks to Dismiss Parts of The New York Times's Lawsuit | work = The New York Times | date = 27 February 2024 | url = https://www.nytimes.com/2024/02/27/technology/openai-new-york-times-lawsuit.html?smid=url-share | access-date = 4 April 2024}}</ref>
=== Eradication === {{Main|Existential risk from artificial intelligence}}
Scientists such as Stephen Hawking are confident that superhuman artificial intelligence is physically possible, stating "there is no physical law precluding particles from being organised in ways that perform even more advanced computations than the arrangements of particles in human brains".<ref>{{Cite news |last1=Hawking |first1=Stephen |last2=Russell |first2=Stuart J. |author2-link=Stuart J. Russell |last3=Tegmark |first3=Max |author3-link=Max Tegmark |last4=Wilczek |first4=Frank |author4-link=Frank Wilczek |date=1 May 2014 |title=Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?' |url=https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-9313474.html |url-access=limited |url-status=live |archive-url=https://web.archive.org/web/20151002023652/http://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence-but-are-we-taking-9313474.html |archive-date=2015-10-02 |access-date=1 April 2016 |work=The Independent}}</ref><ref>{{cite book | last1=Müller | first1=Vincent C. | author-link1=Vincent C. Müller | last2=Bostrom | first2=Nick | author-link2=Nick Bostrom | title=Fundamental Issues of Artificial Intelligence | chapter=Future Progress in Artificial Intelligence: A Survey of Expert Opinion | publisher=Springer | year=2016 | isbn=978-3-319-26483-7 | doi=10.1007/978-3-319-26485-1_33 | pages=555–572 | chapter-url=https://nickbostrom.com/papers/survey.pdf | quote=AI systems will... reach overall human ability... very likely (with 90% probability) by 2075. From reaching human ability, it will move on to superintelligence within 30 years (75%)... So, (most of the AI experts responding to the surveys) think that superintelligence is likely to come in a few decades... | access-date=2022-06-16 | archive-date=2022-05-31 | archive-url=https://web.archive.org/web/20220531142709/https://nickbostrom.com/papers/survey.pdf | url-status=live }}</ref> According to Nick Bostrom, a superintelligent machine would not necessarily be motivated by the same emotional desire to collect power that often drives human beings but might rather treat power as a means toward attaining its ultimate goals; taking over the world would both increase its access to resources and help to prevent other agents from stopping the machine's plans. As a simplified example, a paperclip maximizer designed solely to create as many paperclips as possible would want to take over the world so that it can use all of the world's resources to create as many paperclips as possible, and, additionally, prevent humans from shutting it down or using those resources on things other than paperclips.<ref>{{cite journal | last=Bostrom | first=Nick | title=The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents | journal=Minds and Machines | publisher=Springer | volume=22 | issue=2 | year=2012 | doi=10.1007/s11023-012-9281-3 | pages=71–85 | s2cid=254835485 | url=https://nickbostrom.com/superintelligentwill.pdf | access-date=2022-06-16 | archive-date=2022-07-09 | archive-url=https://web.archive.org/web/20220709032134/https://nickbostrom.com/superintelligentwill.pdf | url-status=live }}</ref>
There are debates on how realistic AI takeover scenarios are. According to a 2026 research paper, many of the arguments about existential risks are based on speculative assumptions about how intelligent AI systems could become, how they would behave and what goals they might develop over time.<ref>{{Cite journal |last=Bareis |first=Jascha |last2=Ackerl |first2=Clemens |last3=Heil |first3=Reinhard |date=2026-02-09 |title=AI going rogue? An integrative narrative review of the tacit assumptions underlying existential AI-risks |url=https://doi.org/10.1007/s43681-025-00928-w |journal=AI and Ethics |language=en |volume=6 |issue=1 |pages=152 |doi=10.1007/s43681-025-00928-w |issn=2730-5961|doi-access=free }}</ref>
A 2023 Reuters/Ipsos survey showed that 61% of American adults feared AI could pose a threat to civilization. Philosopher Niels Wilde refutes the common thread that artificial intelligence inherently presents a looming threat to humanity, stating that these fears stem from perceived intelligence and lack of transparency in AI systems that more closely reflects the human aspects of it rather than those of a machine.<ref>{{Cite journal |last=Wilde |first=Niels |date=2025-04-14 |title=Fear of artificial intelligence or fear of looking in the mirror? Revisiting the Western machine-takeover imaginary |url=https://link.springer.com/10.1007/s00146-025-02355-1 |journal=AI & Society |volume=40 |issue=7 |pages=5347–5357 |language=en |doi=10.1007/s00146-025-02355-1 |issn=0951-5666}}</ref> AI alignment research studies how to design AI systems so that they follow intended objectives.<ref>{{Cite journal |last=Bruiger |first=Dan |date=2025 |title=Reflections on the AI alignment problem |url=https://link.springer.com/10.1007/s00146-025-02211-2 |journal=AI & Society |language=en |volume=40 |issue=6 |pages=4383–4392 |doi=10.1007/s00146-025-02211-2 |issn=0951-5666|url-access=subscription }}</ref>
== Debate == {{missing information|section|skepticisms and doubt about this topic|talksection=Skepticisms toward this topic missing?|date=May 2026}} Physicist Stephen Hawking, Microsoft founder Bill Gates, and SpaceX founder Elon Musk have expressed concerns about the possibility that AI could develop to the point that humans could not control it, with Hawking theorizing that this could "spell the end of the human race".<ref>{{Cite web |last=Rawlinson |first=Kevin |date=29 January 2015 |title=Microsoft's Bill Gates insists AI is a threat |url=https://www.bbc.co.uk/news/31047780 |url-status=live |archive-url=https://web.archive.org/web/20150129183607/http://www.bbc.co.uk/news/31047780 |archive-date=29 January 2015 |access-date=30 January 2015 |website=BBC News}}</ref> Stephen Hawking said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." Hawking believed that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." In January 2015, Nick Bostrom joined Stephen Hawking, Max Tegmark, Elon Musk, Lord Martin Rees, Jaan Tallinn, and numerous AI researchers in signing the Future of Life Institute's open letter speaking to the potential risks and benefits associated with artificial intelligence. The signatories "believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today."<ref>{{Cite web |date=28 October 2015 |title=The Future of Life Institute Open Letter |url=http://futureoflife.org/ai-open-letter |url-status=live |archive-url=https://web.archive.org/web/20190329094536/https://futureoflife.org/ai-open-letter/ |archive-date=29 March 2019 |access-date=29 March 2019 |publisher=The Future of Life Institute}}</ref><ref>{{Cite web |last=Bradshaw |first=Tim |date=11 January 2015 |title=Scientists and investors warn on AI |url=https://www.ft.com/content/3d2c2f12-99e9-11e4-93c1-00144feabdc0 |url-status=live |archive-url=https://web.archive.org/web/20150207042806/http://www.ft.com/cms/s/0/3d2c2f12-99e9-11e4-93c1-00144feabdc0.html#axzz3TNL9lxJV |archive-date=7 February 2015 |access-date=4 March 2015 |publisher=The Financial Times}}</ref>
Some focus has been placed on the development of trustworthy AI. Three statements have been posed as to why AI is not inherently trustworthy:<ref name=":0">{{Cite journal |last1=Zanotti |first1=Giacomo |last2=Petrolo |first2=Mattia |last3=Chiffi |first3=Daniele |last4=Schiaffonati |first4=Viola |date=2024 |title=Keep trusting! A plea for the notion of Trustworthy AI |journal=AI & Society |language=en |volume=39 |issue=6 |pages=2691–2702 |doi=10.1007/s00146-023-01789-9 |issn=0951-5666|doi-access=free |hdl=11311/1252561 |hdl-access=free }}</ref>
{{Blockquote|text=1. An entity X is trustworthy only if X has the right motivations, goodwill and/or adheres to moral obligations towards the trustor; 2. AI systems lack motivations, goodwill, and moral obligations; 3. Therefore, AI systems cannot be trustworthy.|author=Giacomo Zanotti et al.}}
There are additional considerations within this framework of trustworthy AI that go further into the fields of explainable artificial intelligence and respect for human privacy.<ref name="aima4">{{Cite book |last1=Russell |first1=Stuart J. |url=https://www.pearson.com/us/higher-education/program/Russell-Artificial-Intelligence-A-Modern-Approach-4th-Edition/PGM1263338.html |title=Artificial intelligence: A modern approach |last2=Norvig |first2=Peter |publisher=Pearson |year=2021 |isbn=9780134610993 |edition=4th |pages=5, 1003 |access-date=September 12, 2022}}</ref> Zanotti and colleagues argue that while a trustworthy AI may not exist at present that meets all of the requirements of "trustworthiness", one may be developed in the future once clear ethical and technical frameworks exist.<ref name=":0" />
== In fiction == {{Main|AI takeovers in popular culture}}
{{See also|Artificial intelligence in fiction|Self-replicating machines in fiction}} [[File:Capek RUR.jpg|thumbnail|upright=1.2|Robots revolt in ''R.U.R.'', a 1928 Czech play translated as "Rossum's Universal Robots"]]
AI takeover is a recurring theme in science fiction. Fictional scenarios typically differ vastly from those hypothesized by researchers in that they involve an active conflict between humans and an AI or robots with anthropomorphic motives who see them as a threat or otherwise have an active desire to fight humans, as opposed to the researchers' concern of an AI that rapidly exterminates humans as a byproduct of pursuing its goals.<ref name="bostrom-superintelligence">{{Cite book |last=Bostrom |first=Nick |title=Superintelligence: Paths, Dangers, Strategies |title-link=Superintelligence: Paths, Dangers, Strategies}}</ref> The idea is seen in Karel Čapek's ''R.U.R.'', which introduced the word ''robot'' in 1920,<ref>{{Cite news |date=22 April 2011 |title=The Origin Of The Word 'Robot' |work=Science Friday (public radio) |url=https://www.sciencefriday.com/segments/the-origin-of-the-word-robot/ |access-date=30 April 2020 |archive-date=14 March 2020 |archive-url=https://web.archive.org/web/20200314092540/https://www.sciencefriday.com/segments/the-origin-of-the-word-robot/ |url-status=live }}</ref> and can be glimpsed in Mary Shelley's ''Frankenstein'' (published in 1818), as Victor ponders whether, if he grants his monster's request and makes him a wife, they would reproduce and their kind would destroy humanity.<ref>{{Cite news |last=Botkin-Kowacki |first=Eva |date=28 October 2016 |title=A female Frankenstein would lead to humanity's extinction, say scientists |work=Christian Science Monitor |url=https://www.csmonitor.com/Science/2016/1028/A-female-Frankenstein-would-lead-to-humanity-s-extinction-say-scientists |access-date=30 April 2020 |archive-date=26 February 2021 |archive-url=https://web.archive.org/web/20210226203855/https://www.csmonitor.com/Science/2016/1028/A-female-Frankenstein-would-lead-to-humanity-s-extinction-say-scientists |url-status=live }}</ref>
According to Toby Ord, the idea that an AI takeover requires robots is a misconception driven by the media and Hollywood. He argues that the most damaging humans in history were not physically the strongest, but that they used words instead to convince people and gain control of large parts of the world. He writes that a ''sufficiently'' intelligent AI with access to the internet could scatter backup copies of itself, gather financial and human resources (via cyberattacks or blackmails), persuade people on a large scale, and exploit societal vulnerabilities that are too subtle for humans to anticipate.<ref>{{Cite book |last=Ord |first=Toby |title=The precipice: existential risk and the future of humanity |date=2020 |publisher=Bloomsbury academic |isbn=978-1-5266-0023-3 |location=London, England and New York, New York |language=en |chapter=Unaligned artificial intelligence}}</ref>
The word "robot" from ''R.U.R.'' comes from the Czech word ''robota'', meaning laborer or serf. The 1920 play was a protest against the rapid growth of technology, featuring manufactured "robots" with increasing capabilities who eventually revolt.<ref name="surgery">{{Cite journal |last1=Hockstein |first1=N. G. |last2=Gourin |first2=C. G. |last3=Faust |first3=R. A. |last4=Terris |first4=D. J. |date=17 March 2007 |title=A history of robots: from science fiction to surgical robotics |journal=Journal of Robotic Surgery |volume=1 |issue=2 |pages=113–118 |doi=10.1007/s11701-007-0021-2 |pmc=4247417 |pmid=25484946}}</ref> HAL 9000 (1968) and the original Terminator (1984) are two iconic examples of hostile AI in pop culture.<ref>{{Cite news |last=Hellmann |first=Melissa |date=21 September 2019 |title=AI 101: What is artificial intelligence and where is it going? |work=The Seattle Times |url=https://www.seattletimes.com/business/technology/ai-101-what-is-artificial-intelligence-and-where-is-it-going/ |access-date=30 April 2020 |archive-date=21 April 2020 |archive-url=https://web.archive.org/web/20200421232439/https://www.seattletimes.com/business/technology/ai-101-what-is-artificial-intelligence-and-where-is-it-going/ |url-status=live }}</ref>
== Contributing factors == === Advantages of superhuman intelligence over humans === Nick Bostrom and others have expressed concern that an AI with the abilities of a competent artificial intelligence researcher would be able to modify its own source code and increase its own intelligence. If its self-reprogramming leads to getting even better at being able to reprogram itself, the result could be a recursive intelligence explosion in which it would rapidly leave human intelligence far behind. Bostrom defines a superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest", and enumerates some advantages a superintelligence would have if it chose to compete against humans:<ref name=bostrom-superintelligence/><!-- bostrom-superintelligence, Chapter 6: Cognitive Superpowers, Table 8 --><ref name="BabcockKrámar2019">{{Cite book |last1=Babcock |first1=James |title=Next-Generation Ethics |last2=Krámar |first2=János |last3=Yampolskiy |first3=Roman V. |year=2019 |isbn=9781108616188 |pages=90–112 |chapter=Guidelines for Artificial Intelligence Containment |doi=10.1017/9781108616188.008 |author-link3=Roman Yampolskiy |arxiv=1707.08476 |s2cid=22007028}}<!-- in Next-Generation Ethics: Engineering a Better Society, Cambridge University Press --></ref>
* Technology research: A machine with superhuman scientific research abilities would be able to beat the human research community to milestones such as nanotechnology or advanced biotechnology * Strategizing: A superintelligence might be able to simply outwit human opposition * Social manipulation: A superintelligence might be able to recruit human support,<ref name=bostrom-superintelligence /> or covertly incite a war between humans<ref>{{Cite news |last=Baraniuk |first=Chris |date=23 May 2016 |title=Checklist of worst-case scenarios could help prepare for evil AI |work=New Scientist |url=https://www.newscientist.com/article/2089606-checklist-of-worst-case-scenarios-could-help-prepare-for-evil-ai/ |access-date=21 September 2016 |archive-date=21 September 2016 |archive-url=https://web.archive.org/web/20160921061131/https://www.newscientist.com/article/2089606-checklist-of-worst-case-scenarios-could-help-prepare-for-evil-ai/ |url-status=live }}</ref> * Economic productivity: As long as a copy of the AI could produce more economic wealth than the cost of its hardware, individual humans would have an incentive to voluntarily allow the artificial general intelligence (AGI) to run a copy of itself on their systems * Hacking: A superintelligence could find new exploits in computers connected to the Internet, and spread copies of itself onto those systems, or might steal money to finance its plans
==== Sources of AI advantage ==== According to Bostrom, a computer program that faithfully emulates a human brain, or that runs algorithms that are as powerful as the human brain's algorithms, could still become a "speed superintelligence" if it can think orders of magnitude faster than a human, due to being made of silicon rather than flesh, or due to optimization increasing the speed of the AGI. Biological neurons operate at about 200 Hz, whereas a modern microprocessor operates at a speed of about 2 GHz. Human axons carry action potentials at around 120 m/s, whereas computer signals travel near the speed of light.<ref name="bostrom-superintelligence" /><!-- chapter 3 -->
A network of human-level intelligences designed to network together and share complex thoughts and memories seamlessly, able to collectively work as a giant unified team without friction, or consisting of trillions of human-level intelligences, would become a "collective superintelligence".<ref name="bostrom-superintelligence" /><!-- chapter 3 -->
More broadly, any number of qualitative improvements to a human-level AGI could result in a "quality superintelligence", perhaps resulting in an AGI as far above us in intelligence as humans are above apes. The number of neurons in a human brain is limited by cranial volume and metabolic constraints, while the number of processors in a supercomputer can be indefinitely expanded. An AGI need not be limited by human constraints on working memory, and might therefore be able to intuitively grasp more complex relationships than humans can. An AGI with specialized cognitive support for engineering or computer programming would have an advantage in these fields, compared with humans who did not evolve specialized cognitive modules for them. Unlike humans, an AGI can spawn copies of itself and tinker with its copies' source code to attempt to further improve its algorithms.<ref name="bostrom-superintelligence" /><!-- chapter 3 -->
=== Possibility of unfriendly AI preceding friendly AI ===
==== Morality ==== {{main|AI alignment}}
The sheer complexity of human value systems makes it very difficult to make AI's motivations human-friendly.<ref name="bostrom-superintelligence" /><ref name="Muehlhauser, Luke 2012">{{Cite book |last1=Muehlhauser |first1=Luke |title=Singularity Hypotheses: A Scientific and Philosophical Assessment |last2=Helm |first2=Louie |publisher=Springer |year=2012 |chapter=Intelligence Explosion and Machine Ethics |chapter-url=https://intelligence.org/files/IE-ME.pdf |access-date=2020-10-02 |archive-date=2015-05-07 |archive-url=https://web.archive.org/web/20150507173028/http://intelligence.org/files/IE-ME.pdf |url-status=live }}</ref> Unless moral philosophy provides us with a flawless ethical theory, an AI's utility function could allow for many potentially harmful scenarios that conform with a given ethical framework but not "common sense". According to AI researcher Eliezer Yudkowsky, there is little reason to suppose that an artificially designed mind would have such an adaptation.<ref name="Yudkowsky2011">{{Cite book |last=Yudkowsky |first=Eliezer |title=Artificial General Intelligence |year=2011 |isbn=978-3-642-22886-5 |series=Lecture Notes in Computer Science |volume=6830 |pages=388–393 |chapter=Complex Value Systems in Friendly AI |doi=10.1007/978-3-642-22887-2_48 |issn=0302-9743}}</ref>
==== Odds of conflict ==== In a 2018 essay, evolutionary psychologist Steven Pinker argued that human drives toward dominance and resource acquisition were shaped by natural selection, and that a superintelligent AI would not necessarily share those drives.<ref name="pinker now">{{Cite news |last=Pinker |first=Steven |date=13 February 2018 |title=We're told to fear robots. But why do we think they'll turn on us? |language=en |work=Popular Science |url=https://www.popsci.com/robot-uprising-enlightenment-now/ |access-date=8 June 2020 |archive-date=20 July 2020 |archive-url=https://web.archive.org/web/20200720164306/https://www.popsci.com/robot-uprising-enlightenment-now/ |url-status=live }}</ref>
The fear of cybernetic revolt is often based on interpretations of humanity's history, which is rife with incidents of enslavement and genocide. Such fears stem from a belief that competitiveness and aggression are necessary in any intelligent being's goal system. However, such human competitiveness stems from the evolutionary background to our intelligence, where the survival and reproduction of genes in the face of human and non-human competitors was the central goal.<ref>''[http://www.singinst.org/ourresearch/presentations/ Creating a New Intelligent Species: Choices and Responsibilities for Artificial Intelligence Designers] {{webarchive |url=https://web.archive.org/web/20070206060938/http://www.singinst.org/ourresearch/presentations/ |date=February 6, 2007 }}'' - Singularity Institute for Artificial Intelligence, 2005</ref> According to AI researcher Steve Omohundro, an arbitrary intelligence could have arbitrary goals: there is no particular reason that an artificially intelligent machine (not sharing humanity's evolutionary context) would be hostile—or friendly—unless its creator programs it to be such and it is not inclined or capable of modifying its programming. But the question remains: what would happen if AI systems could interact and evolve (evolution in this context means self-modification or selection and reproduction) and need to compete over resources—would that create goals of self-preservation? AI's goal of self-preservation could be in conflict with some goals of humans.<ref>{{Cite conference |last=Omohundro |first=Stephen M. |date=June 2008 |title=The basic AI drives |url=https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf |conference=Artificial General Intelligence 2008 |pages=483–492 |access-date=2020-10-02 |archive-date=2020-10-10 |archive-url=https://web.archive.org/web/20201010072132/https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf |url-status=live }}</ref>
Many scholars dispute the likelihood of unanticipated cybernetic revolt as depicted in science fiction such as ''The Matrix'', arguing that it is more likely that any artificial intelligence powerful enough to threaten humanity would probably be programmed not to attack it. Pinker acknowledges the possibility of deliberate "bad actors", but states that in the absence of bad actors, unanticipated accidents are not a significant threat; Pinker argues that a culture of engineering safety will prevent AI researchers from accidentally unleashing malign superintelligence.<ref name="pinker now" /> In contrast, Yudkowsky argues that humanity is less likely to be threatened by deliberately aggressive AIs than by AIs which were programmed such that their goals are unintentionally incompatible with human survival or well-being (as in the film ''I, Robot'' and in the short story "The Evitable Conflict"). Omohundro suggests that present-day automation systems are not designed for safety and that AIs may blindly optimize narrow utility functions (say, playing chess at all costs), leading them to seek self-preservation and elimination of obstacles, including humans who might turn them off.<ref name="Tucker2014">{{Cite news |last=Tucker |first=Patrick |date=17 Apr 2014 |title=Why There Will Be A Robot Uprising |agency=Defense One |url=http://www.defenseone.com/technology/2014/04/why-there-will-be-robot-uprising/82783/ |access-date=15 July 2014 |archive-date=6 July 2014 |archive-url=https://web.archive.org/web/20140706110100/http://www.defenseone.com/technology/2014/04/why-there-will-be-robot-uprising/82783/ |url-status=live }}</ref>
==== Precautions ==== The AI control problem is the challenge of ensuring that advanced AI systems reliably act according to human values and intentions, even as they become more capable than humans.<ref>{{Cite book |last=Russell |first=Stuart J. |title=Human Compatible: Artificial Intelligence and the Problem of Control |date=8 October 2019 |publisher=Penguin |isbn=978-0-525-55862-0 |oclc=1237420037}}</ref> Some scholars argue that solutions to the control problem might also find applications in existing non-superintelligent AI.<ref name="bbc-google">{{Cite news |date=8 June 2016 |title=Google developing kill switch for AI |work=BBC News |url=https://www.bbc.com/news/technology-36472140 |access-date=7 June 2020 |archive-date=11 June 2016 |archive-url=https://web.archive.org/web/20160611042244/http://www.bbc.com/news/technology-36472140 |url-status=live }}</ref>
Major approaches to the control problem include ''alignment'', which aims to align AI goal systems with human values, and ''capability control'', which aims to reduce an AI system's capacity to harm humans or gain control. An example of "capability control" is to research whether a superintelligent AI could be successfully confined in an "AI box". According to Bostrom, such capability control proposals are not reliable or sufficient to solve the control problem in the long term, but may potentially act as valuable supplements to alignment efforts.<ref name=bostrom-superintelligence/>
== See also ==
{{div col|colwidth=30em}} * Philosophy of artificial intelligence * Artificial intelligence arms race * Autonomous robot ** Industrial robot ** Mobile robot ** Self-replicating machine * Cyberocracy * Effective altruism * Future of Humanity Institute * Global catastrophic risk (existential risk) * Government by algorithm * How to Survive a Robot Uprising * Human extinction * Machine ethics * Machine learning/Deep learning * Transhumanism * Self-replication * Superintelligence * ''Superintelligence: Paths, Dangers, Strategies'' * Technophobia * Technological singularity {{div col end}}
== References == {{Reflist}}
== External links == * TED talk: [https://www.youtube.com/watch?v=8nt3edWLgIg "Can we build AI without losing control over it?"] by Sam Harris
{{Existential risk from artificial intelligence|state=expanded}} {{Artificial intelligence navbox|state=collapsed}} {{doomsday}}
{{DEFAULTSORT:AI takeover}} Category:Doomsday scenarios Category:Future problems Category:Science fiction themes Category:Existential risk from artificial intelligence Category:Technophobia Category:Artificial intelligence controversies