The Future of (Artificial) Intelligence

Stuart RussellUniversity of California, BerkeleyUniversité Pierre et Marie Curie

To create intelligent systems

Why are we doing AI?

To create intelligent systems The more intelligent, the better

Why are we doing AI?

To create intelligent systems The more intelligent, the better

We believe we can succeed Limited only by ingenuity and

physics

Why are we doing AI?

An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made if [we] work on it together for a summer.

John McCarthy and Claude ShannonDartmouth Workshop Proposal, 1956

To create intelligent systems The more intelligent, the better

To gain a better understanding of human intelligence

Why are we doing AI?

To create intelligent systems The more intelligent, the better

To gain a better understanding of human intelligence

To magnify those benefits that flow from it

Why are we doing AI?

Anil Ananthaswamy, “I, Algorithm: A new dawn for AI,” New Scientist, Jan 29, 2011

Once performance exceeds a minimum level, small improvements are worth billions Speech Text understanding Object recognition Automated vehicles Domestic robots

An industry arms race

Military arms race

“The first ultraintelligent machine is the last invention that man need ever make.” I. J. Good, 1965

Might help us avoid war and ecological catastrophes, achieve immortality and expand throughout the universe

Success would be the biggest event in human history

What if we do succeed?

“The first ultraintelligent machine is the last invention that man need ever make.” I. J. Good, 1965

Might help us avoid war and ecological catastrophes, achieve immortality and expand throughout the universe

Success would be the biggest event in human history … and perhaps the last

What if we do succeed?

Along what paths will AI evolve?

What is the (plausibly reachable) best case? Worst case?

Can we affect the future of AI? Technical or societal solutions? What should we do now?

So, if that matters…..

If a superior alien civilization sent us email saying, “We’ll arrive in 30-50 years”, would we just reply, “OK, call us when you get here, we’ll leave the light on”?

The AI community needs a substantial institutional commitment, reasonably soon

This needs serious thought

Rutherford (1933): anyone who looks for a source of power in the transformation of the atoms is talking moonshine.

Sept 12, 1933: The stoplight changed to green. Szilárd stepped off the curb. As he crossed the street time cracked open before him and he saw a way to the future, death into the world and all our woes, the shape of things to come.

Szilard (1934): patent on nuclear chain reaction; kept secret

Precedent: Nuclear Physics

Hahn et al (1939): uranium fissionSzilard and Fermi (1939): uranium chain reaction. “That night, there was very little doubt in my mind that the world was headed for grief.”

Einstein/Szilard (1939): letter to Pres. Roosevelt urging development of nuclear weapons before Nazis

Szilard et 70 al (1945): petition to end war by inviting Japanese to A-bomb test

Emergency Committee of Atomic Scientists

Precedent: Nuclear Physics

1925 Geneva Protocol banned use in warfare, but R&D, stockpiling continued

Long negotiations (until 1972 for biological, 1992 for chemical weapons)

1975 (biological) and 1997 (chemical) treaties ban “development, production, acquisition, stockpiling, retention, transfer or use”

Precedent: Chemical and Biological Weapons

1973-4: Paul Berg stopped his own experiment to insert carcinogenic virus DNA into E. coli; prominent scientists request a moratorium on recombinant DNA experiments

Asilomar conference (1975) set up guidelines: Physical, biological containment; risk

analysis Ban on disease/toxin organism manipulation

Credited with avoiding restrictive legislation

Industry compliance via FDA controls on sales

Precedent: Genetic Engineering

NIH Recombinant DNA Advisory Committee will not approve any protocol modifying human germline

Cartagena Protocol (2003) governs trade in GMOs, enshrines precautionary principle

2010: Pres. Commission proposes federal oversight of synthetic biology activities

2012: 100+ NGOs call for worldwide moratorium

Precedent: Genetic Engineering

MIRI, FHI, CSER, FLI, etc.AAAI task force (Horvitz & Selman)US Air Force: Test, Evaluation, Verification, and Validation for Autonomy

The process is beginning

MIRI, FHI, CSER, FLI, etc.AAAI task force (Horvitz & Selman)US Air Force: Test, Evaluation, Verification, and Validation for Autonomy

Today’s meeting

The process is beginning

1.00-1.30 Introductions (Russell)1.30-1.50 Robots (Veloso)1.50-2.10 Intelligence explosion (Dewey)2.10-2.30 Unintended consequences (Shanahan)2.30-2.50 Autonomous trading (Wellman)2.50-3.10 Ontology, organizations (Mallah)3.10-3.30 break3.30-5.00 Technical research agenda5.00-6.30 Organizational/socioeconomic responses6.30-7.00 walk to reception7.00-8.30 AAMAS reception, Sorbonne “Cordeliers”8.30-10.30 Dinner Le Procope, 13 rue de l'Ancienne Comédie

Meeting Schedule

Verification/validation Designing reward/utility functions

Iterated design/simulation/verification Inductive specification from examples Ensuring compliance

Theory of agents Subsumption, composition, distribution, cooperation,

transparency, etc. Theory of non-agents

Pure question-answering systems Meta-non-agents

Bootstrapping methods Superintelligent self-verification Superintelligent boxing If you were me what utility function would you give yourself?

Technical research agenda

One international, professional society Keep tabs on worldwide effort levels etc. Promulgate culture of responsibility,

safety Develop standards for risk

analysis/verification Keep decision makers and society well

informedLobby for research fundingLegal standards for liability etc.Conferences, conference tracks, journals

Organizational/socioeconomic responses