Wisdom DOES Imply Benevolence
description
Transcript of Wisdom DOES Imply Benevolence
Wisdom DOES Imply Benevolence
Mark R. Waser
Super-Intelligence Ethics(except in a very small number of low-probability edge cases)
So . . . What’s the problem?
Superintelligence does not imply benevolence
Fox, J. & Shulman C. (2010) Superintelligence Does Not Imply Benevolence. In K. Mainzer (ed.), ECAP10: VIII European Conference on Computing and Philosophy (pp. 456-462) Munich: Verlag.
If machines become more intelligent than humans, will their intelligence
lead them toward beneficial behavior toward humans even without specific
efforts to design moral machines?
References• Evolution of reciprocal altruism (Trivers 1971)• Increase in scope of cooperation (Wright 2000)• Reduction in rates of violence (Pinker 2007)• Expanding circle of moral concern (Singer
1981)
• D. Gauthier • J. Haidt• S. Omohundro
One might generalize from this trend and argue that as machines approach
and exceed human cognitive capacities, moral behavior will improve in tandem.
Ceteris Paribus(other things being equal)
intelligence – the ability to achieve goals in a wide range of environments.
• intelligence can be far less important than
• goal system properties & contentin determining benevolence vs. malevolence
For example,
If an intelligence has the single goal to *destroy humanity*,
increased intelligence will only make it more malevolent
The human motivational system is opaque, messy, and conflicted,but most importantly transient!
The primary danger of AIs is entirely due to the fact that their goal system *could* be different
“Friendly AI” (Yudkowsky 2001)
An artificial intelligence with a cleanly hierarchical goal system with a single top-level (monomaniacal) goal of “Friendliness” (to humans)
Imagine a “Friendly AI” where Friendliness has been defined (hopefully accidentally) as *DESTROY HUMANITY*
WisdomThe goal/motivation to achieve maximal goals
in terms of number and diversity. • Avoids “lock-in” and short-sighted over-optimization
of goals/utility functions (smoking)• Avoids undesirable endgame strategies (prisoner’s
dilemma)• Promotes avoiding unnecessary actions that preclude
reachable goals including wasting resources and alienating or destroying potential cooperators (waste not, want not)
Two conceptions of moralityThis picture neglects a critical distinction between
1. A system for cooperation
2. A system to protect the weak/helpless
Advances one’s own ends
Demands revision of our ultimate ends
AIs will out-cooperate humans (Hall 2007)
Will AIs revise their preferences to be more moral (Chalmers 2010)?
Paths from intelligence to moral behavior (ways in which increased intelligence might prompt behavior favorable to humans)
1. noticing direct instrumental motivationsAdvances one’s own ends (transient)
2. noticing instrumental benefits to enduring benevolent dispositions/trustworthiness
Advances one’s own ends (permanent?)
3. causing an intrinsic desire for human welfare independent of instrumental concerns
Revision of ends/desires (maybe?)
If you have a verifiable history of being trustworthy when not forced, others do not have to commit
resources to defending against you – and can pass some of those savings on to you
On the other hand, if you harm (or worse, destroy) interesting or useful entities, more powerful entities
will likely decide that *you* need to spend resources as reparations and altruistic punishment (as well as
paying the cost of enforcement)
Basic AI Drives1. AIs will want to self-improve2. AIs will want to be rational3. AIs will try to preserve their utility 4. AIs will try to prevent counterfeit utility5. AIs will be self-protective6. AIs will want to acquire resources and use
them efficientlySteve Omohundro,
Proceedings of the First AGI Conference, 2008
Instrumental Goals
“Without explicit goals to the contrary, AIs are likely to behave like human sociopaths
in their pursuit of resources.”
Any sufficiently advanced intelligence (i.e. one with even merely adequate foresight) is guaranteed to realize and take into account the fact that not asking for help and not being concerned about others will generally only work for a brief period of time before ‘the villagers start gathering pitchforks and torches.’Everything is easier with help & without interference
Cooperation is an instrumental goal!
Goal Systems, Morality, and David Hume’s Is-Ought Divide
In every system of morality, which I have hitherto met with, I have always remark'd, that the author proceeds for some time in the ordinary ways of reasoning, and establishes the being of a God, or makes observations concerning human affairs; when all of a sudden I am surpriz'd to find, that instead of the usual copulations of propositions, is, and is not, I meet with no proposition that is not connected with an ought, or an ought not. This change is imperceptible; but is however, of the last consequence. For as this ought, or ought not, expresses some new relation or affirmation, 'tis necessary that it shou'd be observ'd and explain'd; and at the same time that a reason should be given; for what seems altogether inconceivable, how this new relation can be a deduction from others, which are entirely different from it.
Ought
• Requires a goal or desire (or, more correctly, multiples thereof)
• IS the set of actions most likely to fulfill those goals/desires
• For the sum of all goals converges to a universal morality
a superset of^
Moral Systems Are . . .
Haidt & Kesebir, Handbook of Social Psychology, 5th Ed. 2010
interlocking sets of values, virtues, norms, practices, identities, institutions, technologies, and
evolved psychological mechanismsthat work together to
suppress or regulate selfishnessand
make cooperative social life possible.
Are values dependent upon intelligence?
Humean view – values are entirely independent of intelligence
Kantian view – many extremely intelligent beings would converge on (possibly benevolent) substantive normative principles upon reflection
Arguments Pro & Con• Against Kantian – AIXI has no room to
move from reason to values• Against Kantian – Humean design is a
stable equilibrium unless the utility function is self-referential
• Pro Kantian – Humans change our goals under reflection and “often acquire intrinsic preferences for correlates of instrumentally useful actions”.
Quick Answer1. Values are dependent upon goals2. Values are dependent upon instrumental
goals as long as they do not conflict with primary goals
3. Intelligence allows you to see this and take advantage of it, so . . . . YES!
EXAMPLE: Waste not, want not.
Thought Experiment
How would a super-intelligence behave if it knew that it had a goal but that it wouldn’t know that goal until sometime in the future?
Preserving that weak entity may be that goalOr it might have necessary knowledge/skills
Reprise: Three Views of Wisdom
• Waste not, want not• Block as few goals as possible, particularly
Omohundro drives• Fulfill as many goals as possible
Power
• Many of those concerned about intelligent machines appear obsessed with power levels
• Yet, interestingly enough, power is notable in *NOT* being on Omohundro’s list ( i.e. a true instrumental goal
• Will greater intelligence eschew power for efficiency (in diversity)?
An Alternate View of Intelligence• Greater cognitive resources leads to marked
improvements in prediction and reductions in time discounting
• Leads to moving planning horizons out and moving from short-term REQUIREMENTS to long-term optimality
• Indeed, a truly intelligent entity should never be caught in a situation where . . . . (unless out-thought by an even greater intelligence)
“Self-Interest” vs. Ethics
• Higher personal utility (in the short term only)
• More options to choose (in the short term only)
• Less restrictions
• Higher global utility• Less risk (if caught)• Lower cognitive
cost (fewer options, no need to track lies, etc.)
• Assistance & protection when needed/desired