Computers and Creativity

Bipin IndurkhyaInternational Institute of Information Technology

Gachibowli, Hyderabad

2

Can computers ever be creative?

A working definition of creativity:A cognitive act is deemed creative if it provides a new perspective or new information about an object,

situation or phenomenon

• It is different from intelligence.• It ignores the utility of the new perspective.• Every cognitive agent has the potential for creativity.

Computational systems considered as cognitive agents can also display creativity.

3

Creativity in Perception: Example 1

:: ::

A B C D

?

What perceptual features does this figure have?

Consider the following proportional analogy

Context makes us see new features in the figure.

4

Creativity in perception: Example 2

T Puzzle

The objective is to arrange the four pieces to form the letter T

5

T Puzzle: Solution

Notice: corner ‘a’ is on the outside!

It is very difficult for people to ignore the urge to fill-in this corner, even

when explicitly hinted!

6

Creativity in Cognition: Problem solving

To make an improved paintbrush using synthetic fibers (Schön 1963)

• In 1940s, a product development team was working to solve this problem.

• Existing synthetic-fiber brushes didn’t work as well as natural-fiber brushes.

• Various methods were tried without success. (No difference could be found between the two kinds of brushes.)

7

Creativity in Cognition: Problem solving

‘paintbrush-as-a-pump’ metaphor

• The breakthrough came when someone suggested that the paintbrushmight be viewed as a pump.

• A new ontology is created for the paintbrush. It is with respect to the new ontology that the paintbrush is similar to a pump.

• In this new representation, the space between the fibers plays a key role.

New features of the paintbrush are revealed by the metaphor!

8

Creativity in Cognition: Legal Reasoning

Ben Johnson, an athlete, was given a lifetime suspension for testing positive twice for a banned substance.

• Ben Johnson’s lawyer:– A lifetime ban … means, one talent that he has taken

pride in, he can’t use to make a living.

• The Athletic Federation’s lawyer– It’s a little like saying a jail sentence imposed on a jewel

thief is a restraint of trade.

9

Haiku from Jomei (丈鳴)

言の葉や動かぬ山も秋の色

Leaves of wordsautumn colors

a still mountain

言 words

葉 leaves

言葉 language

Metaphor reveals new similarities between leaves and words.

10

Cognitive mechanisms of creativity: 1

• Gordon (1961): making the familiar strange

• Schön (1963): displacement of concepts

• Dylan Thomas: juxtaposition of dissimilar

They all emphasize getting away from similarity!

11

Cognitive mechanisms of creativity: 2Gianni Rodari (Grammar of Fantasy)

• Estrangement: describe the object as though seen for the first time.

• Association: the ‘clear surface’ of the description opens up the way for other meanings through images — so seek such images through associations.

• Metaphor: form a metaphor using these images.

It is necessary to break the existing conceptualization of the object!

12

An example of a metaphor by deconceptualization

There are some days the happy ocean liesLike an unfingered harp, below the land.

Afternoon gilds all the silent wiresInto a burning music for the eyes.

On mirrors flashing between fine-strung firesThe shore, heaped up with roses, horses, spires

Wanders on water tall above the ribbed sand.

Seascape by Stephen Spender

13

Implications for human creativity

• Humans are constrained by the conceptual associations acquired in their lifetime.

• These associations are genetically and/or culturally inherited, and it is very difficult to ignore them.

• Various techniques have been suggested to break away from these associations:– For example, random juxtaposition. (Rodari)

14

Implications for computer and creativity

• Various techniques have been proposed to model the human conceptual associations in the computer:– Semantic networks– Frames– Scripts

However, they are still far from adequate!

Therefore, computers may be more predisposed towards modeling creativity.

15

Implications for computers and creativity

• Computers have the potential to stimulate human creativity.

• Of course, humans must evaluate the utility of the created perspective.

• However, computers are able to search the conceptual space more deeply than humans.

Computer-generated art based on free-association:Institute of Artificial Art at Amsterdam

http://iaaa.nl/home.html

16

• Stimulate human creativity using strange juxtaposition of familiar objects– Acquire knowledge about familiar objects interactively– Create strange juxtapositions using these objects– Provide these juxtapositions to the user as stimuli for

• Story writing• Expressing feelings or impressions• Etc….

Many possible applications of this idea!

Designing creativity support system

17

System Outline

Knowledge acquisition

module

Stimuli-creating module(create juxtapositions)

User interface(depends on the goal

of the system)

User

General knowledgerepresentation

User knowledgerepresentation

18

Is this created by a human, or by computer?

How soon the servant sun,(Sir morrow mark),

Can time unriddle, and the cupboard stone(Fog has a bone

He’ll trumpet into meat),Unshelve that all my gristles have a gown

And the naked egg stand straight,

by Dylan Thomas

19

Is this created by human, or by computer?

鈴に猫土手に集まる夜寒かな 更けて目が

映る残雪花に明け

Both are computer-generated haiku

20

Is this created by a human, or by computer?

Soul A haiku written by Ray Kurzweil’s Cybernetic Poetafter reading poems by John Keats and Wendy Dennis

You broke my soul the juice of eternity,the spirit of my lips.

21

Is this created by a human, or by computer?

Open the kingdom (Liquid days, Part II) by David Byrne

Days of Fishes Distant RoarTurning to SpeakTurning to HearOpen the Kingdom Open the Kingdom Open the Kingdom Open the Kingdom…

In my way In my wayBeing most uncertain And This Remains

Still for betterBirds of VoicesThe Field of Living

Returning Love Returning With LoveThen it was Written with Love

I am Asking

I am Asking

I am Asking

22

Is this created by a human, or by computer?

Angel A poem written by Ray Kurzweil’s Cybernetic Poet afterReading poems by Patricia Camarena Rose and Sue Klapes Kelly

Beams of the dawn at the angel with a calm, silent seawith a hundred times we write, with a chance we can open upa steady rhythm in his face silent roomdesolate beach, Scattering remains of love.

23

Example of computer-generated visual art: Aaron

24

Computer and Aesthetics

• In 1960s, a number of Vermeer paintings were analyzed by X-rays.

• Some paintings were found to be different.

• Together with the historical background, art historians concluded that those paintings were done by an apprentice of Vermeer.

• After that, even art students could distinguish a fake Vermeer from a real Vermeer by naked eyes.

25

Discover new aesthetic feature using computer so that the feature can be appreciated intuitively!

• For example, discover new literary features from computer analysis:– Compare different authors, same period– Compare same period, different locations– Compare same author, different periods.– And so on…

Computer and Aesthetics

26

Computer and Creativity: Conclusions

Computer is like a distorting mirror!

• Computer shows the user their own ideas but rearranged in new ways:– Reflecting on these new arrangements stimulates

user’s creativity– Sometime these rearrangements reveal hidden features

of the objects to the user, which may play a major role in human cognition.

Let us explore our world deeper using computer!

27

Modeling Feature Creation : Some Issues

• Where do created features come from?

• What constrains the creation of features?

• Are the features created or discovered?

• Why is feature creation significant in cognition?

• Can feature creation be modeled computationally?

28

Outline of an Interaction View

How is an object (or situation) represented (or described)?

Cognitive Agent interaction Object or Situation

Representation (Description)

• The representation is determined in part by the agent– what are the primitives of representation

• The representation is determined in part by the object– how are the primitives configured in the representation

29

A Multi-layer Model of Cognition

Representation-building process is mediated by several intermediate-level perceptual/conceptual units (features)

For simplicity, we focus on two layers!

conceptual layer

perceptual layer

accommodation projection

conceptual representation

perceptual/imagery/episodic data

(top down)(bottom up)

conceptualmemory

perceptual/episodicmemory

sensory stimuli

conceptual representation

30

Ontology vs. Structure

• Ontology: building blocks of representation• Structure: a certain configuration of building blocks

line1line2

line3line4

line6

line5line7

square1 triangle1

on-top-of

layer N

layer N+1

layer N-1

31

‘Mind-independent Object’ Thesis

In representation-building, a cognitive agent may not choose both the ontology and the structure!

cognitive agent object/situation

chooses ontology constrains possible structures

chooses structure constrains possible ontologies

(accommodation)

(projection)

32

Projection and Accommodation

Two mechanisms that work together in building representations!

conceptual layer

perceptual layer

accommodation projection

structural features induced by ontology

ontology induced by structure

structural features

ontological features

33

Conceptualization Without Perception

• Perceptual layer has its own memory, which reflects prior perceptual experiences with the object!

• Words activate concepts in the conceptual layer.

• These concepts evoke a perception-like image of the object (derived from perceptual memory).

• This imagery constrains possible ontologies and structures that could be assigned to the object.

34

Feature Creation by Projection

Projected structure causes a new ontology to be created!

conceptual layer

reflect-horizontal

reflect-vertical

join

join

perceptual layer

projection

emergent feature induced by

structure

structural features

35

Modeling ‘Juxtaposition of dissimilar’

conceptual layer

perceptual layer

projection

source (vehicle)

target (topic)

target (topic)

36

Feature Creation by Accommodation

New structural features are induced by the given ontology!

A B C D

category X category Y

accommodation

structural features induced by ontology

ontological features

size Š sizeB size � sizeC

sizeA sizeB sizeC sizeD

conceptual layer

perceptual layer

37

Modeling ‘Deconceptualization’

perceptual layer

conceptual layer

source (vehicle)

target (topic)

target (topic)

38

Some Implications of the Proposed View

• Where do created features come from?• The cognitive agent chooses to act upon an object in a

certain way.• The object responds to the action depending on its

autonomous nature.• In comprehending the results of its action, the agent may

see new features in the object.• Emergent features depend both on the agent and on the

object.

(An object can have a large number of potential features!)

39

How is the Creation of Features Constrained?

because of the ‘mind-independent object’ thesis!

• Though the agent may choose to project any structure on the object

• How the object responds to the projected structure depends on its autonomous nature.

(The object does not accept any arbitrary feature!)

40

Are the Features ‘Created’ or ‘Discovered’?

Unrealistic to assume that all possible featuresan object can take already exist!

(requires the God’s eye view of all features,from all objects: past, present, and future!)

Take the cognitive agent’s point of view: the emergent feature seems new

(it was not seen before).(the feature is created in the cognitive agent’s

mental representation of the object.)

41

Why is Feature-Creation Significant in Cognition?

• Any conceptualization involves a loss of information:– objects under the same category: differences are lost– objects under different categories: similarities are lost

• Feature-creation mechanisms make it possible to recover (partially) this lost information:– for aesthetic pleasure: in art and poetry– for making new artifacts: in science and technology– for achieving some desired situation: in problem-solving

42

Can Feature-Creation be Computationally Modeled?

• Some existing systems can do projection and create features:– Machine vision: if a part of the image is identified as an airport,

then features of building are projected onto neighboring images.– Speech recognition: if the topic of conversation is identified,

related features are projected onto the sound pattern.

• Models specifically devoted to representation-building:– Modeling creative analogies: Hofstadter and colleagues (1995)– Modeling creativity in legal reasoning (Indurkhya 1997)