From Semiotics to Computational Semiotics

State University of Campinas UNICAMP - Brazil

Ricardo Gudwin

Semiotics and Computational Semiotics

Semiotics branch of human sciences, that studies the sciences of signification

and representation, involving mainly the phenomena of cognition and communication on living systems.

Intelligent systems systems that exhibits behavior that can be considered intelligent some of the objectives are the study of the phenomena of

cognition and communication, but now explicitly under the scope of artificial systems

Computational semiotics proposition of a set of methodologies that in some way try to use

the concepts and terminology of semiotics, but composing a framework suitable to be used in the construction of artificial systems, in this case, implementable within computers

Computational Semiotics

Computational semiotics new inborn science, but, there are currently some important contributions that

despite still not complete and definitive, help us in understanding the nature of semiotic processes and allow their synthesis and implementation within computational platforms.

In this work we explore one possible pathway along a set of ideas

evolving around, which proposes one way of gathering the transition between traditional semiotics and computational semiotics, making it possible to synthesize semiotic systems by means of artificial computing devices.

Semiotic Analysis

Semiotics Tool of Analysis - main goal is to understand the semiotic

processing happening in nature Semiotic Beings (interpreters) are already there

living organisms (biosemiotics) human beings (human semiotics)

it is easier to create concepts and apply them to things that do already exist and are already working

Questions should it be possible to use the same conceptual background

in order to synthesize new beings (systems), performing the same semiotic behavior as living/human beings would do ?

What would be the challenges in such an endeavor ?

Semiotic Synthesis

Problem things are not already working so, we should put them to work !

Hidden Problems specify the basic entities involved within semiosis

in a way in which it can be produced within a computer specify the mechanism by which signs are interpreted

there are a lot of intermediary steps that are generally not considered within the context of human semiotics

• How from a scene given by a video camera we discover the objects involved into this scene ?

• Can we talk about signs if the system is still not aware of objects ? Are computational devices able to carry on all sign-processing

that living/human beings are able to perform ?

Semiotic Synthesis

Basic Foundations set up a generic scenario in which semiotic synthesis is going

to be discussed try to get clues on how semiotic processes really happens allow the implementation of a computational version of

semiotic processes Terminology

related to standard semiotic terminology but we don’t want to limit the meaning of terms to human/bio

semiotics Requirement

be careful when applying semiotic analysis to our synthesis scenario

Semiotic SynthesisBasic Foundations

Representation Spaces

INTERPRETER

EXTERNALSPACE

INTERNALSPACE

EXTERNALFOCUS OF

ATTENTION

INTERNAL FOCUSOF ATTENTION

Semiotic SynthesisBasic Foundations

Shareable and Non-shareable spaces

EXTERNALSPACE

(SHAREABLE)

FOCUS OFATTENTION

INTERNALSPACES

(NON-SHAREABLE)

Semiotic SynthesisBasic Foundations

Interpreting Fields

INTERPRETER

EXTERNALSPACE

INTERNALINTERPRETINGFIELD I (x,y,z,t)

(UMWELT)

EXTERNALINTERPRETINGFIELD E (x,y,z,t)

Semiotic SynthesisBasic Foundations

Multiple Internal Spaces and Interpreting Fields

CONCRETESPACE

ABSTRACTSPACE

ABSTRACTSPACE

Semiotic SynthesisBasic Foundations

Interpreting Field concept originated from field theory function (energy function ?) that to each point in space and

time determine a unique value state

External Space interpreting field is continuous (that’s the real world) by definition, is not knowable in its entirety

Internal Spaces accommodate a model of external interpreting field internal interpreting fields are functions that depend on the

type of semiotic synthesis we are trying to model

Semiotic SynthesisBasic Foundations

Sign Everything under the interpreter’s focus of attention (internal

or external) that would cause an interpreter action Interpreter Possible Actions

Change in the focuses of attention (internal and/or external) Determination, for the time t = t+1 of a new value for any

interpreting field (internal or external), at a point (x,y,z) covered by the focus of attention in that space

Interpretant any interpreter action caused by a sign any change in internal and external interpreting fields for time

t = t+1, caused by an interpreter action due to the effect of the sign

External Semiosis

Interpretant of signs happens at the external space

Change in external interpreting field change in environment shareable with other interpreters can act as a sign for the same interpreter or to other

interpreters Happens mainly on interpreters that do not have

internal spaces semiosis in molecules and chemical reactions very simple biological organisms

Can be the final result of a chain of internal semiosis

Internal Semiosis

Interpretant of Signs happens within any of the internal spaces

Signs can be at the external space (semiotic transduction) at the internal space

A typical semiosis chain starts with an external sign generates a set of internal interpretants, that become internal signs generating new internal interpretants, until some of them become an internal sign that generates an external interpretant

Information, Signs and Knowledge

Signals and Information signals - values of parts of interpreting fields that can be

differentiated (distinguished) from other values information - meaning of signals

Example suppose that E (x,y,z,t) has a counter-domain like [0,5] but, due to sensor limitations, the interpreter is only able to sense

values in {0,1,2,3,4,5} then, values like 2.3 or 2.2 would equally be understand like 2 so, the information that those signals convey is tied to only 6

discrete values

Signals only describe states they do not cause any actions

Information, Signs and Knowledge

Once signals are able to cause actions they become signs

The information they carry associated to the actions they cause is then called knowledge

Signals - Information Signs - Knowledge Region under a focus of attention of some space

sign knowledge unit

Things to Remember

External Interpreting Field is infinite, continuous and probably take values on continuous

sets can not be known as a whole can be known in parts, with approximations

The only way we are able to know the external interpreting field is due to sensors

The most basic knowledge units that can be stored into internal interpreting field is of sensorial type

Internal Interpreting Field (Concrete Space) our best model of external interpreting field

Things to Think About

Storing sensorial information is not efficient We need better models

Basic mechanism The notion of “Entity”

From sensorial knowledge units the system must try to represent the same external interpreting

field as a collection of entities Entities

may have attributes that would change in time Occurrences

model the change in entities attributes Sensorial knowledge, entities and occurrences

grouped to represent situations

A Hierarchy for Knowledge Units

Indexical

Iconic

Symbolic

Symbolic

Iconic

Knowledge Unit

ArgumentativeDicentRhematic

GenericSpecific

Entities

GenericSpecific

Occurrence

Generic

Specific

Sensation

Analytic Synthetic

AbductiveDeductive

Inductive

Simplification of the Model

Instead working in general spaces and interpreting fields restrict ourselves to memories and places assign sign processing to micro-interpreters

KNOWLEDGE UNITS

SPACE

PLACE

Micro-Interpreter

Micro-Interpreter’s Responsibilities choose the knowledge units that will be used (focus of

attention) eventually destroy them after use create new knowledge units using information contained in

earlier ones

Interpreter

Knowledge Units

Building Intelligent Systems

Using multiple micro-interpreters in cooperation to

each other processing

knowledge units

Conclusions

Dyadic or Triadic ? Some people would say that we are building a dyadic model for a

sign But, there is some kind of “mediation”, due to:

the focus of attention mechanism the influence that some knowledge units may have over the processing

of other knowledge units (catalytic knowledge units) We still need to do further reflections in order to have a better

position on this issue This is not the final word regarding Computational Semiotics

it is only a first exercise in order to get insights to the problem of semiotic synthesis

computational implementations of such model indicate that, up to some point, it is worth the value of working on it