More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The...

22
More than Words: The Generative Power of Transmediation for Learning Author(s): Marjorie Siegel Source: Canadian Journal of Education / Revue canadienne de l'éducation, Vol. 20, No. 4, Cultural Psychology and Semiotics (Autumn, 1995), pp. 455-475 Published by: Canadian Society for the Study of Education Stable URL: http://www.jstor.org/stable/1495082 Accessed: 24/03/2010 17:19 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=csse. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Canadian Society for the Study of Education is collaborating with JSTOR to digitize, preserve and extend access to Canadian Journal of Education / Revue canadienne de l'éducation. http://www.jstor.org

Transcript of More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The...

Page 1: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

More than Words: The Generative Power of Transmediation for LearningAuthor(s): Marjorie SiegelSource: Canadian Journal of Education / Revue canadienne de l'éducation, Vol. 20, No. 4,Cultural Psychology and Semiotics (Autumn, 1995), pp. 455-475Published by: Canadian Society for the Study of EducationStable URL: http://www.jstor.org/stable/1495082Accessed: 24/03/2010 17:19

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/action/showPublisher?publisherCode=csse.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

Canadian Society for the Study of Education is collaborating with JSTOR to digitize, preserve and extendaccess to Canadian Journal of Education / Revue canadienne de l'éducation.

http://www.jstor.org

Page 2: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

More Than Words: The Generative Power of Transmediation for Learning

Marjorie Siegel teachers college, columbia university

The emerging shift from transmission- to inquiry-oriented models of teaching and learning implies that students need more than words to learn. Transmediation, the act of translating meanings from one sign system to another, increases students' opportunities to engage in generative and reflective thinking because learners must invent a connection between the two sign systems, as the connection does not exist a priori. In this article I explore the semiotic basis for this claim and illustrate the generative potential of transmediation.

L'6mergence d'une nouvelle orientation dans le domaine de l'enseignement et de l'ap- prentissage-de la transmission du savoir h des m6thodes qui privil6gient la recherche - implique que les mots ne sont plus, A eux seuls, l'outil d'apprentissage par excellence. La transmEdiation, c'est-A-dire la traduction de significations A partir d'un syst6me de signes vers un autre, multiplie pour les ?l\ves les occasions de faire preuve d'une pensee creative et rEflexive, car ils doivent inventer un lien entre deux syst~mes de signes, le lien n'existant pas a priori. L'auteure se penche sur le fondement s6miotique de cette thdorie et illustre le potentiel cr6atif de la transm6diation.

In recent years, the well-worn image of classrooms as places where teachers talk and students listen, memorize, practice, and display knowledge has begun to fade as educators recognize that there is more to teaching and learning than words. Scholars working in a variety of disciplines have argued that teaching and learning cannot be reduced to the transmission of information from expert to novice through language as this model is rooted in assumptions about knowledge and learning that no longer find full support in the education community. Whereas knowledge was once regarded as stable and absolute, it is now seen as partial and contingent; similarly, the familiar idea that learning is a passive process of acquiring isolated skills and bits of information has given way to the idea of learning as a social process in which students actively construct understandings.

Building on these new assumptions, educators have proposed enquiry as a powerful way to conceptualize the teaching/learning process (e.g., Borasi, 1992; Borasi & Siegel, 1994; Leland & Harste, 1994; Richards, 1991; Siegel & Borasi, 1994; Watson, Burke, & Harste, 1989). Enquiry models invite learners to see themselves as knowledge makers who find and frame problems worth pursuing, negotiate interpretations, forge new connections, and represent meanings in new ways. Unlike the instructional routines associated with the transmission model,

CANADIAN JOURNAL OF EDUCATION 20:4 (1995) 455

Page 3: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

which have led students to believe there is no ambiguity in learning, no risks to be taken, no new knowledge to be made, enquiry models give a central place to instructional practices and strategies that encourage generative and reflective thinking. In this article I explore the idea that learners need more than words to engage in such thinking. Instructional strategies involving transmediation, the process of translating meanings from one sign system (such as language) into another (such as pictorial representation), are critical to enquiry-oriented classrooms because they promote the kind of thinking that goes beyond the display of received meanings to the invention of new connections and meanings.

My purpose here is to show how transmediation achieves its generative potential, drawing on semiotic theory as well as on data from several research projects that incorporated transmediation to do so (Borasi & Siegel, 1988; Siegel, 1984). I begin by discussing schools' bias toward language, then introduce two semiotic concepts basic to understanding transmediation. I go on to define transmediation, analyze it in terms of these concepts, and illustrate it with classroom examples. I conclude by looking at the links between transmediation and metaphor. As my focus shifts from the single sign to crossing sign systems and ultimately to metaphor, I hope to show: (a) that transmediation highlights and intensifies ile generative nature of meaning-making, and (b) that the more familiar concept of metaphor offers a way of talking and thinking about transmediation that calls attention to the need for students to invent their own connections, ask their own questions, and in doing so, open new lines of thinking in school.

THE PROBLEM OF VERBOCENTRISM IN SCHOOLS

Immersed in a world of words of our own making, we have come to regard our reliance on language as natural and, in doing so, fail to recognize that there are multiple ways of knowing - music, dance, visual arts, and so on - each of which offers a distinctive way of making meaning (Langer, 1942). Although some have argued that what makes us human is our ability to use language to organize experience, others suggest that our capacity to create and use symbols of any kind is our most human trait (Percy, 1982). In schools, this verbocentric (Eco, 1976) ideology has led us to regard language as the sole channel for learning and to separate it from other ways of knowing. The privileged status accorded language over images, music, and movement is evident in our curriculum guides, instructional methods and materials, evaluation practices, schedules, and the like.

However central language may be to human activities, some evidence suggests that the cultural tendency toward verbocentrism in schools is limiting for students, as it reinforces the transmission model. The work of Goodlad (1984), for example, has shown that the reduction of teaching to telling, so common in U.S. classrooms, positions students as passive learners. Classes in the arts were

456

Page 4: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

found to be an exception to this trend. In these settings, students did not learn solely through talk but drew, painted, sang, acted, and danced, thus experiencing for themselves what it means to create knowledge and meanings through different modes of representation. Further evidence that children need more than words to learn can be found in the research literature on young children's literacy learning. Unlike older students, schooled to adhere to the conventional boundaries between sign systems, young children turn reading and writing into multimodal events involving drawing, talking, singing, writing, and so on.

Weaving together symbols of all kinds to represent and convey their meanings makes it possible for them successfully to orchestrate literacy events long before language alone can serve them (Dyson, 1986; Fueyo, 1992; Gallas, 1994; Harste, Woodward, & Burke, 1984; Hubbard, 1989). When these findings are considered in light of the need for educational environments that support students as makers of knowledge and meaning, it seems clear that we must re-examine our bias toward language in teaching-learning and consider curricular possibilities that do not marginalize other ways of knowing.

SIGN SYSTEMS AND MEANING-MAKING IN A SEMIOTIC PERSPECTIVE

To suggest that meaning can be made through sign systems other than language is to take the semiotic turn. Semiotics, a broad field of studies that looks at "meanings and messages in all their forms and all their contexts" (Innis, 1985, p. vii), is uniquely suited to understanding transmediation because it examines how all kinds of signs, not just linguistic signs, function. Although two names, Charles Sanders Peirce (1839-1914) and Ferdinand de Saussure (1857-1913), are most closely associated with the development of semiotic thought in modern times, my argument draws primarily on Peirce and his interpreters because of his wider view of sign functioning. Unlike Saussure, who took language as the model for all sign functioning, Peirce did not limit his focus to linguistic signs; instead, he included signs that signify an object by virtue of resemblance (icons) or physical connection (indices), as well as those signs based on a cultural convention which has become a rule or habit (symbols). Two aspects of Peirce's thinking have particular relevance for understanding transmediation. First, although Peirce's well-known trichotomy of index, icon, and symbol is not a characterization of different sign systems, the distinction between icons and symbols hints at the problem presented by crossing sign systems, at least when moving from language to some form of visual representation. Second, Peirce's description of how signs become meaningful, which he termed "semiosis," suggests that sign functioning always involves an enlargement and expansion of meaning, not a simple substitution of one thing for another. I shall develop each of these points further as a preface to analyzing transmediation.

Iconicity has sparked much debate among semioticians, centring on the "motivated" nature of the sign and the question of whether it is possible for icons

457

Page 5: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

to communicate an idea directly, unmediated by cultural conventions (see Eco [1976] for a detailed discussion of the issues surrounding iconicity, including his claim that all icons are mediated by culture). Peirce included images, diagrams, algebraic expressions, and metaphors under the heading of iconic signs, suggesting that in each case the representation was based on similarity or analogy, rather than on a cultural convention as in language. At the same time, Peirce did not treat icons as motivated purely by the object represented but recognized that iconic representation is culturally mediated. The mediated character of iconic signs is evident when one looks at any set of pictures designed for international communication, such as those designating Olympic sports. Although these pictures were designed to be interpreted without reference to shared cultural knowledge and conventions, it is clearly impossible to interpret them without such knowledge. Still, however much iconic signs may depend on cultural conventions to signify, they function in ways distinctly different from language, as Susanne Langer's work on discursive and presentational forms has shown (cf. Innis, 1985). Langer (1942) argues that the chief difference between language and non-verbal representations, such as pictures, lies in the form each takes and not in the fact that the former involves thought and the latter feeling, an idea she rejects. Language, she suggests, is discursive in that a set of discrete units (i.e., words) is arranged in a linear string, whereas the interpretation of a picture depends on the simultaneous rather than successive arrangement of elements that have no meaning in and of themselves (i.e., gradations in shading and line cannot be interpreted apart from the pictorial context). She develops this contrast as follows:

The meanings given through language are successively understood, and gathered into a whole by a process called discourse; the meanings of all other symbolic elements that compose a larger, articulate symbol are understood only through the meaning of the whole, through their relations within the total structure. Their very functioning as symbols depends on the fact that they are involved in a simultaneous, integral presentation. (p. 97)

So, although it is important to keep in mind that icons and symbols - images and words--both depend to some extent on cultural conventions to signify, it is clear that making meaning through drawing is a very different undertaking than making meaning through language. Each sign system is based on a unique organizational principle and involves elements that have no ready equivalents in the other sign system, despite the tendency to speak of the "languages of art."

Transmediation involves this very question of how to "translate" from one sign system to another, given the differences that Langer (1942) outlines. Peirce's discussion of sign functioning or "semiosis," as he called it, has direct bearing on this problem for, as the renowned linguist Roman Jakobson (1960) has noted, "One of the most felicitous, brilliant ideas which general linguistics and semiotics gained from the American thinker is his definition of meaning as 'the translation

458

Page 6: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

of a sign into another system of signs"' (p. 251). This idea is evident in Peirce's explanation of semiosis:

A sign, or representamen, is something that stands to somebody for something in some respect or capacity. It addresses somebody, that is, it creates in the mind of that person an equivalent sign, or perhaps a more developed sign. That sign which it creates I call the interpretant of the first sign. The sign stands for something, its object. It stands for that object, not in all respects, but in reference to a sort of idea, which I have sometimes called the ground of the representamen. (cited in Eco, 1979, p. 180)

The first thing to clarify about this passage is Peirce's distinctive terminology. Peirce used the term "representamen" to designate the sign-vehicle and "object" to denote the thing being represented, that "thing" being a cultural construct rather than a referent since Peirce argued that all knowledge of the world was mediated by signs. Starting with de Saussure, most definitions of a sign involve only these two components so that a sign becomes a dyadic relationship between signifier-signified or symbol-concept. But for Peirce, a sign does not simply stand for an object, it tells something about the meaning of that relationship, and this requires a third component, which he called an "interpretant." The interpretant is another sign that represents the same object as the representamen, as its position in the semiotic triad indicates (see Figure 1).

Buczynska-Garewicz (1981) has explored the ambiguity arising in Peirce's writings on the interpretant, and argues that the vagueness associated with the mediating role of an interpretant had purpose. By suggesting that the interpretant is both "a meaning of a sign and also another sign explaining the former one" (p. 192), she explains, Peirce showed that meaning always involved interpreta- tion, and, moreover, that there were no single signs but rather every sign was connected to other signs. The idea that meaning is created when one sign is translated into another thus sets up the possibility of a "chain of interpretation" (Buczynska-Garewicz, 1981, p. 188) or "unlimited semiosis" (Eco, 1976, p. 68) from which an entire semiotic system may be constructed.

Interpretant

Object Representamen

FIGURE 1

Peirce's Semiotic Triad

459

Page 7: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

Peirce's characterization of a sign as a triadic relation that always stands in relation to other signs shows that generativity lies at the core of meaning-making, as Eco (1984) notes in the following passage:

Thus substitution ... is not the only necessary condition for a sign: the possibility of interpretation is necessary as well. By interpretation ... we mean the concept elaborated by Peirce, according to which every interpretant, besides translating the . content of the sign, also increases our understanding of it. (p. 43)

Here Eco refers to an important passage in Peirce's writing stating in effect that "a sign is something by knowing which we know something more" (Hardwick, 1977, p. 31). The "something more" that we know is provided by the interpret- ant, which, as noted above, brings the object and representamen in relation to another sign, and in so doing, sets in motion an unending process of translation and interpretation. This is why Eco (1984) concludes that "the sign always opens up something new. No interpretant, in adjusting the sign interpreted, fails to change its borders to some degree" (p. 44). In the discussion of transmediation that follows, I show that when the interpretant is from another sign system with a distinctive way of representing meaning, the borders of meaning are "opened up" even further.

UNDERSTANDING TRANSMEDIATION

The two semiotic concepts examined above- the distinctiveness of different sign systems and the generative power of sign functioning-are critical to under- standing transmediation. The term "transmediation" was first introduced by Charles Suhor (1984) as part of his development of a semiotics-based curriculum. Suhor, a language educator interested in integrating media and the arts across the curriculum, defined transmediation as the "translation of content from one sign system into another" (p. 250) and characterized it as a syntactic concept since it deals with the structure of sign systems and the relationship between different sign systems. As such, transmediation is grounded in the idea, presented earlier, that alternative sign systems (linguistic, gestural, pictorial, musical, constructive, and so on) are available for making sense of human experience, though his model not only elevates the linguistic system above all others to show that language is the most powerful sign system but connects language to all other sign systems, suggesting that language nearly always accompanies meanings constructed through alternative modes.

In the classroom examples Suhor (1984) presents, however, he is careful not to analyze the drawings, collages, musical ballads, and films high school students produced after reading Steinbeck's Of Mice and Men in terms of language alone, but considers the distinctive mode of expression each sign system permits. Based on these examples, Suhor concludes that instructional experiences requiring

460

Page 8: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

transmediation, such as writing about pictures, creating book reviews through collage, and role-playing based on the theme of a story, may foster development of a wide range of cognitive, aesthetic, and psychomotor skills which remain untapped in most traditional classrooms.

Implicit in Suhor's definition of transmediation as translation and his discussion of its potential for enhancing students' learning is the idea that moving across sign systems is a generative process in which new meanings are produced. This idea can be better understood if transmediation is conceptualized as a special case of semiosis in the sense that learners use one sign system to mediate another. When a learner moves from one sign system to another, semiosis becomes even more complex in that an entire semiotic triad serves as the object of another triad and the interpretant for this new triad must be represented in the new sign system.

At this point it may be helpful to shift to more straightforward terminology. The work of Umberto Eco (1976) can be of service here. Eco suggests that sign functioning involves the correlation of an expression plane (the sign-vehicle, or representamen in Peirce's terms) to a content plane (the object). Although this definition sounds more like Saussure's signifier-signified dyad than Peirce's semiotic triad, Eco's interpretation of this correlation is firmly rooted in a Peircean perspective which emphasizes the role of the interpretant. The advantage of talking about sign functioning as a correlation of content and expression planes is that it can help sort out the complexities of using two different sign systems to make meaning. In transmediation, the learner does not simply correlate a content and an expression plane, but takes the interpretant arising from that correlation and maps it onto the expression plane of a new sign system.

For example, consider what happens when learners draw their interpretations of a written text, whether a story or an expository piece. They must arrive at some understanding and then find some way to cross ("trans") the boundaries between language and art such that their understanding is represented pictorially; it is in this sense that one sign system is explored in terms of (mediation) another. The drawing one fourth-grader produced in the course of an ethno- graphic study of reader/text transactions can provide an preliminary look at transmediation. The study was carried out over a seven-month period in one Grade 4 classroom and involved the introduction of an instructional strategy called "sketch to stretch" (Harste & Short, with Burke, 1988) which invited children to choose a text from a selection of either stories or informational pieces, read it silently, draw their interpretations, and, finally, explain their drawings to their group (see Siegel [1984] for a complete report of the study).

After reading the picture-story book Nana upstairs, Nana downstairs by Tomie de Paola (1973), Jean wrote the word "love" in big capital letters across the middle of her paper; then she carefully outlined each letter with a red crayon and

461

Page 9: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

filled it in with a pink crayon. Above and below the word, Jean drew a heart and coloured each the same way as the letters. She chose a blue crayon to create the blue sky that surrounded the other images she had drawn and a yellow crayon to draw two falling stars in the sky (in the book, the little boy saw a falling star after each grandmother died). When asked to explain her sketch to a group of her classmates, Jean said: "This story was about a boy-his name was Tommy- and his love for his two grandmothers which he called Nana Upstairs and Nana Downstairs. I just drew a picture of that. So I just made two hearts and the word 'love."' In this example Jean found a way to translate or map her understanding of the story onto the expression plane of another sign system by weaving toge- ther a variety of signs. She used a common cultural icon (a heart) to signify love; she drew two of them to indicate the loss of two grandmothers (one upstairs and one downstairs), another form of iconicity; and, she used the linguistic symbol "love" and filled it in with pink crayon, a creative way to deal with the ambigu- ity of the process and straddle the boundaries between these two sign systems.

If we think of transmediation as a process of translating or mapping the content of one sign system onto the expression plane of another, then Roman Jakobson's communication model, and, in particular, his explanation of the poetic function of language can offer insights into how this mapping is accomplished. After de Saussure, Jakobson (1960) identified two modes of arrangement in

language: the syntagmatic and the paradigmatic, each of which may be thought of as an axis rotating in space. Elements are arranged horizontally by combina- tion on the syntagmatic axis whereas along the paradigmatic axis elements are arranged vertically, since the elements belong to the same class and can be substituted for one another. This construct allowed Jakobson to propose an hypothesis to explain the poetic function of communication, namely, "The poetic function projects the principle of equivalence from the axis of selection [the paradigmatic axis] into the axis of combination [the syntagmatic axis]" (p. 358). His claim, in other words, is that a message becomes poetic or aesthetic when the principle guiding the selection of elements on the vertical axis is applied to the selection of elements along the horizontal axis.

The example Jakobson used to illustrate this point was Eisenhower's campaign slogan "I like Ike." One reason this phrase works so well as a slogan is that equivalence is the basis for both selecting and combining elements of the sentence. Jakobson argues that this rotation of the axes directs our attention to the message because the ordinary is made strange. Usually, each axis is organized in terms of a different constitutive principle; the paradigmatic axis is based on the principle of equivalence (one element can substitute for another) whereas the syntagmatic axis is based on contiguity (elements are combined in particular ways). But when a poetic message is created both axes are guided by the principle of equivalence. If we look again at the campaign slogan, our attention is drawn to the use of three rhyming words to form a sentence; although one could choose any number of other words to express a similar

462

Page 10: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

feeling about Eisenhower, this particular arrangement makes the ordinary strange. Its effectiveness as a campaign slogan, then, derives from its ability not only to convey the message but to almost force the reader to stop and pay attention to that message, at least the first time it is encountered. In Jakobson's terms, the axis of selection melds into the axis of combination, creating a text that is ambiguous, and, hence, self-focusing (cf. Eco, 1976).

Transmediation involves a process not unlike the one Jakobson associates with the poetic function of communication. Learners must rotate the content and expression planes of two different sign systems such that the expression plane of the new sign system conveys the content of the initial sign system. But because the expression plane is that of another sign system, the connection between the two sign systems must be invented, as it does not exist prior to the act of transmediation itself. This is how transmediation achieves its generative power. The absence of a ready-made link between the content and expression planes of two different sign systems creates an anomaly that sets generative thinking in motion. Returning to Jean's drawing, we could say that she connected her interpretation of the story - the theme of love - to the new expression plane by arranging various icons and symbols into a "graph," that is, a spatial organization of meanings that are not themselves spatial (Eco, 1976). This was

just one way the children in this study invented to link language and pictorial representation. Some drew a single character or illustrated a particular scene from a story, others created cartoon-like pictures showing their own version of the story, and still others divided their paper into little boxes and retold the entire story. Yet these inventions did not arise out of nothing, but, rather, were built from conventions (Eco, 1976). These conventions came from a variety of places, including the culture of elementary school, where requests to "draw your favourite story character" or "retell the plot of a story by dividing your paper into four boxes" are frequently heard, as well as the larger cultural scene (including popular culture), as is evident from the abundant use of cartoons, verbal labels for pictures, keys for "decoding" certain images, and familiar icons (hearts, arrows, rainbows) in their "sketches."

Even when students produced what seemed to be a literal translation of a

particular text, as when children retold the plot of a story through a series of little scenes in separate boxes, we must appreciate the fact that they had to invent this connection, as it was not given to them a priori. As a result, their experience with transmediation involved both reflective and generative thinking. I do not mean to suggest that this kind of generativity cannot be achieved when making meaning in a single sign system. The recent emphasis on talking as a mode of learning is an example of an instructional practice grounded in the belief that learners must actively transform the text to make it their own (Hynds & Rubin, 1990), a practice consistent with Peirce's notion of the interpretant. This use of language for instruction is quite different, however, from the kind typically found

463

Page 11: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

in transmission classrooms, where ready-made meanings are supplied with the

expectation that they be reproduced at a later time as an indication of learning. The most extreme examples of this are the use of the recitation mode of classroom discourse (Cazden, 1988) and the use of so-called comprehension questions that can be answered simply by finding the appropriate phrase or sentence in the text and rearranging it to fit the question stem. If, as Peirce and Eco suggest, the interpretant makes the translation from one sign to another a generative act in the sense that it opens up something new, then the translation from one sign system to another amplifies this possibility, since there is no dictionary that tells how to represent language in images.

The two examples presented below clearly illustrate the generative and reflective nature of transmediation. The examples were also selected because they show that the strategy is as powerful for high school as for elementary students, and may be especially helpful for students who have not experienced much success in school. The first vignette is drawn from a collaborative research project, "Reading to Learn Mathematics," which explored the role reading might play in learning mathematics; the second is drawn from the ethnographic study of reader/text transaction mentioned earlier.

Example 1: Sketching Is Like a Giant Key

An interdisciplinary effort designed by a mathematics educator, a reading educator, and a group of secondary mathematics teachers, the "Reading to Learn Mathematics" study sought to develop, document, and analyze a number of classroom experiences in which reading, along with other activities, was used to support students' mathematical learning (Borasi & Siegel, 1988, 1994; Siegel & Borasi, 1994). My colleagues and I wanted to understand the role reading might play in creating opportunities for students to move beyond a curriculum of techniques (Bishop, 1988) and experience the complexity of approaching, generating, framing, representing, and solving mathematical problems. In particular, we focused on how reading strategies that supported active, generative reading could help students understand "rich" mathematical texts (e.g., stories, historical essays, cartoons) that had been integrated into various classroom experiences. The example which follows took place in the context of a semester-long course taught by Judith M. Fonzi at an urban alternative high school. Unlike most high school mathematics classes, which focus on learning technical content, this class explored mathematical connections to everyday life in an attempt to challenge students' narrow and dysfunctional conceptions of mathematics. About halfway through the course, various reading strategies were introduced as a way to help students share information with their peers, something they had identified as a problem during the course of their group enquiry into the connections between mathematics and racing. The students were

464

Page 12: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

asked to pair up and choose one of two short articles, "Mathematics in the Marketplace" or "Mathematics and War" from The Mathematical Experience (Davis & Hersh, 1981), a book exploring the nature of mathematics, including its social, historical, and political dimensions. The directions to each pair of students were to read the text using a strategy called "say something" (Harste & Short, with Burke, 1988) and then produce a single sketch showing what they made of the piece.

The "say something" strategy is supposed to support readers' meaning-making by inviting them to talk their way through a piece, sharing their responses, questions, confusions, and insights with a partner as they go. But Tim and Eric did not find it very helpful in making sense of the article they had chosen, "Mathematics in the Marketplace." They felt frustrated by the article's sophisticated language and conceptual density and found it difficult to engage with the text as they read. Their decision to stop and "say something" after each page rather than each paragraph (the article had three pages of text and a page-long figure) might have added to their difficulties, as it limited the opportunities to stop and work out an interpretation of the material. Because they moved quickly through the article, they had time to finish their joint sketch (see Figure 2) during the remainder of the class period. When they had a chance to explain their sketch to the class at the next meeting, it became evident that the act of moving across sign systems helped them construct an interpretation that made sense to them. Although Tim began the discussion of their sketch, Eric was soon finishing his sentences, creating the sense of one voice.

FIGURE 2

Tim and Eric's Sketch of "Mathematics in the Marketplace"

465

Page 13: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

Tim: See, math is in the centre [of the star]. We did a star. And see, everything here is connected to everything else through mathematics. So, like, you know, science is connected to business and economics through math. And ah-you know, business and economics is connected through mathematics to mechanics - like, you know, machines and stuff-because, you know-business needs machines to produce-

Eric: - their products. Tim: So they can produce their products faster, which is math, which is- Eric: making money.

Judi asked them to explain what they had written along the outer circle connecting the points of the star. Tim started, got confused, and let Eric try his hand at explaining their thinking.

Eric: Okay, it was just a way to make it easier. Because there was the main topic -like life connected with social studies through math and we just put, like, life and connected to business by-

Tim: Because it takes up your life-it takes time to go to work every day, you know. Eric: It was just another subtopic to connect- Tim: And then business hooks into social studies because, you know, you have an

income and then- Eric: Social studies connects to science. Tim: And social studies connects to science because of the demands, you know, by

people wanting better products- things that last longer. Eric: Yeah, and then science to machines, I mean, to mechanics through machines- Tim: Yeah, because machines, you know, science develops machines for mech-

anics machines for businesses and stuff.

Finally, Eric summed up the organization of their sketch.

Eric: It's just that the star connects the major stuff together [through mathematics] and then the circle connects the major [ideas] to major [ideas] -

Tim: Without the interference of mathematics.

Their interpretation of the article thus had two dimensions, one focusing on the central role of mathematics in various enterprises and another focusing on the way social forces affect these same enterprises. When Judi commented on the star-shape and how it showed that someone could start at one point and go through math to get to the other things, Tim spontaneously began to explain how they had constructed their sketch.

Tim: We stumbled upon that too and really liked it because if you look at the back [of the sketch] we were, like, drawing, like that [a few words were scattered on the back of the sketch] and he [Eric] goes, "Hey, that looks like a star. Why don't you do it as a star," and so I drew it as a star. So then I drew a star and said,

466

Page 14: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

"Hey, that is really great." [...] It helped us understand the thing a lot better. You know, even though all that mumble jumble [referring to the article] is going around in our head, that [the sketch] helped us understand what they were trying to say.

Eric: Because it just put it in view. Tim: It put it in perspective. Eric: Yeah, [. . .] they had all this information here and it was like, "I don't know what

it is." But then we drew it [and] it kind of just gave us a better idea of what they were trying to say.

Tim offered an analogy for their experience, suggesting, "It was like a giant key." As Eric explained, "We took the parts that we understood and put them on

paper. And it kind of explained the parts that we didn't understand." There are several things to notice about Tim and Eric's experience with

transmediation. First, like Jean, they used a convention-the image of a star- to translate their understanding of the text into another sign system. In this case, however, the idea of "translating" or "mapping" an interpretation onto the

expression plane of another sign system may not be the best way to characterize Tim and Eric's experience. As they explained to their classmates, they did not

begin the sketching process with much understanding of the text, let alone any way to represent "all that mumble jumble [referring to the article] . . . going around in our head." Instead, they found that experimenting with the new

expression plane itself helped them find an entry into the text. In other words, "messing about" with various images gave rise to the star image and, as they said, this image served as a "key" that allowed them to open the text and begin an exploration of the interrelations between mathematics and the marketplace. The generative power of the sketching experience thus stands in sharp contrast to Tim and Eric's experience with the "say something" strategy; words alone, even their own words, did not provide the support they needed to derive meaning from this text. Their own reflections on the experience indicate that literally "picturing" their thinking gave them a way to engage in meaningful learning.

Example 2: A Diver in Sheep's Clothing

A series of sketches created by one boy who participated in the ethnographic study of reader/text transaction mentioned earlier provides further insights into the nature of transmediation. The study was carried out over seven months, the first three of which I spent getting to know the children and the curriculum while

negotiating a role for myself in the setting. The remaining four months were devoted to planning, teaching, and documenting the children's engagement in

"sketching" lessons in a small group setting. In an attempt not to disrupt the classroom ecology, I met with the children in their math groups, which were organized on the basis of ability; each of the three groups participated in one

467

Page 15: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

lesson a week for a total of 12 weeks. During this time, I continued my role as a participant/observer in the classroom so as to gain a richer understanding of the children's engagement in "sketching."

Max first read "Killer Whales," an article describing the social organization of whales, during the sixth sketching lesson. After he finished reading it, he drew a sketch reminiscent of a food chain (see Figure 3) and explained it thus: "They all have to get food for their babies but only one will get it and that's him [the big one on the left]. Even he is [the one on the bottom]." Two weeks later the students were asked to read their respective articles again and to generate sketches different from their originals. Max did so and told his group that the scuba diver was trying to catch a baby whale and the other whales were trying to save the calf. When I asked him to talk about how his new sketch differed from his original, he said, "Well, [in] this one [the original] just everybody's hungry-they're not saving anybody" (see Figure 4).

Three weeks later the children were asked to select a fable to read. Max chose "The Wolf in Sheep's Clothing" and was soon drawing an underwater scene which showed a diver about to catch a fish in a cage (see Figure 5). To the right of the diver, he drew a threatening shark saying, "You're dead, buddy!" At this point, his partner, Becky, held up the folder containing the fable and, pointing to the words "wolf' and "sheep," said, "You can't do that [draw an underwater scene]! It has to have a wolf and a sheep!" "Yes, I can!" Max replied. "It's the same meaning!" Becky was eventually convinced, as her collaboration on the explanation of his sketch during sharing time demonstrated. After members of the group had talked about Max's sketch, Becky joined the conversation.

FIGURE 3

Max's Original Sketch of "Killer Whales"

468

Page 16: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

FIGURE 4

Max's Second Sketch of "Killer Whales"

Becky: He [the wolf] tried to kill the sheeps so when he [the wolf] tried to kill the sheeps he got his lesson back.

Max: The farmer killed him [the wolf]. M.S.: What? Max: The shepherd killed him-the shepherd killed him. Becky: The farmer killed him [the wolf] so he's getting killed there [pointing to the

sketch to show the large shark threatening the diver]. Max: [He's] saying "You're dead, buddy!" See, look. See, what's going on is. M.S.: Okay, why don't you talk about it. Cliff, be quiet. Max: See, what's going on is-see he [the diver] came down here [to the ocean] and

he caught this little fish [in a cage]. Becky: Like he was a sheep. Max: Say "help" and the little kid-the guy caught him, you know, he's saying "Hey,

help." And there's this big huge shark's saying "You're dead, buddy!" 'cause that's his little-that's his little kid.

Becky: Yeah, like the farmer. That's his sheep [the fish in the cage] and the guy's getting killed because he's got his sheep.

M.S.: Now, if you had to sum this up, what do you think the author was trying to teach you in that fable? [Several children responded.]

Max: See, what the fox was doing is trying to kill the sheep. As I see it, he was this way [pointing to his sketch]-he [the fish dressed up like a diver] is gonna kill him [the fish in the cage] and the farmer got back at the fox by killing him and he's [the large shark] gonna kill him [the diver trying to fit in with underwater life].

469

Page 17: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

FIGURE 5

Max's Sketch of "The Wolf in Sheep's Clothing"

The last sketch the children did was a meta-sketch- a sketch about sketching- and once again Max created an underwater scene (see Figure 6). When he was asked to talk about it he said:

Max: Well, see, there's that diver again. He's the same one. M.S.: Right. Max: And that fish-he's hungry. And he's [the fish] not trying to get him [the diver],

he's trying to get the worm on that hook on there. M.S.: Oh, I see. Max: And he's running and there's that worm with that hook on there. And that's

exciting. M.S.: And that's exciting. Now ... Max: And I think the class is exciting. M.S.: And you think the class is exciting. Is that why you drew this? Max: Yeah.

This series of sketches takes the argument that transmediation promotes generative and reflective thinking one step further by showing how Max's image of a deep-sea drama came to function as a personal convention that he used to make sense of texts and experiences alike. Like Tim and Eric, Max used a variety of cultural conventions to generate a picture that represented his understanding of the article on killer whales. In the first picture, he drew a series

470

Page 18: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

FIGURE 6

Max's Meta-Sketch

of killer whales each with its mouth open wide enough to swallow up the smaller whale, a common representation of a food chain. The next picture was more like a cartoon, complete with characters from popular culture (Super Whale and Robin Whale) along with speech balloons and verbal labels to communicate the ongoing action. By the time Max encountered the fable "The Wolf in Sheep's Clothing," his underwater scene had itself become a convention, at least for Max and his partner, Becky. As Max explained, the drawing did not require a literal image of a wolf and sheep to convey his understanding of the fable because "it's the same meaning."

By the last sketch, it was evident that Max's underwater scene had become a metaphor for expressing his feelings about the sketching lessons themselves. Just as Tim and Eric used the image of a star, Max used the underwater scene as a metaphor to open and extend his understanding of written texts and experiences. What makes Max's drawings so intriguing is the fact he was placed in both the low math group and the low reading group in his classroom. The very student who invented his own metaphor was seen as "at risk" academically by his teacher because of low performance on the fill-in-the-blank activities that constituted instruction in this classroom.

Taken together, the examples presented here suggest a connection between transmediation and metaphor that may prove helpful as teachers and educational researchers begin to deal with limitations of the transmission model and search

471

Page 19: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

for ways to think about teaching and learning that involve more than words. I will now explore this connection and pursue the idea of conceptualizing transmediation as metaphor.

TRANSMEDIATION AS METAPHOR

Metaphor seems an apt way to represent transmediation, for several reasons. First, as the preceding examples indicate, metaphors (and analogies) are often produced when learners engage in transmediation, especially when they move from language to some form of visual representation. This is probably not so surprising, given the fact that Peirce classified metaphor as "a symbol whose iconicity dominates" (Anderson, 1984, p. 456). Second, metaphor is the paradigm case of semiosis or sign functioning; not only do metaphors produce meaning by looking at one thing in terms of another (Lakoff & Johnson, 1980), but they make this process visible and therefore open to scrutiny.

Transmediation can be conceptualized as metaphor because in both cases the generative power of semiosis is intensified and made visible. Recall that Peirce defines semiosis or sign functioning as a three-term rather than two-term relationship to highlight the notion that a sign does not simply function as a substitute for an object, but provides insights into the meaning of that object. Metaphor operates similarly; as Eco (1984) points out, "we are interested in the metaphor as an additive, not substitutive, function" (p. 89). A metaphor such as "the tooth of the mountain" (Eco, 1984, p. 93) works because a connection between the two terms is created through a third term, the notion of sharp (both a tooth and the peak of a mountain suggest sharpness). The contrast between the two terms is so striking as to call attention to the gap between them, which is crossed by generating a third term common to both (sharpness). By crossing the gap between these two disparate terms we learn something more about each of them due to the mediating role of the interpretant, "sharp." Although metaphors can become routine and conventional as in a so-called dead metaphor, like "the leg of the table," a metaphor like "the tooth of the mountain" shows that they achieve their generative potential through the juxtapositioning of contrast and commonality. Metaphor can thus be regarded as the paradigm case of semiosis because semiosis is both the means through which metaphors achieve meaning as well as an object of thought itself. As Eco (1984) notes, "The best metaphors are those in which the cultural process, the dynamics itself of semiosis, shows through" (p. 102).

As I have argued throughout this paper, crossing the gap between different sign systems is also a generative process, one that makes sign functioning visible. The tension between contrast and commonality is the very heart of transmedia- tion inasmuch as the learner must find a way to rotate the content and expression planes of two different sign systems. As Jakobson's work suggests, one aspect

472

Page 20: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING

of transmediation involves the search for commonality, that is, a way to map the

meanings of the content plane onto a new expression plane. At the same time, Langer's work points out that this search sets up an anomaly for the learner because there is no one-to-one correspondence between different modes of representation. As the preceding examples have shown, the tension between these two aspects of transmediation propels learners to invent a way to cross this gap and in doing so they engage in both reflective and generative thinking.

Given these connections between metaphor and transmediation, metaphor may be a valuable way to think and talk about transmediation, as it offers a more accessible language for characterizing a complex semiotic process. The need for a language other than that of semiotics to talk about what it means to cross sign systems should not be overlooked. The technical language of semiotics is unfamiliar to most teachers, teacher educators, and educational researchers and may convey the sense of elitist jargon, thereby alienating the very people who may need a language for justifying the place of transmediation in schools. Metaphor thus has the advantage of being familiar while capturing the most important dimension of transmediation, that is, the generative power that comes from juxtaposing different ways of knowing, not in an endless play of crossing and crisscrossing, but as a way to position students as knowledge makers and reflective enquirers.

CONCLUSION

Although metaphor is the basis of much of our language, thought, and action, it is often thought of as a "device of the poetic imagination and the rhetorical flourish-a matter of extraordinary rather than ordinary language" (Lakoff & Johnson, 1980, p. 3). Transmediation is at risk of being similarly miscast as a curricular ornament, especially in light of the fact that most classrooms remain

deeply rooted in a verbocentric ideology. Transmediation is equally threatened by the historical tendencies in North American education to reduce ways of

knowing to technique and to emphasize the display of received knowledge over critical and creative thinking. Given these tendencies, the challenge ahead will be to keep transmediation from being dismissed as a frill or reduced to a tech-

nique. Transmediation can be powerful or trivial. Its potential as a learning experience can easily be subverted by closing down the ambiguity that crossing sign systems engenders before students have a chance to explore it. It will be important, therefore, for teachers and teacher educators to understand and

appreciate the semiosic processes at the core of transmediation. And metaphor may offer a productive route through the emerging discourse on transmediation.

NOTE

I thank Howard Smith, Arlene Stairs, and Meg Foley for their helpful comments on an earlier draft of this paper. The "Reading to Learn Mathematics" project was made possible, in part, by

473

Page 21: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

MARJORIE SIEGEL

a grant from the U.S. National Science Foundation (Award #MDR-8850548). The opinions reported here, however, are solely the author's.

REFERENCES

Anderson, D. (1984). Peirce on metaphor. Transactions of the Charles S. Peirce Society, 20, 453- 468.

Bishop, A. (1988). Mathematical enculturation. Dordrecht, The Netherlands: Kluwer.

Borasi, R. (1992). Learning mathematics through inquiry. Portsmouth, NH: Heinemann.

Borasi, R., & Siegel, M. (1988). "Reading to learn mathematics" for critical thinking. Proposal to the National Science Foundation.

Borasi, R., & Siegel, M. (1994). Reading, writing, and mathematics: Rethinking the basics and their

relationship. In D. Robitaille, D. Wheeler, & C. Kieran (Eds.), Selected lectures from the 7th International Congress on Mathematical Education (pp. 35-48). Sainte-Foy, PQ: Les presses de l'universitd Laval.

Buczynska-Garewicz, H. (1981). The interpretant and a system of signs. Ars Semliotica, 4, 187-200.

Cazden, C. (1988). Classroom discourse. Portsmouth, NH: Heinemann.

Davis, P., & Hersh, R. (1981). The mathematical experience. Boston: Houghton Mifflin.

de Paola, T. (1973). Nana upstairs, nana downstairs. New York: Putman.

Dyson, A. H. (1986). Transitions and tensions: Interrelationships between the drawing, talking, and

dictating of young children. Research in the Teaching of English, 20, 379-409.

Eco, U. (1976). A theory of semiotics. Bloomington: Indiana University Press.

Eco, U. (1979). The role of the reader. Bloomington: Indiana University Press.

Eco, U. (1984). Semiotics and the philosophy of language. Bloomington: Indiana University Press.

Fueyo, J. (1992). Reading "literate sensibilities": Resisting a verbocentric classroom. Language Arts, 68, 641-648.

Gallas, K. (1994). The languages of learning. New York: Teachers College Press.

Goodlad, J. (1984). A place called school. New York: McGraw-Hill.

Hardwick, C. (Ed.). (1977). Semiotics and significs: The correspondence between Charles S. Peirce and Victoria Lady Welby. Bloomington: Indiana University Press.

Harste, J., & Short, K., with Burke, C. (1988). Creating classrooms for authors. Portsmouth, NH: Heinemann.

Harste, J., Woodward, V., & Burke, C. (1984). Language stories and literacy lessons. Portsmouth, NH: Heinemann.

Hubbard, R. (1989). Authors of pictures, draughtsmen of words. Portsmouth, NH: Heinemann.

Hynds, S., & Rubin, D. (Eds.). (1990). Perspectives on talk and learning. Urbana, IL: National Council of Teachers of English.

Innis, R. (Ed.). (1985). Semiotics: An introductory anthology. Bloomington: Indiana University Press.

Jakobson, R. (1960). Closing statement: Linguistics and poetics. In T. Sebeok (Ed.), Style in language (pp. 350-377). Cambridge: MIT Press.

Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.

Langer, S. (1942). Philosophy in a new key. Cambridge: Harvard University Press.

474

Page 22: More than Words: The Generative Power of Transmediation ... · PDF fileMore Than Words: The Generative Power of Transmediation for Learning Marjorie Siegel teachers college, columbia

THE GENERATIVE POWER OF TRANSMEDIATION FOR LEARNING 475

Leland, C., & Harste, J. (1994). Multiple ways of knowing: Curriculum in a new key. Language Arts, 71, 337-345.

Percy, W. (1982). The message in the bottle. New York: Farrar, Straus, and Giroux.

Richards, J. (1991). Mathematical discussions. In E. von Glasersfeld (Ed.), Radical constructivism in mathematics education (pp. 13-51). Dordrecht, The Netherlands: Kluwer.

Siegel, M. (1984). Reading as signification (Doctoral dissertation, Indiana University., 1984). Dissertation Abstracts International, 45, 2824A.

Siegel, M., & Borasi, R. (1994). Demystifying mathematics education through inquiry. In P. Ernest (Ed.), Constructing mlathematical knowledge: Epistemology and mathematics education (pp. 201-214). London: Falmer Press.

Suhor, C. (1984). Towards a semiotics-based curriculum. Journal of Curriculum Studies, 16, 247-257.

Watson, D., Burke, C., & Harste, J. (1989). Whole language: Inquiring voices. New York: Scholastic.

Marjorie Siegel is at Teachers College, Columbia University, Box 31, 525 West 120th Street, New York, New York, 10027, U.S.A.