Semiotics contributions to accessible interface design.María Inés Laitano. Laboratoire Paragraphe, Université Paris 8.

1

This study was motivated by an observation: that the "communicative wishes" of users with disabilities are still not met by accessible interface design. An enlightening example comes up in the answers to the WebAIM Survey about Preferences in Screen Readers. One question focused on images that enhance the mood or feel of a web page and whether the screen reader should describe them. Out of one thousand one hundred twenty-one respondents, fifty-nine percent wanted these images described, while thirty-one percent said they should be ignored and ten percent had no opinion. However, significantly, the vast majority of respondents wanting images ignored were non-disabled users, i.e. users who only evaluate web accessibility. This result is in clear contradiction with the success criterion 1.1.1 of the Web Content Accessibility Guidelines 2.0 and highlights that the "communicative wishes" of users with visual impairments are still not satisfied.

2

The guidelines limitations has been addressed in the literature in both empirical and theoretical studies.In an article titled "Guidelines are only half of the story" Power et al. (2012) demonstrate through user evaluations, that many of the problems encountered by blind users were not covered by the WCAG. Similar results were obtained by Rømenand Svanæs (2011) through usability testing with disabled users. In brief, these authors propose expanding WCAG with usability principles.In Web for All 2012, Cooper et al. argued that there was a need to address the user and usage context requirements which are not accounted for by metrics which only address factors associated with the digital resource. They argue for a relational approach to accessibility. That is, Accessibility is not a property of the resource, but a property of the relation between the user and the resource in the context of mediation. This approach underlies for instance the British Standard 8878.

3

In the accessibility guidelines approach, content is the main component. Not coincidentally, it is located above all in the scheme. Developers design, code, edit, and create content. End-users get the content and interact with it. Content is defined as the code and markup that determine the structure, presentation, and interaction, as well as text, images, and sounds that convey information to end-users.Since content is a technical component in this approach, the guidelines address formal aspects of the same like format, language, structure, sequence, etc. Some of these formal aspects can be checked automatically, which is an advantage of this approach.

4

Semiotics, in its more social dimension, is the study of communication. Communication understood not as a transmission of content but as a process of influence. When applied in the field of Human Computer Interaction, Semiotics argues that designers communicate with a target audience; they have a communicative strategy towards this target audience. For example, in an e-commerce website, designers can opt for a strategy of persuasion in order to push users-customers to buy. The communicative strategy is expressed not only in the interface content but in the layout as well. Our term for communication through interface layout is modal enunciation.Significantly, communication between designers and users is not a face-to-face communication. As designers and users do not share the same space-time situation, communication between them is deferred. Designers cannot accurately predict the interface interpretation that will make the user, they can only hypothesize it. In consequence, communicative strategy always targets an implicit user, i.e. a hypothesis of user behavior, a presumption about user experience, competences and expectations. This communicative strategy will be efficient only if the empirical user (i.e. a real user in a use situation) recognizes himself in this implicit user.

5

What are the differences between the two approaches?We already said that the guidelines approach is content-centric while the Semiotic approach is Communication-centric.The notion of designers in Semiotic approach is broader than that of developers because it also includes institutions that create websites and usually produce the communicative strategy.In the Semiotic approach, users are not just people using content, they are also the target of communication (customers, students, tourists, etc.) as well as the designers’ hypothesis (how the user carries out its task, what are their skills, etc.).Finally, while the Guidelines approach deals with formal aspects of the layout, the Semiotic approach deals with communication through layout, in other words, with modal enunciation.

6

Let's see a concrete example in order to illustrate these theoretical notions. A research institute wants to design its accessible institutional website. The purpose of the institute is to emphasize the regional identity of the whole organization as well as the individual identity of each research team. For this, the communicative strategy lie in to invite the user to explore artworks of a regional artist. Artworks integrate autochthonous elements and each one is chosen to refer to a research team in a symbolic way. The slide presents a first visual mockup of the home page. The header has the institute logo, a login link and a search field. The main area is occupied by six artworks.

7

In the source code, each artwork is an image within the text of a link that takes the team name and opens the team page. According to the WCAG Images Tutorial, each of these images would be a functional image because it’s used to initiate an action rather than to convey information. The image does not represent different functionality or convey other information than that already provided by the link text, so a null ALT value is applied to avoid redundancy and repetition. It’s clear that, with this solution, the communicative strategy of designers is completely overridden for people who cannot see images. In contrast, an accessible design from a semiotic approach must prioritize and ensure the communicative strategy. Thus, advances in the field of auditory interfaces can be used to provide a similar experience in the auditory modality. So, a short sound can be associated with each research team, evoking by causal or semantic listening, the identity of each team. The sound could be associated to the link through the cue-after propertyof CSS3 Speech Module. Remember that causal listening refers to listening for the clues in sounds that point to the sound’s cause. This is the mode of listening used in auditory icons. Semantic listening is the mode of listening in contexts involving coded audio signals where the listener focuses on decoding that signal to arrive at the message. For instance, the artwork chosen for the team called “Children's cognitive

8

development” look like a pull toy. If we hear the associated sound, through causal listening we recognize the timbre of a little girl's voice. Through semantic listening we recognize that the little girl is learning to talk because she babbles some words in Spanish. Thus, interpretation through semantic listening not only refers to children's cognitive development but also to the regional identity of the institute.

8

Another point to address in accessible design is the reading order of webpage elements for sequential navigation. The success criterion 1.3.2 suggests providing a reading order that follows sequences and relationships within the content —which follow text directionality (left to right or right to left) and markup structures such as tables or lists. In our example, this means navigation from left to right and top to bottom: starting with header elements and continuing through main content elements. While this ordering criterion is not invalid, the semiotic approach provides a higher level criterion founded on the communicative strategy. So the reading order must follow the hierarchy established by modal enunciation. In our example, modal enunciation prioritizes main content over the header: the size of the artworks aims to draw the users’ attention. In fact, the communicative strategy is revealed in the main content. Consequently, following the semiotic approach, it would be more appropriate that sequential reading begins with main content elements and only then continues to header elements. It could be argued that this problem is resolved by success criterion 2.4.1 that proposes providing a mechanism to navigate directly to the main content. However, if our implicit user is a novice, it’s not evident that he would understand the phrase "Skip to main content" at the beginning of a web page. Direct reading of the first

9

research team will be more understandable for such a user. In fact, the WebAIMsurvey shows that the use of skip links is not widespread.

9

Another important issue in accessible design is the name given to textual elements of the interface. For instance, the success criterion 2.4.4 suggests that the text of each link should describe the purpose of the link, i.e. the nature of the result obtained by activating the link. From this perspective, a name like "Log in" is sufficient for the link found on the home header. "Log in" indicates that by activating this link we will find a login webpage. However, this name can be improved by adopting a semiotic approach. The institute communicative strategy targets two audiences: the general public and institute members. Only institute members can log on the website to access information that is not public. Consequently, naming the link as "Members login" instead of "Log in" marks the difference between the two types of implicit user. Institute members will probably recognize himself in this link while non-members will avoid wasting time activating it.

10

The Semiotic approach is based on human communication, a human capacity that cannot be automated in an evaluation tool. However, a kind of semiotic checklist could be added as a further human step in existing methodologies such as Conformance Evaluation. Just like we involve users to ensure usability, we could involve designers in order to check communicative issues. For now, we can summarize this in three general questions:1. Is the communicative strategy available for everyone in the target audience?2. Is modal enunciation preserved in all modalities?3. Do interactive elements take their implicit user into consideration?

11

By way of conclusion we can say that complementing the guidelines approach with a semiotic approach means not only thinking of accessible interface design as content legibility or as a usability problem but rather as more complete communication with the user.

12

References:

Bootz, P. (2007). Éléments d’analyse de l’interface sémiotique des sites Web. In I. Saleh, K. Ghdira, B. Badreddine, N. Bouhai, & B. Rieder (Eds.), Collaborer, Echanger, Inventer : Expériences de réseaux (pp. 107–121). Paris: Hermès Lavoisier.

Chisholm, W. A., & Henry, S. L. (2005). Interdependent components of web accessibility. Proceedings of the 2005 International Cross-Disciplinary Workshop onWeb Accessibility (W4A), 31–37.

Cooper, M., Sloan, D., Kelly, B., & Lewthwaite, S. (2012). A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First. Proceedingsof the International Cross-Disciplinary Conference on Web Accessibility, 201–204.

De Souza, C. S. (2005). The semiotic engineering of human-computer interaction. Cambridge, Mass.: MIT Press.

Leiserson, A. B. (2014). Functional Images. http://www.w3.org/WAI/tutorials/images/functional/.

Pignier, N., & Drouillat, B. (2004). Penser le webdesign: modèles sémiotiques pour les projets multimédias. Paris: Editions L’Harmattan.

13

Power, C., Freire, A., Petrie, H., & Swallow, D. (2012). Guidelines Are Only Half of theStory: Accessibility Problems Encountered by Blind Users on the Web. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 433–442.

Rømen, D., & Svanæs, D. (2011). Validating WCAG versions 1.0 and 2.0 throughusability testing with disabled users. Universal Access in the Information Society, 11(4), 375–385.

Schaeffer, P. (1967). Traité Des Objets Musicaux. Paris: Seuil.

Scolari, C. (2009). The sense of the interface: Applying semiotics to HCI research. Semiotica, 2009(177), 1–27.

Souchier, E. (1998). L’image du texte pour une théorie de l’énonciation éditoriale. Les cahiers de médiologie, 6(2), 137–145.

W3C. (2008). Web Content Accessibility Guidelines (WCAG) 2.0. http://www.w3.org/TR/WCAG20/

WebAIM. (2009). Survey of Preferences of Screen Readers Users. http://webaim.org/projects/screenreadersurvey/

** Slide 8 copyright notice: The author of the visual artwork is Adolfo Nigro and the authors of the sound composition are Marisa Cenacchi and Patricia San Martín.

13