Testing plain language textswith your target audienceby Debra Huron

© Debra Isabel Huron 2012

What kinds of plain language texts do you want to test?

They could be:• information leaflets • consent forms • guidebooks, and• booklets.

You could also test Web content.

The first thing you need to know

. . . is that you are testing the text, not the people.

What do you expect adults to do when they read what you have written?

Most readers want and need to . . .

readunderstand, andact on information.

Case study of testing

Target audience: pregnant women served by a public health unit in southern Ontario, Canada.Here is the cover of the booklet that was tested. Only 7 of 30 pages of text were tested by respondents.

Our goal was to find out whether a plain language text met the needs of the target audience.

The full process took about six weeks.Public health nurses were trained to do the one-on-one interviews with respondents.

Respondents were asked to:

• Participate in the materials testing. • Read seven (7) pages from the

booklet as many times as they wished during a two-week period.

• Agree to an interview time when a public health nurse would deliver the oral interview in their home.

• Attend the interview.

Respondents were not paid to be part of the testing. In other testing I have done, respondents were paid a small fee, in cash.

Public health nurses had to:

• Read a short guide prepared by the consultant.• Participate in a question-and-answer conference call with the consultant hired by the Health Unit.• Coordinate the relationship with their respondent(s) by setting up a time for each interview.• Conduct the interview using guidelines provided by the consultant.

I was “the consultant.”

• direct the project, • prepare the interview sheets, • analyze results, and • write a report on the results.

The health unit wanted to document the process and results as a guide for use in future testing.

My role as a consultant was to:

In other testing projects, I have conducted interviews by phone.

A key feature of the project was that the nurses conducted all interviews one-on-one, at home, with the respondents.

Reading is a solitary activity and text comprehension is personal.

Group dynamics make it hard for someone to say they have NOT understood something.Respondents

must feel safe; they are not being tested, the text is.

Why not do group interviews?

The interviews I designed aimed to test the target audience’s comprehension of:• vocabulary• important concepts in the booklet.

The resource contained sidebars with definitions of important terms. Another goal was to test comprehension of a sample of these definitions.

“What does urinalysis mean?”This term was NOT defined in a margin definition.

To answer the question correctly, respondents needed to reply that urinalysis isa) A prenatal testb) A test that reveals bladder or kidney problems.

The first piece of information appears as a heading on a page entitled “Common Prenatal Tests.” The second piece of information appears as the last item on the same page, under the sub-heading “Urinalysis.”

Example of a vocabulary question from the interview sheet:

Example of a content question from the interview sheet:

“Based on your reading of the text, is nail polish remover a pesticide?”

To answer correctly, respondents needed to cross reference two pieces of information: 1) separate definitions of pesticides and solvents and 2) a list of various types of solvents. These pieces of information appeared in different places on the same page. Further, the reader needed to eliminate nail polish as an example of a pesticide to answer the question correctly.

Sample results

Question Correct responses

What is urinalysis? 100%

Is nail polish remover a pesticide?”

75%

The interviews were conducted like an open book exam, in that respondents were not expected to rely on memory to answer the questions. In fact, interviewers encouraged the respondents to have the 7 pages that were being tested on hand and to allow respondents as much time as they needed to find the correct answer on the pertinent page.

• 85% of respondents comprehended key vocabulary and basic terminology

• 83% of respondents comprehended key messages

• Respondents said the design and layout of the handbook met their needs.

Overall results from this case study:

Results that pertain to Definitions

• 100% of respondents said they read the Definitions along the side of the page.

• 100% of respondents said the definitions helped them understand other parts of the booklet.

• 37% said there were enough definitions.

• 50% said there were not enough definitions.

• Only 12.5% said there were too many definitions.

Conclusions

1. Interview results showed a good match between the reading abilities of respondents and the way the handbook text was written. In other words, the printed material met the needs of the target audience.

2. Because respondents made good use of the sidebar Definitions and because some respondents identified terms that they wished had been defined (but were not), more Definitions could have been included in the handbook.

Recommendations

“If a further plain language review is to be done, the focus of such a review should be in these areas:

Add more sidebar Definitions so that all medical terms in the text are clearly defined (or deleted should the Health Unit decide they do not need to be defined)

Ensure that specific comments from respondents and from the analysis of interview results be considered and addressed.”

In case you were wondering . . .

• The sample size was small: about 15 respondents.

• The interview guide consisted of 22 questions (not including questions on design/layout.)

• Interviews took place over a two-week period in respondents’ homes. (Public health nurses were already familiar with respondents and had visited them at home.) …. more details

In case you were wondering . . .

• Questions on design and layout provided a “breather” for respondents, half way through the interview. We did not really need to know whether respondents liked the colours or font chosen; we knew the design was appropriate.

• Respondents had a strong sense that they were helping the health unit improve the effectiveness of this and future resources. This motivation—doing something for the greater good—reduced the need to offer respondents a cash incentive, which is often done and is appropriate when doing testing of this work.

Go to http://www.clarity-international.net/pastjournals.htmland click on Issue 51. A PDF will download. My article is on page 24 and relates to testing of a consumer magazine by adult learners in Canada (May 2004).

Are you interested in reading another case study?

Since 1997, I have been helping healthcare, government, and non-profit professionals communicate more clearly. I offer: writing editing, and training in plain

language writing.

Learn more at www.debrahuron.com