How to train a Dragon: a comparative investigation into attitudes and efficiency of voice dictation

software for formative and summative feedback

Vicky Reed

Melvyn Chimes

Introduction

What we didn’t do What we did

2

What is Dragon? How does it work?

• Voice recognition/dictation software (VRS) v.12 (we worked with v.11)

• It turns talk into text (grading, emails, reports). It has a 99% recognition accuracy.

• Edits and revises documents.

• Frees you from the constraints of keyboard and mouse.

3

Why?

More assessments

to mark

Time consuming

SedentaryKatzmarzyk (2010) life expectancy

study

Other physical and mental stress

Increasing student

numbers, but, staffing levels

staying the same.

4

Introduction - Aims

• To enable lecturers to provide effective feedback in a timely manner.1

• To enhance the quality of (formative) and summative feedback in order to improve student success and achievement.

2• To utilise new learning technologies

to develop more effective summative feedback mechanisms at UCP.

35

Introduction - Outcomes

Predicted Outcomes• More formative feedback. • Improved quality of feedback.

Predicted Outcomes• Positive engagement with new technology. • Trained staff that can disseminate and train other

members.

Predicted Outcomes• Better support for non – traditional learners.• Better feedback mechanisms for staff with

developmental reading disorder.

6

Background literature

Technological factors

• Honeycutt (2003, 2008) – A good quality sound card and a fast processor are needed.

• Spector (2001) this type of educational technology requires significant investment in resources and training.

Physical and Mental factors

• Yengin (2007) 70.4% of 44 lecturers used computers in excess of 6 hrs a day.

• Blatter and Bongers (2002) working on a computer in excess of 6 hrs a day is associated with an increase in work related upper limb disorders.

7

Background literature

User related factors

• Mauri et al (2006) – user friendly, but time consuming.

• Coniam (2004) – VRS can produce results as reliable as those performed by hand.

• Hux et al (2000) - Dragon NaturallySpeaking was significantly better and more consistent at voice recognition than other VRS systems.

• Roberts (2003) - Dragon software has improved implications for persons with learning difficulties.

Batt and Wilson (2008) VRS represents a valuable tool for producing end comments.

• Anderson et al (2009) Technology has been shown to have mixed results as a tool for staff and students with mild disabilities.

8

Methods & methodologyShort (open-ended questions) questionnaire (Gillham 2008)

• Administered electronically for ease.

• Questions were more detailed.

• Mainly employed a sentence completion model, with some completely open ended questions.

Unstructured interviews (Kvale 2008)

• Rich, narrative driven qualitative data.

• Valid data - Emphasis on depth, exploration and generation

• Questions are more ‘flexible’ and ‘more negotiated’ – there are no limits or restrictions

• This may generate new

meanings

9

Methods & methodologyGrounded theoryGlaser & Strauss

(1978)

Uncover basic social processes

simultaneous collection and

analysis of data

Literature (review) can be

used as data

Open, axial, and selective coding

Memo writing

Rigour:Fit and

relevanceWorkabilitymodifiability

10

Results

Questionnaire

Technology: Hardware, Software

Time and location: getting used to, ‘talking feedback’ and the time it takes to form this and access to ‘quiet’ areas Confidence: No to training others, not aware of all features and how to access them

Interviews

Technology: Hardware, Software

Time and location: problems speaking slowly in a quiet room, time lost in corrections, ‘not worth it ‘Confidence: Improves with practice, still feels the need to be ‘tied to the chair’

11

Summary

• In both the interviews and questionnaires the answered focused on three main themes/areas

• The memos showed that the technological issues and those related to usability (time and location) and user confidence emerged as the main themes.

• As a result, it proved quite difficult to align the emerging themes with any of the predicted outcomes.

12

Evaluation

This project is best described as a ‘pilot study’.

The main aims and predicted outcomes were not achieved, however the data generated was useful.

The questions may have needed to be developed ‘open-ended’ free flowing narrative is unsuitable.

Semi-structured focus groups (as originally planned) would probably be more benefical.

13

Conclusion

• Despite all the complications associated with staff absence and radically revised methods etc, the research did provide some illuminating and useful information.

• VRS is a useful tool, but how it is used or implemented may need further investigation.

14

Recommendations

Future Research• A larger study is needed,

with more participants over longer period of time.

• More focused training and specific time given for training.

• Updated hardware to cope with the demands of the software.

The use of Dragon for feedback

• Once adequate training has been given all participants recommend that VRS be utilised as part of the ‘lecturers toolbox’.

• Investigation into its use in assessment development for those with learning disabilities has been suggested.

• Shared comment banks would be a useful addition.

15

Bibliography• Charmaz K (2006) Constructing Grounded Theory: A Practical Guide through Qualitative Analysis London Sage

Publications• Crooks DL (2001) The importance of symbolic interaction in grounded theory research on women’s health Health Care

for Women International 22,11-27• Dey I (1999) Grounding Grounded Theory Guidelines for Qualitative Inquiry, San Diego: Academic Press.• Glaser BG Strauss AL (1967) The discovery of grounded theory: Strategies for qualitative research New York: Aldine de

Gruyter• Glaser BG (1978) Theoretical sensitivity, California, The Sociology Press• Glaser BG (1992) Basics of grounded theory analysis Emergence vs. forcing California: Sociology Press• Melia, K M (1996) Rediscovering Glaser, Qualitative Health Research 6(3) 368-373. • Strauss A Corbin J (1990) Basics of qualitative research Grounded theory procedures and techniques Newbury Park:

Sage Publications• Strauss A Corbin J (1998) Basics of qualitative research techniques and procedures for developing grounded theory

Second edition Thousand Oaks: Sage Publications• Gillham, B. (2008). Developing a questionnaire (2nd ed.). London, UK: Continuum International Publishing Group Ltd.• Kvale & Brinkman. 2008. InterViews, 2nd Edition. Thousand Oaks: SAGE.• Blatter, B.M. & Bongers, P.M.(2002)Duration of computer use and mouse use in relation to musculoskeletal disorders of

neck or upper limb, International Journal of Industrial Ergonomics, 30, (4–5), p295-306.• De La Paz, S. (1999). Composing via dictation and speech recognition systems: Compensatory technology for students

with learning disabilities. Learning Disability Quarterly, 22, p173-182. Cited in: Honeycutt, L. (2003). Researching the use of voice recognition writing software. Computers and Composition. 20, p77-95. 

• Gross, N. & Judge, P. (1998) Let’s talk!: Speech technology is the next big thing in computing, but will it put a PC in every home? Business Week, 3566, p60-72

• Hacifazlioglu, O., Sacli O.A., & Yengin, I. (2007) Lecturer’s attitudes towards the use of technology: Alternative strategies for faculty administrators. In: ERIC (Educational Resources Information Centre) 7th International Educational Technology Conference, North Cyprus, 2007. 

• Honeycutt, L. (2003). Researching the use of voice recognition writing software. Computers and Composition. 20, p77-95. 16