BENCHMARKING INSTITUTIONAL READINESS FOR TECHNOLOGY

ENHANCED LEARNING

Helen Carter (@HCatME )President, ACode

leading technology enhanced learning and teaching

INTRODUCTION

How can we ‘prove the quality’ or ‘improve the quality’ of technology enhanced learning (TEL)?

How do we support institutional readiness for change?

One approach that has helped many institutions do this is through the regular use of benchmarking.

More particularly using the Australasian Council on Open, Distance and E-Learning (ACODE) Benchmarks.

WHAT IS BENCHMARKING?

“A structured, collaborative, learning process for comparing practices, processes or performance outcomes. Benchmarking can be used to prove quality or improve quality and is evidence-based.” TEQSA Benchmarking Guide

WHAT IS QUALITY?

“Quality of education is inherently a political construct; when considering e-learning quality, institutional leaders need to balance the various externalities with technical and organisational realities to see how the nature of the e-learning experience systematically supports the outcomes desired by the organisation.” Stephen Marshall, Victoria University of Wellington

TENSIONS BETWEEN QA AND QE

Quality Assurance • Inhibits frank

reporting

• Promotes incremental improvement

• Discourages risk taking

Quality Enhancement• Encourages and

requires frank reporting• Facilitates

transformational change

• Supports and manages risk taking

(Raban 2009)

QUALITY FRAMEWORKS

Framework/Tool Theory of Change6EOLE Quality Management Framework

None

ACODE Benchmarks for Technology Enhanced Learning

Collaborative Benchmarking

EADTU E-xcellence Next Collaborative Benchmarking

EFMD Certification of E-learning (CEL)

None

EFQUEL UNIQUe Certification Nonee-Learning Guidelines (eLG) Nonee-Learning Maturity Model (eMM) Maturity ModelQuality Matters (QM) NoneTaking the Lead None

WHERE DO BENCHMARKS FIT?

Gove

rnm

ent

regu

lator

y ag

encie

s req

uiring

evide

nce

Instit

ution

al Re

porti

ng

ACOD

E Be

nchm

arks

Instit

ution

al pr

oces

ses

E-Le

arnin

g gu

idelin

es/st

anda

r

ds Cour

se le

vel

proc

esse

s

Areas of overlap

WHAT IS ACODE?

ACODE is the peak body for Australasian universities engaged or interested in technology enhanced learning (TEL)Originally established by the Australian government, its mission is to enhance policy and practice around TEL at institutional, national and international levels

www.ACODE.edu.au

Developed 2004Revised 2007Updated 2014Used by many

since …

DEVELOPMENT OF THE BENCHMARKS

UPDATED BENCHMARKS

Focus shifted from: eLearning to Technology Enhanced Learning (TEL)

Stronger alignment with L&T standards and assuming a way to measure the quality of an individual course/unit/subject.

Greater emphasis on emerging technologies & innovation, particularly in planning and budgeting.

A new measure around open education practices and the sustainable use of resources.

A measure on how institutions are assuring a level of quality in their externally hosted services.

THE 8 BENCHMARKS FOR TEL

1. Institution-wide policy and governance for technology enhanced learning;2. Planning for institution-wide quality improvement of technology

enhanced learning;3. Information technology systems, services and support for technology

enhanced learning;4. The application of technology enhanced learning services;5. Staff professional development for the effective use of technology

enhanced learning;6. Staff support for the use of technology enhanced learning;7. Student training for the effective use of technology enhanced learning;8. Student support for the use of technology enhanced learning.

BENCHMARKING TOOLKIT

Methodology developed to provide institutions with:• a platform to self-assess their standing

against some/all of the 8 benchmarks, and to stimulate meaningful conversations, at a local level, around how they are using technology to support their L&T.

• an opportunity to share & learn from each other, based on their individual institution’s responses (via an inter-institutional event).

New Self-Assessment templateNew Guidelines developed

EXAMPLE - BENCHMARK 5Staff professional development for the effective use of technology

enhanced learning Scoping Statement The key focus is on developing teaching staff to make effective use of a range of approaches to technology

enhanced learning (TEL). Staff development activities encompass individual and group delivery, face-to-face, as well as online.

Self-directed learning activities and resources are also included. Some professional development will be designed and delivered to meet the strategic needs of the organisation, whilst other activities will be provided to meet the demands of teaching staff as they arise.

Good Practice Statement Quality learning and teaching is brought about where people are confident, enthusiastic, skilled and well

supported, and learning experiences are designed to engage the learner and employ a variety of approaches.

Engagement in professional development should not be limited by factors of physical location, equity or technological skills. This means that staff development is offered flexibly, accommodates a range of entry points, is evaluated and is informed by the work of related units.

A good practice approach to the use of technology enhanced learning reflects an understanding of learners’ characteristics and needs as required by different discipline contexts.

EXAMPLE - BENCHMARK 5Staff professional development for the effective use of technology enhanced learning Performance indicators1. A framework for staff development in technology enhanced learning is part of the

institution's learning and teaching strategy. 2. Processes are in place and in use to identify staff development needs in support of the

institution’s strategy for technology enhanced learning. 3. Educational and technical expertise is used to develop quality programs and resources

addressing staff development needs. 4. Coordination occurs between those areas providing staff development for technology

enhanced learning across the institution. 5. Staff development for technology enhanced learning is resourced. 6. Staff development programs are delivered flexibly and address differing skill levels. 7. Evaluation data is used to inform the planning for continuous improvement of staff

development processes.

PERFORMANCE INDICATORS & MEASURES

CONSOLIDATION

COMPARE

BENCHMARKING ACROSS THE SECTOR

teams institutions

Consolidate

Self-asses

s Compare

individuals

Each institution completes a self assessment:• People come together from different areas• Agree on where they stand• Provide a rationale and evidence as to why

BENCHMARKING SUMMIT

Major trial of the Benchmarks conducted in Sydney in June 2014

24 institutions from 5 countries compared their TEL capacity using the ACODE benchmarking process.

Institution BM 1

BM 2

BM 3

BM 4

BM 5

BM 6

BM 7

BM 8

Asia Pacific International College       X     X  Auckland University     X          Auckland University of Technology           X   XAustralian Catholic University   X     X X    Christchurch Polytechnic X     X        Curtin University   X     X      Federation University X       X X   XFlinders University   X   X        Lincoln University   X   X        Macquarie University     X   X      Open University, UK X X X X        Queensland University of Technology X       X      University of Canberra X         X    University of Otago X  X X X X      University of New England X       X   X XUniversity of Southern Queensland         X X X XUniversity of South Africa X   X   X      University of the South Pacific     X X        University of Technology Sydney   X   X        University of Western Australia     X   X      University of Western Sydney X     X   X    University of Wollongong         X X X XVictoria University (Melbourne) X         X    Victoria University Wellington X X X X X X X XTotal 11 8 8 10 12 9 5 6

HAVING THE CONVERSATION

Assoc. Prof. Michael Sankey

convening the inaugural

Benchmarking Summitat Macquarie

University

VALUE OF BENCHMARKING

The ACODE Benchmarks allows institutions to: Self-assess their capacity in TELShare this with other like-minded institutions Incorporate this into their ongoing QA/QE processes

Develop an enterprise-level perspective

VALUE OF BENCHMARKING

The beauty of benchmarking is not around which tool or set of standards you are using, it's more about the dialogue that emerges and the sharing of practice that is the real winner for all concerned.

It opens the door for further collaboration.It serves as a mechanism to facilitate

discussion at senior leadership level.

BENCHMARKING SUMMIT

“Great opportunity to meet and share where everyone is at. The benchmarking exercise is a great self reflective practice that is reinforced

through the feedback and deliberation from other institutions”

“I really enjoyed this Benchmarking Summit, I have learned a lot from the inter-institutional activity

and will definitely be sharing and pushing for these benchmarks to be accepted at our institution”

CONCLUSION

Many of the issues we face can be remediated by simply taking the time to self-assess against the performance indicators.We can then extend that by sharing our current practice with those in similar circumstances. This build relationships and stronger ties - preparing our institutions to meet the unique challenges of building a strong digital future.The ACODE Benchmarks provide a catalyst to help make this happen.

FURTHER INFORMATION

ACODE Benchmarking Summit – June 2016University of CanberraEmail secretariat@acode.edu.au to register your interest

To download the ACODE Benchmarks:Go to http://www.acode.edu.auand look under Publications