Assessing the Effects of Virtual Communities of

Practice on Professional Development

2003 AAHE Assessment Conference

Darren Cambridge & Vicki Suter EDUCAUSE/NLII/AAHE Joint

Project

Copyright Vicki Suter & Darren Cambridge, 2003. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of

the author. To disseminate otherwise or to republish requires written permission from the author.

Virtual Communities of Practice• “Communities of practice are groups of

people who share a concern, a set of problems, or a passion for a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis.” (Wenger 2002)

• Virtual communities of practice are communities of practice (and the social ‘places’ that they collectively create) that rely primarily (though not necessarily exclusively) on networked communication media to communicate, connect and carry out community activities.

Our questions

• How can we know that VCoPs (designed to promote professional development) substantially contribute to the learning of their participants?

• What kinds of professional learning do VCoPs facilitate?

“Your” questions (write 5 minutes; question based on card color)

• GREEN: How would you go about assessing your own learning as a VCoP member?

• BLUE: What kinds of indicators would you look for to assess members’ learning, if you were a VCoP developer or staff person?

• PURPLE: What kind of information would you need as an administrator to decide about the success (and further funding) of a VCoP designed to support individual/organizational/institutional learning?

Transformative Assessment Model

• Understand purposes• Define desired outcomes (nature of

transformation)• Define elements of outcomes and relationships

(between outcomes/activities/technology)• Recognize key questions to ask in relation to

desired outcomes in context of conceptual framework

• Identify key indicators for key questions • Choose measurement methodologies for key

indicators

Professional Development Learning in VCoPs

• Four elements of professional development– understand and use best practice in your

work– learn how to collaborate with others in your

profession– build connections to professional network(s)– develop knowledge and skills for next phase

of career/development and to contribute to practice

• Use technology and community facilitators to build and support virtual community as professional development environment to advance these four elements

Green/Blue/Purple Triads

Keeping your role (green-VCoP member; blue-builder; purple-decision-maker) in mind, for 10 minutes discuss with your neighbor(s) – someone you didn’t come in with:

• What is the relationship of a VCoP to learning and development (individual/organizational/in the profession)?

• From your perspective and using the information from your card, what are the key questions about learning you want to ask about a VCoP?

• What is the purpose of your assessment?

Clarify Whose Purpose for What

• Purpose of the VCoP assessment – who is the audience and what kinds of decisions will they be making on the basis of the assessment?

• Purpose of the VCoP– from the individual VCoP members’

perspective– from the VCoP developers’ perspective– from the sponsoring institution’s

perspective, in relation to the institution’s mission

Some VCoP Members’ Purposes

• Discuss common problems, experiences and issues

• Identify and share resources, documents, solutions or best practices

• Collaborate on mutually-helpful projects • Plan for face-to-face meetings; continue

relationships beyond face-to-face events

Institutional purpose

• AAHE and EDUCAUSE/NLII are member organizations (individuals/institutions)

• A shared purpose is professional development of higher education professionals (faculty/instructional technologists)

• Another purpose is building the body of knowledge and practice in selected domains

EDUCAUSE mission

The mission of EDUCAUSE is to advance higher education by promoting the intelligent use of information technology.

NLII• The NLII vision: transform the

academy so that it supports education that is – active and learner-centered, – dynamic and lifelong, – collaborative, – cost-effective, – high-quality, and – accessible.

NLII Mission

• Create new collegiate learning environments that harness the power of information technology to improve the quality of teaching and learning, contain or reduce rising costs, and provide greater access to higher education.

EDUCAUSE VCOP Initiative’s Purposes

• Test community-building tools

• Provide new, more flexible mechanisms (time and place independent) for information sharing and collaboration

• Integrate content and communication resources in a professional development learning environment

• Test VCOPs as knowledge generation and management strategy

For more information

• See flyer.• Four VCoP Pilot Initiatives:

– Electronic Portfolios (E-PAC)– Learning Objects (LOVCOP)– New Academy– Teaching and Learning

• http://www.educause.edu/vcop/• vcop@educause.edu

Our Context

• Facilitator teams (ten volunteers, 2-3 per VCOP, supported by two staff members)

• Multiple media– Community platform– Chats– Video conferences– Conference calls– Web tours

• Community Platform: Worktools– Email integration– Resources library– Asynchronous

discussion– Next generation:

chefproject.org• Face-to-face

meetings• Project timeline10/03 facilitator training–

1/04 final recommendations

Transformative Assessment Model

• Understand individual purposes• Define desired outcomes (nature of

transformation)• Articulate relationships

(outcomes/activities/technology)• Recognize key questions to ask in relation to

desired outcomes in context of conceptual framework

• Identify key indicators for key questions • Choose measurement methodologies for key

indicators

Interaction & Information Exchange

Collaborative

Problem Solving

Knowledge Generation

Content

Management

1. Outcomes

2. Key Questions

3. Indicators

4. Methods

Connection & Presence

Heterogeneity

Accommod.

Impact on Practice

Your turn again

• For a few minutes, look at a key question you identified earlier, and think about how it fits into the matrix

• Then, talk over what you’ve discovered with your other small group members

• Suggestions, questions, issues with the matrix?

Our next steps

• Six month preliminary evaluation and report this month (will post on http://www.educause/edu/vcop)

• VCoP activities and EDUCAUSE 2003 Annual Meeting prep

• VCoPs continue through 1/04 and NLII 2004 Annual Meeting

• Final Report, 1/04 (will post)

Your next steps?