Quality Matters Overview Deborah Adair, Ph.D. Director, Quality Matters October 14, 2008.
-
Upload
rodney-gaines -
Category
Documents
-
view
217 -
download
0
Transcript of Quality Matters Overview Deborah Adair, Ph.D. Director, Quality Matters October 14, 2008.
Quality Matters Quality Matters OverviewOverview
Deborah Adair, Ph.D. Director, Quality MattersOctober 14,
2008
“Quality Matters: Inter-Institutional Quality Assurance in
Online Learning”
Quality Matters is a not-for-profit subscription service providing tools and training for quality assurance of online courses
Colleges and universities, both public and private, throughout the country are joining our community of practitioners
www.qualitymatters.org
QM Process Provides:
Institutional toolset and process to meet quality expectations: Online course design Student learning Improved instruction Assessment and feedback loops Professional development
QM as a National Standard
270 + current subscribers(http://www.qualitymatters.org/Documents/Subscriber%20List%20for%20Publication.pdf
)
42 states represented QM has trained 4000+ faculty and
instructional design staff Recognized by
- Sloan C Excellence in Online Teaching and Learning Award
-2008 USDLA Outstanding Leadership in the field of Distance Learning
MONTANA
WYOMING
IDAHO
WASHINGTON
OREGON
NEVADA
UTAH
CALIFORNIA
ARIZONA
NORTH
DAKOTA
SOUTH DAKOTA
NEBRASKA
COLORADO
NEW MEXICO
TEXAS
OKLAHOMA
KANSAS
ARKANSAS
LOUISIANA
MISSOURI
IOWA
MINNESOTA
WISCONSIN
ILLINOISINDIANA
KENTUCKY
TENNESSEE
MISS
ALABAMA GEORGIA
FLORIDA
SOUTHCAROLINA
NORTH CAROLINA
VIRGINIA
WV
OHIO
MICHIGANNEW YORK
PENN
MARYLAND
DELAWARE
NEWJERSEY
CONNRI
MASS
MAINE
VTNH
ALASKA
HAWAII
PUERTO RICO
VIRGIN ISLANDS
Current Subscribers
Statewide Subscribers
QM Subscribers
Principles of QM
The QM toolset and process A faculty-driven, peer review process
that is…CollaborativeCollegialContinuous Centered - in academic foundation - around student learning
Courses do not have to be “perfect” but QM aims at better than just “good enough”
What Quality Matters is NOT
Not about an individual instructor (it’s about the course design)
Not about faculty evaluation (it’s about course quality)
Not a win/lose or pass/fail test (it’s a diagnostic tool to facilitate continuous improvement of online/hybrid courses)
Goal: Make online instruction as good as it can be
Better than average; more than “good enough”
An attempt to capture what’s expected in an effective online or hybrid course at about an 85% level
85 %
HistoryHistory of Quality of Quality Matters Matters
Winter 2008
Quality Matters Was Launched by MarylandOnline
Maryland consortium for distance learning received 3-year FIPSE grant in 2003
Motivation: Need to provide inter-institutional quality assurance for courses in consortium seat bank
Approach: Research-based, collaborative, peer-centered
Outcome: Sustainable quality assurance process embraced by institutions beyond MOL and Maryland
Success of the QM Grant
Early presentations generated widespread interest
MarylandOnline began to receive recognition for QM WCET Outstanding Work (WOW) Award, 2005 USDLA 21st Century Best Practice Award, 2005 Maryland Distance Learning Association (MDLA) Best
Program Award, 2005.
The Sloan Consortium online workshops introduced hundreds of faculty members and staff to QM.
Peer reviewer training spread far beyond Maryland: 700+ faculty trained to review online courses using
the rubric individuals from 158 different institutions in 28 states
QM Today
Self-supporting non-profit program subscriptions, trainings, and course reviews
Wide-spread adoption Independent subscribers (75), consortium
subscribers (67), and statewide system affiliates (136)
Program development through collaboration User community directs program
improvement
Continuous improvement of tools and services
QM Basics: QM Basics: More than the Sum of its (3) More than the Sum of its (3)
Parts Parts
Winter 2008
The Rubric
The Rubric
Eight standards: Course Overview and
Introduction Learning Objectives Assessment and
Measurement Resources and Materials Learner Interaction Course Technology Learner Support Accessibility
Key components must align.
Rubric Scoring
Points are awarded for 40 specific standards based on: the team majority, AND the pre-assigned weighting of each standard
Specific standards have a point value of 1, 2, or 3; the total points possible in a review is 85
If 2-3 Reviewers believe that a standard is: met, then the full pre-assigned points are
awarded not met, then zero points are awarded
To Meet Expectations…
A course must achieve:
“Yes” on all 17 of the 3-point “essential” standards
A minimum of 72 out of 85 points
72/85 = 85%
The Peer Review Process
Peer Course Peer Course ReviewReview
FeedbackFeedback
CourseCourse
Instructional Designers
InstitutionsFaculty Course Developers National Standards &
Research Literature
Rubric
Course Meets Course Meets Quality ExpectationsQuality Expectations
Course Course RevisionRevision
Quality Matters:Quality Matters: Peer Course Review Peer Course Review
ProcessProcess
TrainingFaculty ReviewersCourse
continues to be offered
Characteristics:
Experienced online/hybrid instructor Trained in applying the Quality Matters
Standards and certified by QM
Training updated with each new edition of the rubric
Receives a small stipend for each review
The Peer Reviewer
Consists of:
3 QM Certified peer reviewers
The chair is a Master Reviewer (having received additional training)
One reviewer must be a subject matter expert
At least one reviewer must external to the institution sponsoring the course
The Peer Review Team
What to Expect from Course Reviews
Consistency and Rigor
Professionalism and Commitment
Useful and Constructive Feedback
Positive Outcomes for Faculty
Survey of faculty whose course wasreviewed indicates that … 91% of respondents made changes in the
course as a result of the review
89% of respondents felt that the quality of course design improved as a result of the review
Sample comments:“I was too close to see what could be improved.”
“Provides a great way to get an objective view of your course.”
“It made all of my online courses better.”
“It provides a view from a more student oriented perspective.”
“Many elements that might contribute to a student withdrawing can be eliminated.”
Quality Matters Training
QM Training
QM trainings support the other program components and are integral to a quality assurance effort
QM principles and the rubric standards are at the heart of all the trainings
There are two general categories of training: Faculty Development and Implementing Quality Matters
Faculty Development Implementing Quality matters
Types of Trainings
Applying the QM Rubric Building Your Online
Course Using the QM Rubric
Improving Your Online Course Using the QM Rubric
Design that Welcomes Your Students
Measurable Learning Objectives at the Course and Module Level
Choosing and Using Media Effectively
Peer Reviewer Certification Master Reviewer Train-the-Trainer Institution Representative/
Course Review Manager
Why Quality Matters? Why Quality Matters?
Winter 2008
Multiple Uses of QM
Reported Uses of QM Tools: Guidelines for initial online course
development Quality assurance of existing courses Ongoing faculty professional
development Institutional re-accreditation packages Focus attention on distance learning
policies & steering committees
Alignment with Accrediting Best Practices
Best Practices Principles* Quality Matters PrinciplesThat education is best experienced within a community of learning where competent professionals are actively and cooperatively involved with creating, providing, and improving the instructional program;
QM is a peer review process involving faculty, instructional designers and other support staff in a cooperative effort to continuously improve online instruction.
That learning is dynamic and interactive, regardless of the setting in which it occurs;
QM treats interactivity and active learning a critical component of every online course.
That instructional programs leading to degrees having integrity are organized around substantive and coherent curricula which define expected learning outcomes;
QM treats the alignment of expected learning outcomes with the contents, activities and assessments as a critical element in every online course.
That institutions accept the obligation to address student needs related to, and to provide the resources necessary for, their academic success;
QM expects every online course to address student access to the academic, technical, and student support services essential to student success.
hat institutions are responsible for the education provided in their name;
Adoption of QM standards reflects institutional commitment to online instructional quality, wherever an institution has endorsed the rubric standards.
hat institutions undertake the assessment and improvement of their quality, giving particular emphasis to student learning;
The QM standards are based on research and best practices to enhance student learning in online environments. Adoption of the QM review process is a clear demonstration of institutional or programmatic commitment to assessment and continuous improvement.
hat institutions voluntarily subject themselves to peer review. QM is essentially a peer review process involving both internal and external peers in the evaluation of courses.
*“Best Practices for Electronically Offered Degree and Certificate Programs” adopted in 2001 by CHEA and 8 regional accreditation bodies.
Collaboration and Community in Distance Education
QM provides platforms to help users collaborate... With QM in building tools that work for Distance
Education With their own institutional users to adapt QM tools to
their specific institutional needs Across the community of QM subscribers to share best
practices, find new resources, and support collegial efforts to improve student learning outcomes
www.qminstitute.org
What’s In It For Faculty?
Improvement of online courses
External quality assurance
Review other courses and gain new ideas for own course; expand professional community
Participation useful for professional development plan and portfolio
Faculty development trainings
Winter 2008
Quality Matters Quality Matters Implementation Implementation
QM’s Role in Quality Assurance
QM looks at course designThe harnessing of technology to deliver
instruction and promote student learning
QM provides a process for peer-to-peer feedback for faculty in the continuous improvement of their course
Quality Matters is not the complete answer to quality assurance for online education, but it can be a critical component
Campus Decisions for Quality
Governance Who will lead a QM project and where will it report?
Engagement How will you gain faculty commitment? Communication, training opportunities, incentives…?
Rubric Use For course assessment, development, maintenance?
Course Reviews Mandatory or optional, official or informal?
Selection Criteria Which courses in what order?
Types of Course Reviews
QM-Managed Course Reviews QM staff assigns review team and
manages process Available with a Supplemental Package,
otherwise on an as-available basis
Official, Subscriber-Managed Course Reviews Trained institution representative
manages according to QM criteria
Independent Course Reviews Schools set up own informal process
Scalability in Implementing QM
Managing Your Reviews Delivering Your Training (APPQMR)
Implementing QM – Roles and Criteria
Preparing for Course Reviews
Course is mature
Learning outcomes (course & unit) are specified
Review team has access to all elements students do
Use of QM rubric in course development
Faculty familiar with QM rubric
Helpful Links
Current Subscriber List http://www.qualitymatters.org/Documents/Subscriber List for Publication.p
df
Courses Recognized in 2008 http://www.qualitymatters.org/Courses 2008.htm
Training Calendar http://www.qualitymatters.org/Training.htm
Fee Schedule http://qminstitute.org/home/Public%20Library/Fee%20Schedule/Quality%2
0Matters%20Fee%20Schedule%20effective%208_1_2008.pdf
QM Staff http://www.qualitymatters.org/Contact.htm
Winter 2008
Thanks to YOU…Thanks to YOU…Quality Matters! Quality Matters!
2008 Post Review Interviews: The peer reviewers fulfilled their responsibilities on the review team.
2008 Post Review Interviews: The review followed the official QM course review process:
2008 Post Review Interviews: The feedback provided by the peer reviewers in the final report was constructive and useful: