Developing Program and Course Learning Outcomes in ... · PDF fileCourse Learning Outcomes in...

Developing Program andCourse Learning Outcomes in Alignment

with QFEmirates

Ian P. Cumbus

National Qualifications Authority

To consider the:

requirements in relation to the Qualifications Framework(QFEmirates).

principles underpinning the development of soundlearning outcomes for programs and constituent courses.

alignment process of outcomes with QFEmirates leveldescriptors and the evidence needed to demonstratealignment.

Introduction- purpose and outcomes

2

Delegated Responsibilities to

Commissions

NQA Board

National Qualifications

Authority (NQA)

MOHESR

MoE

Others

Delegated responsibility

Commission for

Academic Accreditation

(CAA)

New entity

Vocational Education

Training and Awards

Commission (VETAC)

Delegated responsibility

General Education

Commission (GEC)

Existing body

Op

era

tio

nal

Level Qualification Knowledge SkillsAutonomy &

responsibility

Self-

development

Role in

context

10 Doctoral

9 Master

8 Graduate Diploma

7 Bachelor

6 Higher Diploma

5 Diploma

1-4 Certificates

Level descriptors must reflect all five strands Increasing

level of

complexity

QFEMIRATES: GRID OF LEVEL

DESCRIPTORS

Level descriptors are:

statements for each level, found in each of the 5 strands

cumulative: assumes the inclusion of all the outcomes in preceding levels.

The 10 levels and 5 strands form a grid of ‘level descriptors’

• Level descriptors are a SET of learning outcomes thatdefine LEVELS in a framework of qualifications.

• 10 levels in UAE - Levels 5-10 relate to HE qualifications.

• Each level has 5 strands:

Knowledge

Skill

Autonomy and Responsibility

Self-development

Role in Context

Grid Structure

5

Rationale for UAE NQF

• Single framework

• Common benchmark

• Common language

• Transferability

• Quality and consistency

• International alignment

• Lifelong learning

QFEmirates

6

QFEMIRATES GRID OF

LEVEL DESCRIPTORS

Level Knowledge (Version 01022141) Skill Autonomy and responsibility Self-development Role in context

10

comprehensive, deep and overarching knowledge at the frontier of a field of work or learning and at the interface between different fields

new knowledge, as judged by independent experts applying international standards, created through research or scholarship, that contributes to the development of a field of work or learning

a range of advanced and specialised skills and techniques, including synthesis, evaluation and reflection, required to extend and redefine existing knowledge or professional practice or to produce new and original knowledge

advanced skills in developing innovative solutions to critical problems in research using highly developed cognitive and creative expert skills and intellectual independence to the field of work or learning

highly developed expert communication and technology skills to present, explain and/or critique highly complex and diverse matters

can act with substantial authority, creativity, autonomy, independence, scholarly and professional integrity in a sustained commitment to the development of new ideas or processes or systems in challenging and novel work or study contexts

can account for overall governance of processes and systems

can lead action to build and transform socio-cultural norms and relationships

can analyse and critique the state of learning in a field and contribute to its advancement

can originate and manage complex professional processes

can lead and take full responsibility for the development and strategic deployment of professional teams and self

9

comprehensive, highly specialised knowledge in a field of work or learning and at the interface between different fields, including frontier concepts; including advanced knowledge of applicable research principles and methods

critical awareness of knowledge issues, as the basis for original thinking; encompassing appropriate processes of enquiry and current processes of knowledge production

detailed body of knowledge of recent developments in a field of learning or work and/or professional practice

specialised skills required in research, analysis, evaluation and/or innovation of complex ideas, information, concepts and/or activities

advanced problem-solving skills required to develop new knowledge and procedures and to integrate knowledge from different fields using highly developed cognitive and creative skills and intellectual independence to the field of work or learning

highly developed specialist communication and technology skills to present, explain and/or critique highly complex matters

can take responsibility for managing professional practice or work, processes or systems, or learning contexts that are complex, unpredictable and require new strategic approaches and/or intervention or conceptual abstract solutions

can account for high level governance of processes and systems

can analyse and reflect on socio-cultural norms and relationships and act to build and transform them

can self-evaluate and take responsibility for contributing to professional knowledge and practice, as well as implementing ethical standards and further learning

can initiate and manage professional activity

can take responsibility for leading the strategic performance and development of professional teams and self

8

advanced or specialised knowledge and critical understanding in a specialised field of work or learning and at the interface between fields

critical approach to a systematic and coherent body of knowledge and concepts gained from a range of sources

problem-solving skills applied to a specialist field and the integration of knowledge from different fields of work or learning to solve complex unpredictable and/or abstract problems with intellectual independence

critical selection of appropriate research tools and strategies associated with the discipline field of work or learning

highly developed advanced communication and technology skills to present, explain and/or critique substantively complex matters

can take responsibility for developing and implementing new or creative approaches to managing complex work processes and organisation, resources or learning, including leading and managing teams within a technical or professional activity

can work effectively as an individual or in team leadership contexts

can express a comprehensive, internalised, personal world view, while accepting responsibility to society at large and to socio-cultural norms and relationships

can self-evaluate and take responsibility for contributing to professional practice and further learning, in complex and sometimes unfamiliar learning contexts

can lead, contribute and observe ethical standards

can manage professional activity

can take responsibility for leading the strategic performance of professional teams and self

can coordinate peer relationships with qualified practitioners and lead multiple, complex groups

7

specialised factual and theoretical knowledge and an understanding of the boundaries in a field of work or learning, encompassing a broad and coherent body of knowledge and concepts, with substantive depth in the underlying principles and theoretical concepts

familiarity with sources of new research and knowledge with integration of concepts from outside fields

technical, creative and analytical skills appropriate to solving specialised problems using evidentiary and procedural based processes to predictable and new contexts associated with a field of work or learning, encompassing evaluating, selecting and applying appropriate methods, procedures or techniques in processes of investigation to identified solutions

selection of appropriate research tools and strategies associated with the field of work or learning

highly developed advanced communication and technology skills to present, explain and/or critique complex matters

can take responsibility for developing new and advanced approaches to managing or evaluating complex and unpredictable work procedures and processes, resources or learning, including leading teams within a technical or professional activity

can manage technical, supervisory or design processes in unpredictable contexts

can work creatively and/or efficiently as an individual or in team leadership or managing contexts

can express an internalised, personal view, and accept responsibility to society at large and to socio-cultural norms and relationships

can self-evaluate and take responsibility for contributing to professional practice and further learning

can manage learning tasks independently and professionally, in complex and sometimes unfamiliar learning context

can take initiative to address learning needs and function independently within learning groups

can contribute and observe ethical standards

can function with full autonomy in technical and supervisory contexts and adopt professional roles with little guidance

can take responsibility for the setting and achievement of group or individual outcomes and for the management and supervision of the work of others or self in the case of a specialisation in field of work or learning

can take responsibility for managing the professional development of individuals and groups

can participate in peer relationships with qualified practitioners and lead multiple, complex groups

6

comprehensive, specialised factual knowledge and an understanding of the boundaries in a field of work or learning, encompassing a broad and coherent body of knowledge and concepts, with depth in the underlying understanding of the principles and concepts

familiarity with sources of existing knowledge and the integration of concepts from outside fields

specialist technical, creative and conceptual skills appropriate to solving complex problems associated with a field of work or learning

a comprehensive range of specialist cognitive and practical skills appropriate to planning and implementing solutions to varied, unpredictable and unfamiliar problems within a field

identification and use of appropriate research tools and strategies associated with the field of work or learning

advanced communication and technology skills to present, explain and/or critique complex matters

can take responsibility for developing appropriate approaches to managing complex work procedures and processes, resources or learning, including leading teams within a technical or professional activity

can manage technical, supervisory or design processes in varied, unpredictable and unfamiliar contexts

can work effectively as a specialist role or in team leadership contexts

can express an internalised, personal world view, reflecting engagement in society at large and in socio-cultural relationships

can take initiative to address learning needs and function independently within learning groups

can support and observe ethical standards

can evaluate own learning and identify learning needs in a familiar and unfamiliar environment

can function with full autonomy in technical and supervisory contexts and adopt professional roles under guidance

can take responsibility for the setting and achievement of group outcomes and for the supervision of the work of others

can take responsibility for managing the development of individuals and groups

can participate in peer relationships with qualified practitioners and lead multiple groups

5 comprehensive, specialised knowledge within a broad field of work or learning, including an understanding of the underlying theoretical and abstract concepts with significant depth in some areas

technical, creative and conceptual skills appropriate to solving a range of problems associated with a field of work or learning

a comprehensive range of specialist cognitive and practical skills appropriate to identifying and implementing solutions to familiar and non-routine problems within a field

use of appropriate research tools and strategies associated with the field of work or learning

comprehensive communication and technology skills to present, explain and/or critique complex matters

can take responsibility for coordinating the implementation of appropriate approaches to complex work procedures and processes, resources or learning, including leading teams within a technical or paraprofessional activity

can exercise coordination and/or supervision in routine, familiar and some non-routine work or learning contexts

can coordinate technical, supervisory or design processes in routine, familiar and non-routine contexts

can express an internalised, personal world view, in the context of an understanding of socio-cultural relationships

can take responsibility for own learning within a managed and non-routine environment

can comprehend and observe ethical standards

can evaluate own learning and identify learning needs in a developed environment

can function with autonomy in technical and coordination contexts and support professional and adopt paraprofessional roles under guidance

can function both independently and in a coordination role within multiple groups

can take responsibility for coordinating the development of individuals and groups

can review and develop the performance of self and others

QFEMIRATES: AUTHORITY TO

ISSUE QUALIFICATIONS

• Commission for Academic Accreditation (Ministry of Higher Education and Scientific

Research)

• Licensure of institutions and accreditation of academic programs of one academic year

or longer

• Once licensed and accredited for programs, institutions can issue qualifications

Higher education sector

• Vocational Education and Training Awards Commission (VETAC)

• VETAC has authority to issue qualifications, however may delegate this function to

Registered Training Providers (RTPs) that meet VETAC’s quality assurance conditions.

Vocational education and training sector

• General Education Commission (Ministry of Education, ADEC and KHDA)

• General Education (G12) Secondary School Certificate or any equivalent qualifications.

General education sector

CAA STANDARDS FOR

HIGHER EDUCATION

Standards for Licensure and Accreditation 2011

e-Learning Standards

for Licensure and

Accreditation 2007

Standards for Licensure

and Accreditation of

Technical and

Vocational Education

and Training 2009

http://www.caa.aeProcedural Guidelines 2011

3. The Educational Program

The academic programs and courses offered by the institution are

appropriate to its mission. International academic norms are reflected

in program design and composition, in the delivery of teaching and

instruction, and in the assessment of student achievement. The

institution demonstrates that academic programs and courses are

delivered as they are specified, are reviewed and continuously

improved, and that students meet the intended outcomes. Program

learning outcomes are appropriate to the level of qualifications

awarded, consistent with the National Qualifications Framework.

EXTRACTS FROM CAA

STANDARDS 2011

The burden of proof is on the institution to evidence alignment and demonstrate

that there is a consistent approach across the organisation to alignment.

• The implication is that the institution has (should have) an internal protocol

and process that supports alignment and that it is applied consistently to all

programs before they are submitted for accreditation.

INSTITUTIONAL RESPONSIBILITIES

The institution ensures that a suite of documentation and evidence of supporting

practice is produced that will enable it to confirm alignment BEFORE submission for

accreditation.

• Standard documents /templates should be used.

• The documentation will need to address the criteria in CAA’s Standards and

Procedural Guidelines for Accreditation, particularly Section 3.

Statements of what a student is expected to know,understand and/or be able to demonstrate on successfulcompletion of their studies.

They represent a move away from traditional models oflearning, which emphasise inputs such as content andteaching hours, to a more modern ‘student-centred’approach which emphasises outputs in terms ofcompetencies.

What are Learning Outcomes?

What is the difference between Aims, Objectives and Outcomes?

• Aims are broad purposes or Goals (College)

• Objectives are intentions of the instructor in measurable terms (Faculty)

• Outcomes are what results from the learning process (students/learners) – Minimum acceptable level for a pass (threshold level)

• Myths say the level of LOs is based largely onthe verbs used, e.g.

• ‘describe’ is level 5• ‘analyse’ level 6• ‘critically analyse’ is level 7

• Scope and quality of guidance are moreimportant

Myths about learning outcomes

JUDGING THE LEVEL OF

LEARNING OUTCOMES

Scope of what students

are asked to do:

The detail, the depth of

understanding, the amount

of complexity, the use and

evaluation of different

theories and approaches, and

the amount of uncertainty.

Quality of guidance

How much guidance they will

be given by the instructors

and how much they are

capable of doing things

independently.

• Learning Outcomes should be judged by scope and guidance.

• Don’t get too focused on technical detail – try to get an overview of level and progression. Do the teaching team members know how it fits together to make a coherent and progressive whole for students?

Learning Outcomes are an art not a science!

Are writing learning outcomesan art or science?

Level 5 (year 1 undergraduate)

• Analysis: can analyse with guidance using givenclassifications /principles;

Level 7 (final year undergraduate)

• Analysis: can analyse new and/or abstract data andsituations without guidance, using a range oftechniques appropriate to the subject.

Distance travelled ?

This is an opportunity to practice writing a programlearning outcome for a skill – the skill of analysis -

• and to practice pitching it at postgraduatelevel building on lower undergraduate leveloutcomes.

• Working with the person next to you look atthe following slide and write a programoutcome for level 9 (Masters) for analysis.

Group Exercise and Discussion

Level 5 (year 1 undergraduate)

• Analysis: can analyse with guidance using givenclassifications /principles;

Level 7 (final year undergraduate)

• Analysis: can analyse new and/or abstract data andsituations without guidance, using a range oftechniques appropriate to the subject.

Level 9 (Masters level)

• Analysis: ??

Group Exercise and Discussion

Level 9 (Masters level)

• Analysis: can analyse new and/or abstract data(including incomplete data)and situations using a rangeof advanced techniques relevant to the subject anddemonstrate self direction and originality in explainingand interpreting outcomes.

• Look at skill at Level 9 in NQF. Does this match?

• Does yours match?

Feedback

Should include three things:

• What students should be able to do at the end of the program –active verbs.

• The conditions under which they should be able to do it.

• The standard or level to which they should know or be able to do it.

What should be included

in writing learning outcomes

21

1. Written in the future tense

2. Identify the MOST IMPORTANT learning requirements –danger is that people try to use too many

3. Be achievable and assessable - LO’s are the ESSENTIAL thingsstudents need to achieve to pass and gain the qualification

4. Use language which students will understand

5. Must be able to relate to the level descriptors in theQFEmirates

Writing Program Outcomes - some tips.

22

Intended program learning outcomes

On completion of the program, the student will be able to:

LO1 Medical knowledge:

Describe the structure, development and function of human reproductive system

including the deviations from normal, the cytogenetic and molecular basis of disease

and apply this knowledge in the diagnosis and management of infertility.

LO2 Patient care:

Demonstrate patient care through clerking, diagnosing and managing the patients at

the level of a practitioner entering the specialty.

LO3 Communication:

Communicate appropriately with patients, relatives, agencies, peers, superiors and other

concerned parties through verbal, written, electronic and other relevant channels.

Etc……………………….

EXAMPLE: MASTERS’ LEVEL

PROGRAM OUTCOMES

• Apply fundamental security concepts to network communications, infrastructure and operations √

• Design, implement and document a security policy and disaster recovery plan for large-scale networks √

• Recognize the need for, and an ability to engage in life-long learning?????

2/7/2017 24

Examples of Program Outcomes:Information Security

• Explain the basic characteristics of domestic animals and their husbandry in arid environments √

• Conduct research using appropriate statistical methods for data analysis √

• Foster the animal production and fisheries sector in the UAE and Arab world ????

• Engage in life-long learning ????

2/7/2017 25

Examples of Program Outcomes:Marine Fisheries & Animal Science

• Ensure that program outcomes are achieved at the point of successful completion of the program – not in employment (See BSc Hort.)

• Select a variety of active verbs appropriate to the increasing level through a program (See CIT – “ability to…” and BSc Nutritional Science “understand…)

• Active Verbs to avoid: Foster, Know, Understand, Appreciate (See BSc Fisheries…)

2/7/2017 26

Common issues for attention:

2/7/2017 27

2/7/2017 28

• Knowledge is the cognitive representation of ideas,events or happenings. It can be learned from practicalor professional experience as well as from formalinstruction or study and can comprise description,memory, understanding, thinking, analysis, synthesis,debate and research.

• The UAEQF descriptors refer to theoretical and/or factualaspects of knowledge.

Strand Definitions- Knowledge

• Skill is the learned ability to perform a function that in someway responds to or manipulates the physical, informationalor social environment of the person. This strand of learningoutcome incorporates the concept of ‘know-how’, which isthe procedural knowledge required to carry out a task.Know-how may be assessed directly or implied fromperformance; otherwise skill can only be measured byperformance.

• The UAEQF descriptors refer to skills as cognitive (use oflogical, intuitive, creative and conceptual thinking) andpractical (involving manual dexterity and the use ofmethods, techniques, processes, materials, tools andinstruments).

Strand Definitions- Skill

• Aspects of competence, is the effective and creativedeployment of knowledge and skill in human situations,including general social and civic, as well as specificoccupational contexts. Aspects of competence, alsoencompasses the learner’s ability to acknowledge theboundaries of their knowledge and skill and plan totranscend these through further learning. Aspects ofcompetence, is typically acquired by practice andreflection. For the description of aspects of competence, itis essential to make explicit the contexts in which thelearner can apply their knowledge and skill.

• The UAEQF descriptors include statements describingaspects of competence in terms of autonomy andresponsibility competence, self-development competenceand role competence.

Strand DefinitionsAspects of Competence

• Autonomy and responsibility can be thought of asopposite ends of a spectrum of learned abilities to applyknowledge and skill in situations of varying challenge andcomplexity.

• Autonomy refers to the level of internalised self-management and individuation achieved by the learner.Responsibility refers to the learner’s acquired ability tounderstand, appreciate and interact with the social andcultural structures of his/her community and society.

• The UAEQF descriptors contain statements describing theoutcomes that signify balanced achievement in both ofthese aspects of learning appropriate to each level.

Strand Definitions-Aspects of Competence –

Autonomy and Responsibility

• This strand encompasses the learned ability of anindividual to recognise and acknowledge theboundaries of his/her current knowledge, skill andcompetence and engage to transcend these limitationsthrough further learning.

• Self-development competence relates to how and towhat extent the learner can manage his/her ownlearning processes.

Strand Definitions –Aspects of Competence -

Self-Development

• For many purposes, functioning in various kinds of groups is akey aspect of how knowledge and skills are put to effectiveuse. This requires learning to adopt appropriate roles withinthe group and apply social skills and an understanding of thetasks of the group.

• Higher levels of competence are associated with the ability toadopt multiple roles as well as with roles requiring leadership,initiative, autonomy and responsibility as well as participationin more complex groups.

• The UAEQF detailed level indicators include statements aboutthe ability to adopt appropriate roles in specified operationalcontexts.

Strand Definitions Aspects of Competence

Role in Context

1. To describe what alignment means, the evidence to

demonstrate it, and possible changes to underlying

practices in teaching, learning, and assessment, to

ensure student outcomes are aligned to QFEmirates.

2. To define the approaches to alignment, including how

student outcomes are aligned across the relevant Level

Descriptor.

INTENDED SESSION

OUTCOMES

WHY?

• The alignment of all qualifications in the Emirates tothe QFEmirates signifies a move from IMPLICITVALUES to EXPLICIT STANDARDS

• From an unspoken currency to tangible evidence ofstudent outputs that can be benchmarked against asingle reference point – QFEmirates

36

• The major type of qualification associated with formal recognition at

each level of the framework.

PRINCIPAL

QUALIFICATIONS

• Provide formal recognition for learners who achieve a multiple set of

cohesive learning outcomes encompassing, in varying combinations, all

five strands of learning outcomes, but not the full combination of learning

outcomes required for a Principal Qualification.

COMPOSITE

AWARDS

• Provide the smallest parcel of cohesive learning outcomes that can be

achieved by learners for formal recognition purposes within the

Framework. It may relate to all or only some of the five strands of learning

outcomes defining the level.

COMPONENT

AWARDS

QFEMIRATES:

QUALIFICATION TYPES

QFEMIRATES:

A TEN (10) LEVEL NQF

Each of the levels represent a

hierarchy of relative difficulty,

complexity and depth.

Levels:

• Simplest: level 1

• Most complex: level 10

• Post-secondary education:

levels 5 to 10

• Compliance or alignment – what is required?

• CAA Standards require that “Program learningoutcomes are APPROPRIATE to the level ofqualifications awarded and CONSISTENT with theQFEmirates”

• The qualifications descriptors are generic statementsof the outcomes of study; thus compliance notpossible but alignment is.

40

• What does this mean for universities andcolleges?

• What evidence is required to demonstratealignment?

Alignment to QFEmirates

41

1. Burden of proof is on the university to evidencealignment and ensure that there is a consistentapproach across the university to alignment.

• Implication is that the University has (shouldhave) an internal protocol and process thatsupports alignment and that is appliedconsistently to all programs before they aresubmitted to CAA for accreditation

Our (the university’s) responsibilities

42

2. The University ensures that a suite ofdocumentation and evidence of supporting practiceis produced that will enable it (the University) toconfirm alignment BEFORE submission to CAA foraccreditation.

• Standard documents/templates should be used

• The documentation will need to address thecriteria in CAA’s Procedural Guidelines forAccreditation, particularly Section 3.

Our (the university’s) responsibilities

43

INPUTS

• Universities will need to test out and be satisfiedthat:

Program learning outcomes align across the SETof outcomes in the qualification descriptor

Program learning outcomes align with the LEVELof outcomes in the qualification descriptor

What will universities need to ensure before submission to CAA?

44

OUTPUTS

Universities will need to be convinced that:

the design and content of the curriculum enablesstudents to acquire the knowledge and skills across theSET of learning outcomes at the correct LEVEL in thequalification descriptor;

The range of learning opportunities and assessmentpractice proposed will enable students to demonstratethe achievement of the SET of learning outcomes at theright LEVEL

What will universities need to be convinced of?

45

• It’s not just a paper exercise

• Implies changes to our curriculum design aswell as approaches to teaching, learning andassessment practice

Implications of this?

46

• No magic formula!

• But there are effective approaches which canbe developed and built upon

How?

47

Actual student learning outcomes align withQFEmirates

• What’s on paper is just that – what’s on paper!!

• Alignment of actual student outcomes requiresplanning, effort and coherent practice in learningteaching and assessment as well as curriculumdesign

Remember our Goal

48

• Bad practice- where the LO’s align to theQFEmirates BUT that's where it ends!

• Very bad practice is where even the LO’s do notalign!

Bad practice and... very bad practice

49

Alignment of existing programs

• Existing programs will need to be re-examinedand changes to current LO’s and underlyingteaching, learning and assessment practice arelikely to be required in order to achievealignment

Alignment of existing and new programs to QFEmirates

50

• Knowledge is no longer “king”- look at thestrands

• Shift of focus to what our students will learn,know and be able to do – NOT what we teachthem

• Assessment covers a range of methods toenable students to demonstrate achievementof the LO’s

Effective practice that supports good alignment

51

• State clear expectations – learners know what they need to achieve

• Describe achievements that are significant, essential and verifiable

• Preferably state one achievement per outcome.

• Refer to learning that is transferable

• Do not dictate curriculum There may be a number of ways to achieve the outcome)

• Try to accommodate the needs of diverse learners

• Represent the minimal acceptable level of performance that a student needs to demonstrate in order to be considered successful

Learning outcomes at the course level should:

In the 1990’s global economies and work place requirements shifted towards broad based transferable skills. For example, the Conference Board of Canada lists the following skills as essential for the workplace:

• Ability to work in teams

• Effective communication

• Ability to solve problems

Global Influences

• Understand what alignment means in thiscontext, the evidence to demonstrate it andpossible changes to underlying practice relatingto teaching, learning and assessment to ensureSTUDENT OUTCOMES are aligned to QFEmirates

• Have developed approaches to alignmentincluding an appreciation of how studentoutcomes are aligned ACROSS the LEVELDESCRIPTOR and WITH the LEVEL

Outcomes – delegates will:

54

• In UK, we had to face up to a number ofchanges in some areas of our academicpractice and thinking

• Early days - we got the paperwork right but theunderlying practice did not enable students toachieve the outcomes

Effective practice that supports good alignment

55

• LO’s are those specific outcomes which areobservable through students’ performances andwhich constitute the basis on which we asacademics can make an assessment students’achievements

• LO’s are what students will know and be able todo as a result of engaging in the learning process

Writing LO’s to

align with QFEmirates

56

• Using the program and course alignment guidesyou have been provided with we will workthrough the steps that you will need to take.

Steps to alignment

57

Alignment of Program LO’s

Identify the level of the qualification in QFEmirates and the essentials of each strand at that level

Interpret these for your program as a series of learning outcomes

For each strand develop a summary of how teaching and learning practice will enable students to develop the knowledge, skills and competencies required and how the assessment practice will enable them to demonstrate achievement of the LO’s.

58

QFEMIRATES LEVELS

Remember how we judge the QFEmirates level at which we need to

pitch the outcomes for our courses and program?

Two very important factors:

Verbs used

Scope, and

quality of guidance

At Program level, we will have:

a set of learning outcomes which are aligned to a qualification level

in QFEmirates and underlying teaching and learning practice that

enables students to develop the knowledge, skills and

competencies required.

underlying assessment practice that is linked to the learning

outcomes and allows students to demonstrate the achievement of

the learning outcomes.

WHERE ARE WE NOW?

But what about the individual courses which need to contribute to

the program learning outcomes?

Alignment of course outcomes with program outcomes

Course learning outcomes must:

fulfil the requirements of one or more of the program learning outcomes;

be specific enough to describe what a student will be able to do

determine the content, delivery and assessment of each course; and

along with other courses meet the program outcomes

Alignment must be planned and tested to ensure all program outcomes areaddressed;

The contribution of each course to the overall program outcomes must be calculated

61

HOW TO ALIGN ACTUAL STUDENT

OUTCOMES - 5 STEPS TO SUCCESS

This is the responsibility of your colleagues who form the program team.

1Identify essential elements of each strand at the level of the qualification –

Bachelors degree = level 7.

2Interpret these for your subject and program and turn them into program learning

outcomes – the essentials form the basis for the program learning outcomes.

3

Decide how individual courses at each level of the program – so for a bachelors

degree this is levels 5, 6 and 7 – will develop the breadth, depth of the essential

elements through the program and collectively contribute to students achieving

the learning outcomes – use map or matrix.

4Design and plan assessment that enables students to demonstrate they have

acquired the learning outcomes – needs to be done across the program.

5Be able to describe and demonstrate the nature of change between the levels (L5

and L6) so that student outcomes at level 7 match those in QFEmirates.

PROCESS TO DEMONSTRATE PROGRAM

ALIGNMENT WITH QFEMIRATES

Institutional/College Program

Review Committee

Program team to modify

curriculum

Negative alignment and need

for curriculum revision

CAA Substantive Change

Submission

Next CAA Program

Accreditation Review

Report changes Report no changes

Oversight and

guidance from

Institutional

QA Office

Positive alignment with

appropriate QFEmirates level

Outcomes Matrix Map against

QF level descriptors

National Standards of Learning Outcomes for Master

Program (QFEmirates level 9)

Learning Outcomes of Masters in Human

Reproductive Biology

KNOWLEDGE LO 1 LO2 LO3 LO4 LO5 LO6 LO7

1. Comprehensive and highly specialised knowledge in

a field of work, discipline and/or professional

2. Practice, and at the interface between different fields,

including frontier concepts and recent developments

3. Advanced knowledge of applicable research

principles and methods

4. Critical awareness of knowledge issues, as the basis

for original thinking; encompassing appropriate

processes of enquiry and current processes of

knowledge production

5. Detailed body of knowledge of recent developments

in a field of work, and/or discipline

OUTCOMES MAPPING MATRIX

• Program learning outcomes must align to the allessential elements in each strand so thatstudents achieve the SET of outcomes thatcomprise the qualification descriptor at the rightLEVEL

• Universities must have a process that allowsthem to confirm alignment in a consistentmanner

Conclusions

65

• Underlying learning, teaching and assessmentpractice must direct attention to enablingstudents to achieve and demonstrate theachievement of the LO’s if alignment is to bemore than just a paper exercise.

• University program documentation mustevidence this.

• Role of CAA accreditation should be to verifywhat the university is doing.

Conclusions

66

QUESTIONS

Level Knowledge Skill

Aspects of Competence

Autonomy and

responsibilityRole in context Self-development

10

comprehensive, deep

and overarching

knowledge at the

frontier of a

professional field of

work or discipline and

at the interface

between different fields

or disciplines

new knowledge, as

judged by independent

experts applying

international standards,

created through

research or scholarship,

that contributes to the

development of a field

of work or discipline

a range of mastered skills and

techniques, including

synthesis, evaluation,

planning and reflection,

required to extend and

redefine existing knowledge

or professional practice or to

produce original knowledge

advanced skills in developing

innovative solutions to critical

problems in research using

highly developed cognitive

and creative expert skills and

intellectual independence

highly developed expert

communication and

information technology skills

to present, explain and/or

critique highly complex and

diverse matters to specialist

academic, peer

specialists/experts and/or

professional audiences

can act with substantial

authority, creativity,

autonomy,

independence, scholarly

and professional integrity

in a sustained

commitment to the

development of new

ideas or processes or

systems in challenging

and novel work or

learning contexts

can account for overall

governance of processes

and systems can lead

action to build and

transform socio-cultural

norms and relationships

can originate and

manage complex

professional processes

can lead and take full

responsibility for the

development and

strategic deployment of

professional teams and

self

can initiate and deploy

qualities associated with

professional

leadership of peer groups

and teams

can analyse and critique

the state of learning in a

specialised field and

contribute to its

advancement

can self-evaluate and

lead contributions to

professional knowledge,

ethics and practice

including in unfamiliar

and unpredictable

learning contexts

can consistently and

sensitively manage highly

complex and diverse

ethical issues leading to

informed, fair and valid

judgements



Autonomy and


9

comprehensive, highly

specialized knowledge

in a field of work,

discipline and/or

professional practice,

and at the interface

between different fields,

including frontier

concepts and recent

developments

advanced knowledge of

applicable research

principles and methods

critical awareness of

knowledge issues, as

the basis for original

thinking; encompassing

appropriate processes

of enquiry and current

processes of knowledge

production

detailed body of

knowledge of recent

developments in a field

of work, and/or

discipline

advanced skills required in research,

analysis, evaluation and/or innovation of

complex ideas, information, concepts

and/or activities

skills to develop new knowledge and

procedures and to integrate knowledge

from different fields using highly

developed cognitive and creative skills

and intellectual independence to the

field of work or discipline

advanced problem-solving skills to

analyse highly complex issues

with incomplete data and develop

innovative solutions and proposals

relevant to an academic/professional

field, field of work or discipline

planning skills to develop and execute a

major project or comparable activities

(that includes a significant range of

variables and complexity) with

appropriately selected research

methodologies producing sound

conclusions

highly developed specialist

communication and information

technology skills to present, explain

and/or critique highly complex matters

can function

autonomously and/

or take responsibility

for managing

professional practices,

work, processes or

systems, or learning

contexts that are

highly complex,

unpredictable and

unfamiliar, and

require new strategic

approaches and/or

intervention or

conceptual abstract

solutions

can account for high

level governance of

processes and systems

can analyse and reflect

on socio-cultural

norms and

relationships

and act to build and

transform them

can initiate and

manage professional

activities that may

include a highly

complex

environment

can take responsibility

for leading the

strategic performance

and development of

professional

teams and self


take responsibility for

contributing to

professional

knowledge and

practice including

unfamiliar learning

contexts

can develop and

implement further

learning consistently

and sensitively

can consistently and

sensitively manage

highly complex ethical

issues leading to

informed, fair and valid

decisions

Level Knowledge SkillAspects of Competence

Autonomy and


8

advanced specialised

knowledge and critical

understanding in a

specialised field of work or

discipline and at the

interface between fields

comprehensive

understanding of

critical approaches to

creating a systematic and

coherent body of

knowledge and concepts

gained from a range of

sources

comprehensive knowledge

of current research and

innovations in a field of

work, discipline or

professional practice and

impact of these

developments on

accepted theory and

practice

problem-solving skills applied

to a specialist field and the

integration of knowledge from

different fields of work or

disciplines to solve complex

unpredictable and/or abstract

problems with intellectual

independence

identify appropriate sources of

information or analytical

techniques in investigations

that lead to conclusions and

solutions to problems

critical selection of appropriate

research instruments and

strategies associated with the


highly developed advanced

communication and

information technology skills

to present, explain and/or

critique substantively complex

matters


for designing and

developing creative

approaches to

managing and

evaluating complex

work processes and

organisation, resources

or learning, including

leading and managing

teams within a

technical or

professional activity or

working effectively as

an individual

can express a

comprehensive,

internalised, personal

world view, while

accepting responsibility

to society at large and

to sociocultural


can manage professional

activity that may be in a

complex environment


for leading the strategic

performance of

professional teams and

self

can coordinate peer

relationships with

qualified practitioners

and lead multiple,

complex groups

can initiate and support

the management of

professional

development mentoring

activities



contributing to

professional practice in

complex and

sometimes unfamiliar

learning contexts



maintaining and

enhancing currency in

the profession

or discipline

can lead, contribute

and implement

ethical standards



Autonomy and


7

specialised factual and

theoretical knowledge and an

understanding

of the boundaries in a field of

work or discipline,

encompassing a broad and

coherent body of knowledge

and concepts, with substantive

depth in the underlying

principles and theoretical

concepts

an understanding of allied

knowledge and theories in

related fields of work or

disciplines and in the case of

professional disciplines

including related regulations,

standards, codes, conventions

understanding of critical

approach to the creation and

compilation of a systematic and

coherent body of knowledge

and concepts gained

from a range of sources

a comprehensive understanding

of critical analysis, research

systems and methods and

evaluative problem-solving

techniques

familiarity with sources of

current and new research and

knowledge with integration of

concepts from outside fields

technical, creative and analytical

skills appropriate to solving

specialised problems using

evidentiary and procedural based

processes in predictable and new

contexts that include devising and

sustaining arguments associated

with a field of work or discipline

evaluating, selecting and applying

appropriate methods, procedures

or techniques in processes of

investigation towards identified

solutions evaluating and

implementing appropriate research

tools and strategies associated with

the field of work or discipline

highly developed advanced

communication and information

technology skills to present, explain

and/or critique complex and

unpredictable matters

can take responsibility for

developing innovative and

advanced approaches to

evaluating and managing

complex and unpredictable

work procedures and

processes, resources or

learning

can manage technical,

supervisory or design

processes in unpredictable,

unfamiliar and varying

contexts

can work creatively and/or

effectively as an individual, in

team leadership, managing

contexts, across technical or

professional activities

can express an internalised,

personal view, and accept

responsibility to society at

large and to socio-cultural


can function with full

autonomy in technical and

supervisory contexts and

adopt para-professional roles

with little guidance

can take responsibility for the

setting and achievement of

group or individual outcomes

and for the management and

supervision of the work of

others or self in the case of a

specialisation in field of work

or discipline

can participate in peer

relationships with qualified

practitioners and lead

multiple, complex groups


managing the professional

development and direct

mentoring of individuals and

groups

can self-evaluate and take

responsibility for contributing

to professional practice, and

undertake regular

professional development

and/ or further learning

can manage learning

can manage learning tasks

independently and

professionally, in complex and

sometimes unfamiliar

learning contexts

can contribute to and observe

ethical standard



Autonomy and


6

specialised factual knowledge and an

understanding of the boundaries in a

field of work or discipline,

encompassing a broad and coherent

body of knowledge and concepts, with

depth in the underlying understanding

of the principles and concepts

an understanding of allied knowledge

and theories in related fields of work or

disciplines and in the case of para-

professional respective discipline

including related regulations,

standards, codes, conventions

an understanding of critical approach

and analysis, research approaches and

methods and analytical problem-

solving techniques from a range of

sources familiarity with sources of

current and existing knowledge and

the integration of concepts from

related fields

literacy to comprehend and/or

produce coherent texts, covering

complex and/or diverse relations from

a wide-range of information

numeracy covering a wide-range of

mathematical procedures and

representations used across a broad-

range of contexts

specialist technical, creative and

conceptual skills appropriate to

solving complex problems associated

with a field of work or discipline

a comprehensive range of specialist

cognitive and practical skills

appropriate to planning and

implementing solutions to varied,

unpredictable and unfamiliar

problems within a field of work or

discipline

selection and use of appropriate

research tools and strategies

associated with the field of work or

discipline

advanced communication and

information technology skills to

present, explain and/or critique

interdependent complex matters

literacy skills to comprehend and/

or produce, from a wide-range of

information, coherent texts covering

complex and/or diverse

relations

numeracy skills to select, apply,

assess and communicate a wide

range of mathematical procedures

and representations in a broad-range

of contexts


developing appropriate

approaches to managing

complex work procedures

and processes, resources or

learning, including

leading teams within a

technical or professional

activity with little

support

can supervise technical,

supervisory or design

processes in varied,

unpredictable, unfamiliar

and a broad-range of

contexts

can work effectively as a

specialist or in team

leadership roles


personal world view,

reflecting engagement in

society at large and in

socio-cultural relationships

can function with full

autonomy in technical and

supervisory contexts and

adopt para-professional

roles under guidance


the setting and

achievement of group

outcomes and for the

supervision of the work of

others


supervising the

development of

individuals and groups

can participate in peer

relationships with qualified

practitioners and lead

multiple groups

can evaluate own learning

and identify learning

weaknesses and needs, in a

familiar and unfamiliar

environment

can take initiative to

address learning

needs and function

independently and

within learning groups

can support and observe

ethical standards



Autonomy and

responsibilityRole in context

Self-

development

5

comprehensive, specialised

knowledge within a broad field

of work or discipline, including

an understanding of the

underlying theoretical and

abstract concepts with significant

depth in some areas

a broad understanding of allied

knowledge and theories in

related fields of work or

disciplines including related

regulations, standards, codes,

conventions and procedures

an understanding of information

assembly, retrieval methods and

logical problem-solving

techniques from a range of

sources

recognition of sources of current

knowledge and the integration of

concepts from related fields

literacy to comprehend and/or

produce coherent texts covering

complex relations from an array

of information and contexts

numeracy covering an array of

mathematical procedures and

representations and contexts

technical, creative and conceptual

skills appropriate to solving

a wide-range of problems

associated with a field of work or

discipline that include a

comprehensive range of specialist

cognitive and practical skills

appropriate to diagnosing and

implementing solutions to

abstract, familiar and non-routine

problems within a field of work or

discipline

use of appropriate information

retrieval methods and tools and

techniques associated with the


comprehensive communication

and information technology skills

to present, explain and/or critique

complex matters literacy skills to

comprehend and/or produce,

from array of information,

coherent texts covering complex

relations

numeracy skills to select, apply,

reflect and communicate an array

of mathematical procedures and

representations and contexts


coordinating the implementation

of appropriate approaches to

complex work procedures and

processes, resources or learning,

including leading teams within a

technical or para-professional

activity

can exercise coordination and/

or supervision in routine, familiar

and some non-routine work or

learning contexts can coordinate

technical, design processes in

routine, familiar, nonroutine and

an array of contexts with support

available, if required


personal world view, in the

context of an understanding of

socio-cultural relationships

can function with autonomy

in technical and coordination

contexts and support

paraprofessional roles under

guidance

can function both

independently and in a

coordination role with

multiple groups


coordinating the

development of

individuals and groups

can review and develop the

performance of self and

others

can evaluate own

learning and

identify learning

needs in a familiar

environment

can take

responsibility for and

plan own learning

within a managed

and non-routine

environment

can comprehend and

observe ethical

standards

Level Summary Level Descriptors

10

Learning outcomes at Level 10 indicate a systematic mastery of a highly specialised field of knowledge that is

comprehensive, deep and overarching and at the frontier of a professional field of work or discipline, with the

capacity for critical analysis, evaluation and synthesis of

new and complex ideas. It also includes:

conceptualising, designing, implementing and adapting substantial research processes using highly

developed cognitive and creative expert skills and intellectual independence

leadership experience and expertise in the development of new and creative approaches that extend or

redefine existing knowledge or professional practice, encompassing responding with substantial authority and

autonomy to the development of new ideas or processes or systems in challenging and novel work or

learning contexts

accounting for overall governance of processes and systems

analysing and critiquing the state of learning in a specialised field and contribute to its advancement

self-evaluating and leading contributions to professional knowledge, ethics and practice including in

unfamiliar and unpredictable learning contexts

leading and managing complex professional processes

consistently and sensitively managing highly complex and diverse ethical issues leading to informed, fair and

valid judgements.


9

Learning outcomes at Level 9 indicate self-directed, comprehensive, highly specialised knowledge and practical learning,

some of which is at the forefront of knowledge in a specialised field that provides a basis for originality and advanced

knowledge and skills in research, analysis, evaluation and/or innovation encompassing complex ideas, information, concepts

and/or activities and developing and/or applying ideas, often within a research context. It also includes:

advanced problem-solving skills

the integration of and formulation of judgements

taking account of social and ethical issues and responsibilities and reflecting experience of managing change in a highly

complex, unpredictable and unfamiliar context that requires new strategic approaches and/or intervention or

conceptual abstract solutions

planning skills to develop and execute a major project/activity outcome with appropriately selected research

methodologies to produce sound conclusions

presenting, explaining and/or critiquing highly complex matters

managing professional activities in such context

self-evaluating and taking responsibility for contributing to professional knowledge and practice including unfamiliar

learning contexts

developing and implementing further learning consistently and sensitively

consistently and sensitively managing highly complex ethical issues leading to informed, fair and valid decisions.


8

Learning outcomes at Level 8 indicate advanced specialised knowledge and critical understanding in a specialised field of

work or discipline and at the interface between fields as well as a comprehensive understanding of critical approaches to

creating a systematic and coherent body of knowledge and concepts gained from a range of sources. It also covers:

acquiring comprehensive knowledge of current research and innovations in a field of work, discipline or professional

practice

specialist field and integration of knowledge from different fields of work or disciplines

skills in solving complex unpredictable and/or abstract problems with intellectual independence and making

judgements that take into account social or ethical issues

skills to adopt a professional approach to operating in a complex environment including the design and development

of creative approaches to the management of complex work processes and organisation, resources or learning

leading and managing teams within a technical or professional activity or working effectively as an individual

leading the strategic performance of professional teams and self

presenting, explaining and/or critiquing substantively complex matters

taking responsibility for contributing to professional practice in complex and sometimes unfamiliar learning contexts

leading, contributing and implementing ethical standards.


7

Learning outcomes at Level 7 indicate knowledge and critical understanding of the well-established principles and practice in

a field of work or discipline and is specialised factual and theoretical. It includes an understanding of the boundaries in a field

of work or discipline, encompassing a broad and coherent body of knowledge and concepts, with substantive depth in the

underlying principles and theoretical concepts. It also covers:

allied knowledge and theories in related fields of work or disciplines and in the case of professional respective discipline

using methods of enquiry, critically analysis and selection of different approaches to solving problems

an understanding of the limits of the knowledge and skill acquired

technical, creative and analytical skills to solve specialised problems using evidentiary and procedural based processes

in predictable and new contexts that include devising and sustaining arguments associated with a field of work or

discipline

evaluating and implementing appropriate research tools and strategies

evaluating and managing complex and unpredictable work procedures and processes, resources or learning

managing technical, supervisory or design processes in unpredictable, unfamiliar and varying contexts

presenting, explaining and/or critiquing complex and unpredictable matters

self-evaluating and responsibility for contributing to professional practice

undertaking regular professional development

contributing to and observing ethical standards.


6

Learning outcomes at Level 6 indicate specialised factual knowledge and an understanding of the boundaries in a field of

work or discipline and an understanding of allied knowledge and theories in related fields of work or disciplines. In the case of

the para-professional respective

discipline, it also covers:

applying specialist technical, creative and conceptual skills in planning and developing strategic solutions to varied,

unpredictable and unfamiliar problems, be they abstract and/or concrete

using appropriate research tools and strategies

managing with autonomy complex work procedures and processes, resources or learning, including leading or

supervising teams within a technical or professional activity with little support and adopting, and where appropriate,

para-professional roles under guidance

presenting, explaining and/or critiquing interdependent complex matters

producing from a wide-range of information, coherent texts covering complex and/or diverse relations

selecting, applying, assessing and communicating a wide-range mathematical procedures and representations in a

broad-range of contexts

drawing on experience of operational interaction in work or learning including supervision of people and projects

addressing own learning needs and function independently and within learning groups

supporting and observing ethical standards.


5

Learning outcomes at Level 5 indicate knowledge is comprehensive and specialised within a field of work or discipline and

encompasses the underlying theoretical and abstract concepts with significant depth in some areas as well as a broad

understanding of allied knowledge and

theories. It also covers:

technical, creative and conceptual skills appropriate to solving a wide-range of problems including diagnosing and

implementing solutions to abstract, familiar and non-routine problems within a field of work or discipline and using

appropriate information retrieval methods, tools and techniques independently with autonomy

supervising others or coordinating subordinates and peer groups

supporting para-professional roles under guidance

presenting, explaining and/or critiquing complex matters

producing from information, coherent texts covering complex relations

applying, reflecting and communicating an array of mathematical procedures and representations and contexts

self-direction in learning and have experience of practice in both common and exceptional situations

comprehending and observing ethical standards.

Developing Program and Course Learning Outcomes in ... · PDF fileCourse Learning Outcomes in...

Documents

Transcript of Developing Program and Course Learning Outcomes in ... · PDF fileCourse Learning Outcomes in...