Supporting the Design of Discipline-Specific Learning Outcomes:

Experiences of the Tuning Group for Physics

Gareth Jones

Imperial College London

OUTLINE• The Tuning Project

– What, Why, Who?– Competences and Learning Outcomes– Hierarchy of Learning Outcomes and link to Level and Standards– Surveys and Results

• Degree Programme (Re)Design– Main Requirements– A fresh start or improve what exists– Incorporating competences and content requirements

• Specific Examples– IOP Accreditation requirements for Physics degrees– Example of Module and Thematic Learning Outcomes

What is the Tuning Project?

• The universities response to the Bologna Process: Most work done by separate but coordinated teams of discipline experts each with one member from each EU country

• To find ways to implement a three-cycle degree structure

– To develop learning outcomes and competences for each cycle (reference points) on basis of consensus after much discussion

– To survey views of students, graduates, academics and employers on importance of both generic and subject specific competences

– To survey and compare programme content and structure

• Development of ECTS as a credit accumulation system

• Best Practice in teaching & learning and quality enhancement

• Tuning Coordinators/Leaders: Julia Gonzalez & Robert Wagenaar

• Tuning Physics Group Leader: ‘Lupo’ Dona dalle Rose

From the Tuning Final Report

Two of the key driving ideas of the Tuning Project

• One of the main objectives of the Bologna process is to make study programmes and periods of learning more comparable and compatible. This objective is strongly promoted by making use of the concept of levels, learning outcomes, competences and ECTS credits.

• The Tuning emphasis on competences and learning outcomes is intrinsic to the paradigm shift from a professor-centred to a student-centred approach which is seen as a key way of improving the effectiveness of European HE.

Competences• Ability to do something. • Competences range from:

– specific and small, e.g. competence to use an oscilloscope, to – general and large, e.g. competence to solve problems

• Acquired by students and assessed either in a specific part of a course or throughout programme in an integrative, holistic way

• Learning Outcomes often expressed in terms of competences (but not all)

• Generic Competences, e.g. general cognitive abilities, interpersonal skills

• Subject Specific Competences– Competences required and/or valued by profession/discipline– Different universities may emphasise particular competences and de-

emphasise others Profile of degree

Examples of Generic Competences from Tuning

• Ability to apply knowledge in practical situations• Capacity for analysis and synthesis• Capacity to learn• Creativity• Adaptability• Critical and self-critical abilities• Concern for quality• To act in accordance with a basic knowledge of the profession

Tuning Survey 2008 – Employers’ Response:Most important generic competencesTuning Survey 2008 - General Competences of Graduates - Employers' Response

0

0.5

1

1.5

2

2.5

3

3.5

4

Apply Knowledgein Practice

Identify, Pose andResolve Problems

Determination &Perserverence

Oral & WrittenCommunication

Teamwork Make ReasonedDecisions

Stay up-to-datewith Learning

Competence

Sco

re Importance

Achievement

Physics Specific Competences/Learning Outcomes

• Able to enter new fields through independent study• Familiar with the ‘work of genius’, i.e. with the variety and delight of physical

discoveries and theories, thus developing awareness of the highest standards

• Have a good understanding of the most important physical theories including a deep knowledge of the foundations of ‘modern’ physics

• Able to evaluate orders of magnitude in situations which are physically different but show analogies

• Able to understand and master the most commonly used mathematical and numerical methods

• Able to perform calculations, including the use of numerical methods and computing, to solve problems

• Able to construct mathematical models of a process/situation by identifying the essentials of a process/situation and making justified approximations

• Have a good knowledge of at least one frontier physics specialty

Physics Specific Competences/Learning Outcomes (Practical/Experimental/Research)

• Able to perform experiments independently, as well as to describe, analyze and critically evaluate experimental data and to be familiar with the most important experimental methods

• Understanding of the nature and methods of physics research and how it can be applied in other fields e.g. engineering

• Familiar with the ‘culture’ of physics research, including the relation between experiment and theory and ability to span many areas

• Able to find physical and technical information relevant to research work and technical project development using literature search methods

• Able to work with a high degree of autonomy, accepting responsibility in planning and managing projects

• Able to carry out professional activities in the area of applied technologies and industry

Physics Specific Competences (Human Dimension)

• Able to present one’s own results (research or literature search) to professional and lay audiences orally and in written form using appropriate language

• Able to work in interdisciplinary teams

• Prepared to compete for school teaching positions in physics

• To show a personal sense of responsibility, e.g. meeting deadlines, and to show professional flexibility

• To behave with professional integrity and an awareness of the ethical aspects of physics research and its impact on society

Tuning Survey on Competences 2008

Opinions on the most important Physics Specific Competences Employers Graduates Students Academics Ability to enter new fields Ability to enter new

fields Ability to enter new fields

Mathematical Skills

Modelling Skills Experimental Skills Deep Knowledge & Understanding

Estimation Skills

Problem Solving Literature Search Mathematical Skills Deep Knowledge & Understanding

Estimation Skills Estimation Skills Problem Solving Ability to enter new fields

Foreign Language Skills Deep Knowledge & Understanding

Modelling Skills Experimental Skills

Managing Skills Specific Communication Skills

Experimental Skills Problem Solving

Tuning Survey of Competences 2008 - Physics

2

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

Competence

Imp

ort

ance

Sco

re

Employers

Graduates

Students

Academics

Learning Outcomes: What and Why?

• Statements of what students should know, understand or be able to do as a result of following a course– Knowledge and understanding

– Problem solving

– Skills: experimental, mathematical, design, …

– Ability at communication, teamwork etc.

• Use in defining levels: 1st and 2nd cycle level descriptors

• Part of Bologna Process and Qualification Frameworks

• Use in Programme Design & QA methodology– What education is all about

– Must be assessed

Hierarchy of Learning Outcomes• Module Level Learning Outcomes

– Specified by Module Teacher and Programme Director

– Should/must be assessed mark or grade

– Desired and threshold Learning Outcomes criteria

– Need to be specific but not too detailed

• Thematic Learning Outcomes, e.g. Quantum Mechanics– Refer mainly to overall or final abilities. Forest not the trees

• Year Learning Outcomes: useful for progression criteria• Programme Learning Outcomes, e.g. BSc (Hons) in

Physics– General and summative statements

– Holistic

– ‘Dublin Descriptor’ type statements but applied to discipline Refer to Academic Level

Academic Level and Learning Outcomes

• Intended Learning Outcomes give a good indication of competence for performing particular tasks, but:– Need to be fairly specific, e.g. able to use time dependent

perturbation theory to solve problems in atomic and nuclear physics. But:

• What kind of problems?• How difficult?

– Need to refer to how assessed, e.g. exam questions.

• Learning Outcome statements for programmes are not enough to compare standards. How do you add up Learning Outcomes? Need to specify content/volume.

Are Learning Outcomes Helpful?

• Can be very helpful for programme design – Focus mind on “What are the students getting out of it?”

• Can improve teaching and the output competences of graduates

• How to assess whether or not they are achieved?– Exams OK for academic problem solving but not so good for

realistic problem solving

– Difficulty of questions is crucial for standards but is hard to control and interpret

– Mark Scale: Raw data for testing hypothesis “Has this LO been achieved?” but what is threshold mark?

– Practical competences easier to test

Traditional Programme Design

• (Professor) i (Course) i

– “I will teach them what I know”

• Programme = Σ (Course) i

• Leads to content and professor dominated curriculum

• Danger of – Content overload and excessive ‘derivations’– Obscurity of purpose: “Why are we doing this?”– Little increase in competence

• Advantages (if have good professors!):– Produces deep understanding for best students– Good for producing future professors!!!

The Programme Design Problem

• An existing module synopsis can be basis for a list of Learning Outcomes for that module

• The general characteristics of a degree programme can be defined by Qualifications Framework statements

• But what goes in the middle?– Subject specific qualification and level descriptors (Benchmark)

– Thematic Learning Outcomes

– Structuring of content to ensure linkage and progression

– Development of teaching, learning and assessment methods to enable learning outcomes to be achieved and assessed holistically

• Construction of a matrix of competences vs. modules is very helpful– Helps to ensure competences appear explicitly in the design

Matrix of Competence vs. Content

Knowledge & Understand

Apprec.work of Genius

Problem

Solving

Maths

Skills

Experimental

Skills

Communication Skills

Mechanics

& Relativity50% 10% 30% 10%

Maths 1 20% 30% 50%

1st Yr Lab 10% 10% 50% 30%

Quantum Physics

60% 20% 15% 5%

Professional Skills

20% 10% 30%

Steps in Physics BSc Programme Design

• INPUTS– (a) IOP Accreditation Requirements and QAA Benchmark statement– (b) National Framework of Qualifications (NQAI)– (c) Desired Qualification Profile (e.g. Applied, Pure,…)– (d) Desired/expected student intake and potential employers– (e) Resources and existing degree programme modules– (f) Tuning results on Competences, Learning Outcomes, Content, …

• PROCESS– Internal Discussion: where we are → where we want to be, SWOT– Construct Matrix of Competences vs. Modules, using (a), (b), (f)– Check (c), (d), (e)– Develop Learning Outcomes for whole programme, themes and modules– Check academic level– Develop Teaching and Learning Methods and Assessments– ITERATE! Will it work? Does it meet requirements? Is it realistic?– Seek wide support and administrative approval

Use of Learning Outcomes in Practice(‘Reverse Engineering’)

• Start from where we are now– LO’s for each module: Improve them, check how assessed– Examine content: remove redundancies, add missing items– Check accreditation, benchmark, Tuning competences are met– Construct matrix of competences vs. modules– Iterate! It is likely there are gaps or deficiencies– Construct more generalised LO’s for themes, years, programme– Ensure logical progression, e.g. C depends on A and B– Check requirements of NQAI. Check academic level.– Iterate, again! Pay particular attention to assessment and

recent student results (marks, drop-out rates, employment, …)

• Present new programme for approval

Example of Approaches to Teaching & LearningTuning Physics Group

Modelling (second cycle)

• Modelling in a narrow sense means finding a simplified mathematical description of a complex phenomenon. It often means also applying tools of theoretical physics to non-physics situations.

• … There is no course unit named Modelling. Students learn the modelling description of nature throughout their whole degree-course. Possible examples are: the “modelling” neglect of friction in the description of free fall, the abundant use of harmonic oscillator for phenomena in the neighbourhood of stable equilibria, the shell model average field for nucleons in nuclei, the modelling of two-nucleon and three-nucleon forces, and so on.

• The whole teaching offer is then important: in lectures, exercise classes, in lab classes, in student seminars and during research training students learn about how theories were developed, how to select and then apply theoretical tools (e.g. models) to a particular physical problem and how to model the building blocks of a theory, by adapting these latter to the experimental data description.

Example of Approaches to Teaching & LearningTuning Physics Group

Problem solving skills (first cycle)

Active Learning: in all classes (theory, lab or problem solving)• Several questions are posed to the theory class and a certain amount of time is

allowed for discussion in the same class.• Several question-problems are set to the class and assigned to groups of students.

They should find an answer (either exact or approximate) in a certain amount of time. They are also requested to explain their reasoning to other students (Did they divide the problem in simpler problems? did they use analogies with problems, for which they already knew the answer? why are they confident about their own answer?…)

• In the exercise classes the students are requested to correct and comment other students ways of solving the exercises.

• In the lab classes students are frequently asked to solve experimentally or propose ways for solving other more complex problems that may be considered extensions of the material proposed in the class. (ex: after studying an LC circuit they are encouraged to solve the problem of coupled LC circuits and think about the problem of impedance adaptation in a transmission line).

IOP Accreditation Requirements

• The degree programme should foster intellectual curiosity in the minds of students

• Graduates should have acquired– A secure knowledge of an agreed core of physics + a few extra

frontier topics– Competences represented by ‘graduate skills base’

• The degree programme must incorporate project work– BSc level project work may be a dissertation– MSc/MSci level project work must involve research skills

• The degree programme must be consistent with QAA Benchmark

IOP Graduate Skills Base(Part of Programme Learning Outcomes)

• Physics Skills: Physics students should be able to– Tackle problems in physics– Use mathematics to describe the physical world – Plan, execute, analyse and report experiments– Compare results critically with predictions from theory

• Transferable Skills: A Physics degree should enhance– Problem solving skills (well defined and open-ended)– Investigative skills– Communication skills– Analytical skills– IT skills– Personal skills (group work, use of initiative, meet deadlines)

Graduates should have a secure knowledge of the IOP Core of Physics

• Mathematics for Physicists• Mechanics and Relativity• Quantum Physics

– including atomic, nuclear and particle physics

• Condensed Matter Physics• Oscillations and Waves• Electromagnetism• Optics• Thermodynamics and Statistical Physics

Example of Module and Thematic LO’s

• 1st Year Mechanics Module LO’s (selection)– Understand the concept of conservative force and its relation to the

potential function (in 3 dimensions)– Be able to solve single particle motion from a given potential function in

two dimensions– Be able to use angular momentum and energy conservation in central

force problems

• Can be tested by answers to exam questions but how to interpret exam marks– Not just “Yes or No” but partial “Yes”– Index of “cleverness” or speed of working

• Thematic Learning Outcome for Mechanics– Able to use Newton’s Laws in a wide range of areas of physics– Aware of the power of conservation laws– Aware of more advanced methods of Lagrangians etc.

Conclusions• The traditional approach to programme design stresses content too

much and does not pay sufficient attention to the change we are trying to produce in students in terms of their competences.

• A Learning Outcomes approach requires a re-thinking of why, what and how we teach and of how we assess students’ achievements.

• It will require more effort initially from teachers but will probably enable reductions to be made in the amount of content taught.

• Students must be given more scope for activities like problem solving, team-work and communications but also must accept more responsibility for their own learning.

• The Learning Outcomes approach is firmly embedded in the Bologna Process. Tuning has shown how it can be used in a Pan-European way