Balancing Quality Enhancement and Accountability

Reforming the Dutch and Flemish accreditation system

Stephan van Galen

INTRODUCTION: HISTORICAL BACKGROUND

1985 Universities in the Netherlands and Flanders agree on external

peer review of all degree programmes

Programmes reviewed every 6 years

Focus on Quality enhancement

2002 introduction of bachelor master structure

2003 introduction of programme accreditation

Accreditation: Validation of external peer review by NVAO as an

independent government body

Focus on Accountability

1. REFORMING ACCREDITATION

Point of departure:• The balance between enhancement and accountability should be restored;• The system should focus more on the content of the programmes and less on

procedural aspects;• There is a pressing need to reduce red tape;• Academic ownership of the system should be increased.

Conditions:• Accreditation remains on the level of the programme;• International acceptance of the system: Compliance with European Standards and

Guidelines (ESG);• An assessment framework with generic criteria: respecting institutional autonomy

and diversity in degree programmes;• Limiting the number of standards;• Cutting down on bureaucracy;• All decisions and reports are public and informative;• NVAO appoints peer panels.

2. REFORMING ACCREDITATION: GOALS

1. Stimulating a Quality Culture in Higher Education Institutions (HEI) by introducing an Institutional Audit (IA);

2. Committing the professionals (teachers);

3. Focus on learning outcomes and the programme;

4. Stimulating HEI to aim above threshold quality;

5. Providing Accountability and Quality Enhancement;

6. Introducing a light touch approach based on earned trust;

7. Reducing administrative burdens with 25%.

3. OVERVIEW OF THE NEW SYSTEM (NL)

Institutional Audit

Positive Negative

Comprehensive programme assessment

Limited programme assessment

Positive (initial )accreditation decision

Negative (initial)accreditation decision

Yes No

4A. INSTITUTIONAL AUDIT

Focus on the functioning of the (internal) Quality Assurance System of HEI with respect to teaching and learning

Institutional audits involve five related questions:

1.What is the vision of the institution regarding the quality of the education it provides?

2.How does the institution intend to realise this vision?

3.How does the institution measure the degree to which this vision is realised?

4.How is the institution working towards effecting improvements?

5.Who is responsible for what?

4B. INSTITUTIONAL AUDIT    

5. COMPREHENSIVE PROGRAMME ASSESMENT

• 16 topics;• Judgements on a four point scale: Unsatisfactory, satisfactory, good,

excellent;• A detailed assessment of content, policy and procedures;• Data to facilitate the dialogue between peers and programme;• Peer panel selected by HEI, NVAO has to give its consent.

Topics :

1. Intended learning outcomes.

2. Programme.

3. Staff.

4. Services.

5. Quality assurance system.

6. Achieved learning outcomes & Learning assessment.

6. LIMITED PROGRAMME ASSESSMENT

• 3 topics;• Judgements on a four point scale: Unsatisfactory, satisfactory, good,

excellent;• Assessment of the content of the programme with a focus on

performance;• Data to facilitate the dialogue between peers and programme;• Peer panel selected by HEI, NVAO has to give its consent.

Topics:

1. Intended learning outcomes.

2. Programme and staff quality.

3. Achieved learning outcomes & Learning assessment.

7A. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT

Intended learning outcomes

Standard 1The intended learning outcomes of the programme have been set down in concrete terms and satisfy international requirements regarding content, level and orientation.

ExplanationIn terms of level and orientation (bachelor’s or master’s programmes; with a professional or academic orientation) the intended learning outcomes are in keeping with the Dutch or Flemish qualifications framework. Furthermore, they are in line with the current requirements set by the occupational field and the discipline from an international perspective in terms of the programme content.  DecisionUnsatisfactory, satisfactory, good, excellent.

7B. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT

Teaching and learning environment

Standard 2The programme, the staff and the programme-specific facilities make it possible for admitted students to realise the intended learning outcomes. ExplanationThe content and structure of the programme allow the admitted students to achieve the intended learning outcomes. The quality of the staff and of the programme-specific facilities are essential to this end. For the students, the programme, staff and facilities constitute a cohesive teaching and learning environment.  DecisionUnsatisfactory, satisfactory, good, excellent.

7C. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT

Learning assessment and learning outcomes realised

Standard 3The programme has an adequate system of exams which demonstrates that the learning outcomes are realised. ExplanationInterim and final exams, final projects and the way in which graduates function in practice or in subsequent education demonstrate what level has been realised. Exams and assessments are valid, reliable and clear to the students.  DecisionUnsatisfactory, satisfactory, good, excellent.

General conclusionThe quality of the programme is: Unsatisfactory, satisfactory, good, excellent.

8A. ASSESSMENT PROCEDURE: DOCUMENTATION

Critical Reflection: a self-reflection which follows the standards of the framework and describes strengths and weaknesses that also indicates any improvement measures that have been taken as a result of the previous assessment.

Mandatory Quantitative information on the programme:1.Intake, transfers and graduates, if possible for the last 6 cohorts;2.Realised teacher-student ratio;3.Average number of face-to-face teaching hours per phase of the programme

8B. ASSESSMENT PROCEDURE: DOCUMENTATION

Mandatory appendices:

1.Domain-specific reference framework and the learning outcomes of the programme;2.Overview of the curriculum; content description of the components of the curriculum, stating the learning outcomes, learning objectives, teaching method(s), testing methods, prescribed literature, teacher and credits;3.Literature list (mandatory / recommended);4.Tuition and examination regulations;5.Overview of the relevant staff listing name, position, nature of the appointment, grade and expertise;6.List of the most recent 25 final theses or the final projects that demonstrate the level achieved by the students).

8C. ASSESSMENT PROCEDURE: DOCUMENTATIONDocuments for inspection during the site visit:

1.Reports of consultations within the relevant bodies;2.Testing statements with accompanying assessment criteria and standards (answer models) and a representative selection of actual completed tests (e.g. presentations, work experience placements, assessments or portfolio) and evaluations;3.A representative selection of final projects from the past two years chosen by the panel together with the accompanying assessment criteria and standards;4.Manuals and other course material;5.Summary and analysis of recent evaluation results and relevant management information.

9. ASSESSMENT PROCEDURE: SITE VISIT

• The mandatory site visit for the limited programme assessment takes one day;

• Prior to the visit, the assessment panel studies several final projects in order to gain insight into the exit level achieved in the programme. To this end, the panel makes a selection from an overview drawn up by the programme;

• During the site visit, the assessment panel speaks to at least:

1. Programme management;2. Members of the examinations board;3. Teachers;4. Students;5. Alumni;6. Representatives from the occupational field if

relevant.

10A. INDEPENDENT PEER PANELS

Programme accreditation:

NVAO sets the rules for panel composition and independence. Peers have to sign a declaration of independence and a code of conduct set by NVAO:•The panel consists of at least four members, one of whom is a student;•The panel comprises at least two authoritative subject-matter experts. At least one of these members has teaching experience at the relevant level of education; •The panel is aware of the latest international developments in the discipline and, in as far as applicable, has expertise in the occupational field; •The panel also has educational expertise and assessment or audit expertise; •The panel is independent. The members have had no ties with the institution offering the programme for at least five years;

10B. INDEPENDENT PEER PANELS

• The panel is supported by an independent, external secretary who has been trained and certified by NVAO;

• Panel composition and declarations of independence will be published and made public;

• The parties involved in the assessment, such as panel members, staff or students, can report any matters that could affect the independence of the assessment panel to NVAO.

Institutional audit and initial programme assessment:

NVAO selects and appoints peer panels in accordance with the rules that apply for panel composition for programme accreditation.

In Flanders the rules for panel composition will not be changed.

11. COMPLIANCE WITH ESG

• No changes in the position of NVAO as an independent organization established by treaty between Flanders and the Netherlands;

• NVAO’s role in guaranteeing the independence and quality of peer panels for the accreditation of programmes is strengthened: HEIs provide suggestions for peers but NVAO has to give its consent. At present this process is in the hands of independent but commercial quality assurance agencies;

• NVAO selects and appoints peers for the Institutional Audit.• Extensive consultations with stakeholders ensures that the new

system is more in tune with day-to-day realities in HEIs and should be more effective in fostering a quality culture;

• The introduction of an institutional audit allows NVAO to take into account the effectiveness of the internal quality assurance processes of HEIs as a whole;

• System wide analyses is added to NVAO’s tasks.