Manual for Assessor Certification Amsterdam University of Applied Science

Amsterdam, February 2019

2

Table of contents Introduction 3

1. Assessor profile 4 1.1 Core responsibilities 4 1.2 Basic assessor competences 4 1.3 Assessment skills 4

2. Aptitude test for assessor certification 5

Introduction 5 2.1 Aptitude test for certification 5 2.1.1 Registration for aptitude test 6 2.2 Assessment and certificate 6 2.2.1 Expert and independent evaluators 6 2.3 Resits 7 2.4 Objections 7

3. Preconditions for certification: responsibility of the higher education (HE) programme 7 3.1 Ongoing professional development 7 3.1.1 Professional development in preparation for the aptitude test 8 3.1.2 Ongoing professional development to retain the assessor certificate 8 3.2 Quality of assessment practice 9

4. Quality 10

4.1 Quality assurance of the aptitude test and assessment 10 4.2 Quality assurance by the AUAS Certification Committee for Lecturer Competences 10 4.3 Quality assurance by the HE programme for which the assessors work 10

5. Costs 10 Sources 11

Appendix 1: Overview of BEQ, SEQ and assessor certification 12 Appendix 2: Explanation of the behavioural indicators in the assessor profile 13 Appendix 3: Assessor evaluator profile 15 Appendix 4: Retention of assessor certification form 16

3

Introduction

Because the level of graduates in higher professional education (HBO) must be undisputed, monitoring has been tightened in recent years. This led to the introduction of personal certification for HBO examiners in 2013. 'After all, the quality of the assessment is largely dependent on the quality of the examiner' (Andriessen et al., 2017, p. 9). In general, the certification of persons pertains to 'the impartial determination of the professional competence of persons to be able to carry out professions, or specified tasks, in accordance with applicable (internationally) established criteria or, in the absence thereof, in accordance with criteria accepted by the parties involved.' (Hobéon Certificering, 2014). Dutch universities of applied sciences have agreed that every lecturer must demonstrate their assessment competence by obtaining a Basic and/or Senior Examiner Qualification (BEQ/SEQ). AUAS lecturers also offers assessor certification (see Appendix 1). This manual is about obtaining this latter certification. Assessors are responsible for evaluating integrated, complex assessment formats, such as portfolio and behavioural assessments (see Appendix 1). For the student's talents to be done full justice during an assessment, expert assessors are a first requirement. For them, the art of assessment starts with sound professional expertise and broad, up-to-date knowledge of professional practice. In addition, training and ongoing professional development are needed to develop and maintain assessment skills, realise a uniform working method and guarantee a generally valid judgement. No less important are the means by which assessors carry out their tasks, such as a workable assessment model and protocols. There is also a need for an unambiguous and workable assessment procedure. All these aspects have been worked out in the assessor certification programme and are explained in this manual, which is intended both for examiners who wish to obtain the assessor certification and for their supervisors. Section 1 discusses the profile of the assessor: the tasks that they1 must carry out and the expertise and behaviour that are expected of them. Section 2 describes the aptitude test in which the expertise and skills of the assessor are assessed for certification. Section 3 contains the conditions for participation in the test. These relate to the achieved and ongoing professional development of assessors and the conditions under which they conduct assessments. Section 4 deals with the quality assurance of the certification programme: a shared responsibility of the AUAS Certification Committee for Lecturer Competences and the higher education (HE) programme the assessors work for. Section 5 provides information on costs.

For more information about the certification programme or participating in it, please email assessor@hva.nl or call +31 (0)6 21 15 58 12.

Amsterdam, February 2019

1 This manual uses gender-neutral pronouns.

4

1. Assessor profile Assessors conduct behavioural and portfolio assessments. They assess whether a student meets the required professional competences. The profile outlines the assessor's core responsibilities as well as the basic competences and skills required to carry out these tasks.

1.1 Core responsibilities The assessor: 1. assesses whether a portfolio meets the requirements2 in order to be able to take part in the assessment interview; 2. conducts a criterion-based interview3 based on the portfolio (also referred to as the assessment or portfolio interview) to determine the level of professional competence; 3. determines the student's professional competence and confirms this to the student; 4. gives development-focused feedback; 5. records the findings and final judgement on the assessment form or in a report.

1.2 Basic assessor competences4 The assessor: - possesses relevant and wide expertise and practical experience in the domain of the HE programme; - works for the HE programme (internal assessor) or is a practitioner in a relevant field (external assessor); - has obtained a higher professional education diploma or a higher level diploma; - is customer-oriented and genuinely interested in the development of students; - is empathetic, i.e. able to empathise with the student; - is aware of the assessor’s own frame of reference and uses this adequately when carrying out the

assessor's tasks; - works efficiently and accurately and observes agreements; - communicates in an open and convincing manner, both verbally and in writing; - is learning-oriented and prepared to invest in personal development.

1.3 Assessment skills The skills that are expected of assessors with regard to their core responsibilities are expressed in eight behavioural indicators. These indicators are the starting point for practising assessment skills in assessor training courses. The indicators are used as assessment criteria for the assessor aptitude test. These indicators have been elaborated in an appendix in terms of characteristic and desirable behaviour.

Behavioural indicators The assessor: 1. creates an atmosphere during the assessment interview in which the student's talents are done full justice; 2. structures the assessment interview; 3. applies different questioning and interviewing techniques adequately and at the right time in order to:

purposefully gauge the student's level of professional competence

maintain control during the interview; 4. relates the experiences, working method and evidence provided by the student to assessment criteria

(assessment triangle):

recognises products (professional and otherwise) as usable evidence 5. reaches a substantiated judgement and conveys this in a convincing manner; 6. gives constructive feedback;

2 In the case of a behavioural/performance assessment, there usually is no portfolio. The assessment interview concerns the behaviour of the student that the assessor has observed. This also applies to the assessor's second task: 'conducts a criterion-based interview on the basis of observed behaviour'. 3 This criterion-based interview is also referred to as a CBI, assessment or portfolio interview. 4 These are competences that assessors have before participating in a training course. They are used as criteria for the selection of assessors.

5

7. records the judgement in an accessible manner; 8. handles any objections to the judgement adequately and in a customer-oriented manner. Appendix 1 gives details of the behavioural indicators in terms of characteristic behaviour.

2. Aptitude test for assessor certification

Introduction The AUAS certification programme for assessors provides for the independent determination of assessor quality and expertise. Assessors demonstrate that they possess the necessary skills in an aptitude test. In case of a positive result, they will receive a - personal - assessor qualification certificate. This section discusses successively the aptitude test and the role of the parties involved in it (Section 2.1), the assessment of the aptitude test (Section 2.2), possibilities for a resit (Section 2.3) and the procedure for objections (Section 2.4).

As mentioned above, expert assessors are important to achieve high-quality assessments. However, they will not be able to carry out their tasks properly if the assessment conditions and materials are not up to standard. Section 3 discusses the preconditions for professional development and assessments. It only makes sense for assessors to sit an aptitude test once these preconditions are met.

2.1 Aptitude test for certification Assessors are certified once they have demonstrated their ability to expertly perform the core responsibilities in the assessor profile. This is done in an aptitude test. As part of the test, assessors must conduct an authentic portfolio or behavioural assessment for the HE programme they work for. An independent assessment expert is present to judge the assessor on the behavioural indicators of the assessor profile. Such a test is also referred to as a performance assessment. It is important to keep in mind that the assessor does not have to carry out any extra work in connection with this test. Three parties are involved in the aptitude test: the assessor sitting the aptitude test, an assessor expert in the role of evaluator and the AUAS Academy. Their respective responsibilities are as follows: The assessor sitting the aptitude test

1. informs the student and co-assessor about the presence of an assessor evaluator prior to the test; 2. shall be present 20 minutes prior to the aptitude test to meet the assessor evaluator and discuss the

following subjects:

– the preparation of the assessor for the assessment: review of the assessor's (written) preparations with the assessor evaluator.

– the information that the assessor wants to receive from the student during the assessment. The assessor should be able to specify and substantiate this in a number of concise questions or themes.

3. takes the lead in the assessment (= aptitude test) to be able to demonstrate all required behavioural indicators in the assessor profile.

The assessor evaluator

1. observes and records their findings during the aptitude test; 2. evaluates the assessor at the end of the test on the tasks carried out and the behavioural indicators

in the assessor profile (Section 1); 3. substantiates their judgement with registered observations and findings from the aptitude test; 4. gives the assessor their judgement orally, supported by observations and findings (= feedback)

immediately after the aptitude test (maximum duration: 20 minutes); 5. makes a sound recording of the test to support the writing of the report, the provision of feedback to

the assessor and the substantiation of their judgement;

6

6. records their judgement and the substantiation thereof within five working days using the appropriate reporting format;

7. submits the report to the assessor and any adjusts any factual inaccuracies; 8. emails the digital final version of the report to assessor@hva.nl, after approval from the assessor

concerned. The AUAS Academy

1. provides a qualified assessor evaluator for the aptitude test; 2. provides the assessor evaluator with the necessary materials to conduct the assessment, including

the assessment model used by the assessor during the test and a format for the assessment report. 2.1.1 Registration for aptitude test In order to participate in the certification programme, an assessor must meet a number of preconditions. These have been elaborated in Section 3. Ongoing professional development is an important part of this. If the conditions have been met, the supervisor will decide in consultation with the assessor concerned when the assessor will be eligible for the test. The following guidelines apply: 1. The assessor meets the basic competences in the assessor profile; 2. The assessor has participated in professional development activities that guarantee a common interpretation of the assessment model5 and a uniform working method for assessors; 3. The assessor has gained experience in conducting assessments and has thus demonstrated clearly that they meet the behavioural indicators. Aptitude test registration procedure: The supervisor (or their representative) registers the assessor(s) at least two weeks prior to the test by emailing assessor@hva.nl or calling +31 (0)6 21 15 58 12 and providing the following information:

– the names and email addresses of assessors who are going to sit an aptitude test;

– a timetable showing when and where the assessors in question will conduct the assessment that will serve as an aptitude test. A period of 20 minutes before and after the assessment should be reserved in connection with a preliminary and follow-up interview with the assessor evaluator (see Section 2.1).

2.2 Assessment and certificate There are two possible outcomes to the assessment of assessors: 'suitability demonstrated' and 'suitability not demonstrated'. 'Suitability demonstrated' means that the assessor has demonstrated all required behavioural indicators in the assessor profile in the aptitude test, and hence that the assessor has achieved the assessor qualification. The corresponding certificate is issued by the AUAS Certification Committee for Lecturer Competences. 'Suitability not demonstrated' means that the assessor did not sufficiently demonstrate all behavioural indicators. The assessor evaluator provides oral feedback to the assessor on each indicator, immediately after the test and in a report. The assessor can use the feedback obtained to work on their points for improvement in a targeted manner and resit the test at a later stage. 2.2.1 Expert and independent evaluators The assessor evaluator determines in a professional manner whether the assessor meets the behavioural indicators in the assessor profile. Since 2006, AUAS has had a network of ten assessor evaluators available. Half of them are affiliated with AUAS programmes, the other half with external HE programmes. Their expertise is reflected in the basic competences in the assessor evaluator profile (see Appendix 3). The assessor evaluators meet on a regular basis to exchange experiences and critical situations that could arise when evaluating assessors. There are also calibration sessions in which assessment recordings are played back and reports critically examined to ensure uniformity in the use of the assessment criteria, cut-off scores and the standard. To guarantee the independence of assessor evaluators, they are not allowed to evaluate assessors known to them. Another rule of thumb is that they do not conduct assessments for their own faculty. In case of a resit, a different assessor evaluator is used than for a previous test.

5 This concerns the assessment model (or standard) used by assessors when conducting assessments.

7

Sound recording of the aptitude test For organisational reasons and to reduce costs, a single assessor evaluator is assigned to each aptitude test. To guarantee the general validity of the judgement, in addition to the previously mentioned calibration sessions, a sound recording of the aptitude test is made so that an assessor evaluator can, in case of doubt, consult a fellow assessor evaluator in order to arrive at an independent and well-founded judgement. If the assessor objects to the judgement on well-founded grounds, a second opinion can be obtained with the aid of the recording.

2.3 Resits Assessors who failed to prove their aptitude in the aptitude test may sit another aptitude test at a later stage. This is arranged by the supervisor in accordance with the registration procedure in Section 2.1.1. The costs associated with this (see Section 5) are for the account of the HE programme the assessor works for. It is advisable to work in a focused manner on the points for improvement listed in the assessment report between the tests. Section 3 includes suggestions for professional development.

2.4 Objections and appeals If assessors are able to demonstrate that the aptitude test was not conducted in accordance with the applicable procedure in Section 2.1, or disagree with a judgement, they may lodge an objection in writing. Objections to a judgement must be substantiated by arguments. The principle of hearing both sides will be applied. In the event of an objection, assessors can call +31 (0)6 21 15 58 12 or send an email to: assessor@hva.nl

If assessors believe that their objections with regard to the procedure have not been dealt with in a satisfactory manner, they can lodge an appeal with the appeals committee by sending a substantiated notice of appeal to the secretary of the AUAS Certification Committee for Lecturer Competences. Notices of appeal with regard to the judgement are inadmissible, in accordance with Section 8:4 (3b) of the General Administrative Law Act ("Algemene Wet Bestuursrecht").

3. Preconditions for certification: responsibility of the HE programme concerned

Assessor certification – the impartial determination of the ability of assessors to carry out their tasks

according to set criteria – requires ongoing professional development and adequate assessment materials and procedures. These are prerequisites for achieving high-quality assessments, which are the responsibility of every programme for which assessments are conducted. This section deals with professional development activities in preparation for the aptitude test and for the retention of the certificate (see Sections 3.1. to 3.1.2). Section 3.2 deals with the conditions surrounding assessments. These must be satisfactory in order for assessors to be able to carry out their tasks optimally.

3.1 Ongoing professional development In addition to broad practical experience and solid professional expertise, qualified assessors must also have interview and assessment skills. These require practice and maintenance in order to reach and maintain the required assessor level. Section 3.1.1 contains information on professional development in preparation for the aptitude test. Section 3.1.2 discusses activities relating to the retention of the assessor certificate.

8

3.1.1 Professional development in preparation for the aptitude test Any professional development activities prior to the aptitude test should be aimed at ensuring that assessors have a thorough grasp of the required skills, i.e. the behavioural indicators in the assessor profile. From the literature it appears that assessment skills training is a requirement to develop a uniform working method and a common interpretation of the assessment model when conducting assessments (Van der Vleuten et al., 2010; Straetmans, 2006). Training also makes assessors aware of personal assessment effects (Straetmans, 2014). The training course is designed for a team of lecturers from an HE programme that is responsible for conducting assessments for that programme. During the training, assessors gain experience with the behavioural indicators in the assessor profile (Table 2) and the assessment model for the assessment. The training6 is a first step towards the intended assessor quality for certification. The vast majority of assessors also prefer to gain experience by conducting a number of assessments in order to put the skills they have learnt into practice. However, this does not yet mean that they meet the behavioural criteria in the assessor profile. Professional development activities for assessors must be tailored to the individual and their effectiveness increases if they are demand-driven and development-oriented (Joosten-ten Brinke et al., 2014). They concern both individual and joint activities. Individual activities are tailored to the qualities of an assessor. For example, one person might need an extra workshop to practise their interview techniques, while another might want feedback on their skills when conducting an assessment. Joint activities for lecturer teams are intended to promote a generally valid judgement. Examples include peer review meetings to take a deeper look at critical situations and exchange experiences and calibration sessions to clarify and sharpen assessment criteria, standards and cut-off scores (Kok, 2014, p. 24; Andriessen et al., 2017).

Have a look at the AUAS Academy offer (academie.hva.nl) or email assessor@hva.nl for tailor-made advice and/or support in the implementation of professional development activities for assessors.

3.1.2 Ongoing professional development to retain the assessor certificate After obtaining the certificate, professional development is also necessary to maintain one's assessment skills and to continue to guarantee a uniform working method and a common interpretation of the assessment model7 (Van Berkel, 2015). The most important condition for the maintenance of assessment skills is that assessors continue to conduct assessments on a regular basis. Professional development is thus integrated into their work (Kools et al., 2014a). Assessors can also use their assessment knowledge and experience to prepare prospective assessors for the aptitude test (learning by working together, Kools et al., 2014b). Examples of additional professional development activities are given in Table 1. Working method and standard to demonstrate expertise In order to ensure continued availability, assessors must demonstrate that they have maintained their expertise annually. This should preferably be done during the annual appraisal and/or performance review, during which the supervisor will discuss the maintenance of assessment skills with the assessor. In preparation for this, assessors should map out how they have worked on their professional development. The form in Appendix 4, on which assessors enter their relevant professional development activities, is a useful tool in this regard. The standard for the continued availability of the assessor is that the following is done on an annual basis: - They conduct a number of assessments (Part A of the form); - They actively participate in some additional professional development meetings, including calibration

(Part B of the form).

6 For experienced assessors, training may not be the appropriate instrument as a first step towards certification. In this case, the requirements for the aptitude test should be assessed on an individual basis. 7 Assessment models contain open and generic criteria. This is necessary in order to be able to assess the diversity of professional practice using one and the same standard. By exchanging concrete examples and experiences, a common interpretation of the standard is created.

9

Table 1: Guidelines for retaining the assessor certificate 3.2 Quality of assessment practice HE assessment practices are very diverse and do not always meet the basic requirements of good assessment. This concerns both the materials, such as assessment models and protocols, and the assessment procedures. The most important requirements are listed and explained below. They help the assessor perform successfully and thereby get the best out of students during assessments. The following conditions must be met before assessors sit their aptitude test: -The assessment model must be workable, i.e. it must include representative and recognisable core responsibilities from the HE programme profile, for which the student provides evidence in their portfolio8 (portfolio assessment). The model must also contain a clear number of criteria (behavioural and otherwise) on the basis of which assessors are able to determine whether the tasks have been carried out at the desired HE level or Bachelor’s/Master’s exit level. As a rule, these criteria are derived from the higher professional education standards (HBO Raad, 2009) and they should preferably be elaborated in concrete behaviour. Interview and assessment protocols associated with the model are available in order to arrive at a well-founded judgement in a structured manner.

-Portfolios must be accessible9, i.e. requirements must be set for the content (the type and quality of evidence) and structure of the portfolio (Van Berkel, 2012). The purpose of this is twofold: it requires the student to think carefully about their best evidence for each core responsibility and it makes the information transparent for the assessor. -There must be an unambiguous and workable assessment procedure, i.e. the procedure has been worked out in clear steps, in which the tasks and responsibilities are clear to all parties involved. This should be laid down (in a manual) with consistent use of terminology. Considerations regarding the procedure: - The use of two assessors per assessment for a generally valid judgement (Straetmans, 2014). According to

AUAS testing policy, the use of two assessors is a requirement when assessing exit qualifications (Education and Research Policy Office, 2017, p. 9).

- The time available for assessors to conduct an assessment, including preparation and aftercare. More information about portfolio and behavioural assessments and about AUAS assessment and quality policies is available at www.score.hva.nl.

8 In a behavioural assessment, the student demonstrates the behaviour by carrying out a core responsibility/professional task. This behaviour is assessed by the assessor. 9 This only applies to portfolio assessments.

The assessor: - works in the discipline/domain for which assessments are conducted and has kept abreast of

current developments with regard to conducting assessments; - conducts a number of assessments every year; - has asked a colleague for feedback on their performance as an assessor in the interim;* - has worked specifically on the points for improvement raised in this feedback; - has given feedback to fellow assessors (prospective and otherwise) on their performance; *The behavioural indicators in the assessor profile serve as a guide for giving feedback. - has attended meetings relating to:

o assessing the practical experiences and assignments of students o the improvement, sharpening and deepening of interview and/or questioning techniques o the clarification of criteria, standards and cut-off scores in assessments (calibration

session) o discussing critical situations with fellow assessors (peer review) o current developments in the field of HE assessments

o other, namely…

10

4. Quality In order to guarantee the independence of the judgement, some distance from the person being assessed is necessary. This explains why the certification programme for assessors is carried out by a central authority, i.e. independently of any specific programme. In addition, the quality of the implementation of the certification programme is monitored by an independent and expert party: the AUAS Certification Committee for Lecturer Competences. There is also a role for HE programmes that wish to deploy assessors on a permanent basis. In this section, the quality assurance provided by these three parties is explained in more detail.

4.1 Quality assurance of the aptitude test and assessment When assessors register for the aptitude test: - it will be checked whether the preconditions of ongoing professional development and the quality of

assessment practice as described in Section 3 of this manual have been met; - independent and expert assessor evaluators are assigned to the aptitude tests. In addition, there is a knowledge network of assessor evaluators. They meet on a regular basis to recalibrate the assessment criteria and discuss critical situations in order to ensure an unambiguous interpretation of the assessment criteria and sharpen the aptitude test (and the working method for assessing it) where necessary. An (AUAS-wide) register of certified assessors is kept and the certification is included in the personnel files of the assessors concerned.

4.2 Quality assurance by the AUAS Certification Committee for Lecturer Competences The role of the Certification Committee consists of: - appointing assessor evaluators on the basis of the assessor evaluator profile (Appendix 3); - randomly checking aptitude test reports for completeness, correct use of the assessment criteria, logical

realisation and substantiation of the judgement; - handling objections (in the second instance); - issuing the certificate in the case of a positive assessment of the aptitude test.

4.3 Quality assurance by the HE programme for which the assessors work The assessment committee and examination board of the HE programme is responsible for the overall quality of assessments for the programme and for the expertise of assessors. With regard to the assessor certification (and the retention thereof), their quality assurance is aimed at: - supervising and facilitating the ongoing professional development of assessors (training, calibration, peer

review); - supervising and facilitating a sound and workable assessment practice with regard to

o a workable assessment model o accessible portfolios (for portfolio assessments) o a workable procedure.

An explanation of these criteria can be found in Section 3 of this manual.

5. Costs The costs for assessment and certification amount to €475.10 These costs comprise the activities of the assessor evaluator (see Section 2.1), quality assurance (see Section 4.1) and planning. Any costs associated with 'no shows' of students or assessors for the aptitude test will be passed on to the HE programme.

10 This is the internal AUAS rate for assessor certification as from 1 January 2019. The rate for external certification may deviate from this.

11

Sources Education and Research Policy Office (2017). HvA Toetsbeleid, Kaders en criteria (AUAS assessment policy: frameworks

and criteria). Amsterdam: Amsterdam University of Applied Sciences. Andriessen, D., Sluijsmans, D., Snel, M. & Jacobs, A. (2017). Protocol verbeteren en verantwoorden van afstuderen 2.0

(Protocol for the improvement and substantiation of graduation procedures 2.0). The Hague: Netherlands Association of Universities of Applied Sciences

HBO Council (2009). Kwaliteit als opdracht (Quality as a requirement). The Hague: HBO Council.

Joosten-ten Brinke, D., Arts, J., Burghout, C., Jaspers, M., Lansu, W., Maas, M., Verhoosel, J., & Kratsborn, S. (2014). 'Bewust en bekwaam toetsen en beoordelen' (Conscientious and competent assessment and evaluation). In Eigentijds toetsen en beoordelen. De opbrengst van vier jaar praktijkonderzoek (Contemporary assessment and evaluation: the outcomes of four years of practical research), pp. 89-98. Tilburg: teacher training programme, Fontys University of Applied Sciences, Tilburg.

Kok, M., & Te Lintelo, L. (2014). Leidraad eindniveau (Exit level guidelines). Amsterdam: Amsterdam University of Applied Sciences. Kools, Q., Bouckaert-den Draak, M., Gootzen, M., Mathijsen, I., Schildwacht, R., Tielman, K., &

Van de Reijt, V. (2014a). 'Informeel leren' (Informal learning). In Inzicht in professionalisering. Vier jaar lectoraatsonderzoek naar professionalisering van leraren en lerarenopleiders (Professional development insights: four years of professorship research into the professional development of teachers and teacher trainers), pp. 13-21. Tilburg: teacher training programme, Fontys University of Applied Sciences, Tilburg.

Kools, Q., Bouckaert-den Draak, M., Gootzen, M., Mathijsen, I., Schildwacht, R., Tielman, K., & Van de Reijt, V. (2014b). 'Professionaliseren door samen werken' (Professional development through collaboration). In Inzicht in professionalisering. Vier jaar lectoraatsonderzoek naar professionalisering van leraren en lerarenopleiders (Professional development insights: four years of professorship research into the professional development of teachers and teacher trainers), pp. 33-48. Tilburg: teacher training programme, Fontys University of Applied Sciences, Tilburg.

Hobéon Certificering (15 October 2014). Certification of persons. Obtained on 15 October 2014: http://www.hobeon.nl/certificering/personen/certificatie_en_registratie_bij_hobeon_sko#acc28/184/

Straetmans, G. (2006). 'Assessment van competentie: definitie, knelpunten, ontwerpbeslissingen' (Competence assessment: definition, bottlenecks, draft decisions). In E. Roelofs & G. Straetmans. Assessment in Actie (Assessment in action), pp. 13-40. Arnhem: Cito B.V.

Straetmans, G. (2014). 'Toetsen met performance assessment methodieken' (Assessment using performance assessment methods). In H. van Berkel, A. Bax & D. Joosten-ten Brinke (Eds.). Toetsen in het hoger onderwijs (Assessment in higher education), pp. 181-192. Houten: Bohn Stafleu Van Loghum.

Van Berkel, A. (2012). 'Kritische reflectie op competentietoetsen in het hbo' (Critical reflection on competence development in higher professional education). In Onderwijsinnovatie (Educational Innovation), 2 (14), pp. 17-26. The Hague: OBT B.V.

Van Berkel, A., Sluijsmans, D. & Joosten-ten Brinke, D. (2015). 'Kwaliteit van toetsbekwaamheid' (Assessment competence quality). In Sluijsmans, D., Joosten-ten Brinke, D. & Van Schilt-Mol, T. (Eds.). De kwaliteit van toetsen onder de loep (Assessment quality: a closer look). Garant: Antwerp, Apeldoorn: Garant-Uitgevers N.V.

Van der Vleuten, C. P. M., Schuwirth, L.W.T., Scheele, F., Driessen, E.W., & Hodges, B. (2010). 'The assessment of professional competence: building blocks for theory development'. In Best Practice & Research Clinical Obstetrics and Gynaecology, 24, pp. 703-719.

12

Appendix 1: Basic/Senior Examiner Qualification (BEQ/SEQ) at AUAS This appendix contains an overview of assessment formats in relation to BEQ/SEQ and assessor certification. Assessment

formats →

Phase in the assessment

cycle ↓

Single assessment formats Integrated assessment formats

Open-ended and closed questions Progress test

Essay Case study test

Skills test Reflection assignment

Peer assessment (Single) project assignment

(Complex) project assignment Research assignment

Work placement and practical assignment Thesis

Portfolio assessment Behavioural assessment

Designing, constructing and improving (based on evaluation)

SEQ BEQ Assessor

Assessing, judging, analysing and evaluating

Source: Opzet en invoering certificering BKE en SKE (HvA) [Design and implementation of BEQ and SEQ certification (AUAS)], updated version dated 29 April 2016

13

Appendix 2: Explanation of the behavioural indicators in the assessor profile

Assessor evaluators use the behavioural indicators in the assessor profile as evaluation criteria for certification. The indicators are discussed in detail in the assessor training, both in theory (digital training programmes) and in practice (simulation with an actor). The indicators have been worked out into desirable behaviour for an unambiguous interpretation. This desirable behaviour is explicitly NOT meant as a check list!

1. Creates an atmosphere during the assessment interview in which the student's talents are done full justice

Behaviour Puts the student at ease

Breaks down or neutralises resistance or irritation during the interview

Result: Students feel at ease during the interview. This enables them to have an open dialogue with the assessors and to provide the required information.

2. Structures the assessment interview

Behaviour Starts the interview by highlighting introduction, unfreezing/relaxation, goal of the interview, explanation of procedure (interview), agenda, division of roles between assessors, control question, room for student questions

Collects the required information about the competences to be assessed throughout the interview

Mentions the topic of discussion, asks probing questions, summarizes in between and rounds off by summarising the topic

Ends the interview with a substantiated judgement and feedback, a deadline by which the student will receive the report and, if applicable, a procedure for objections

Monitors the time for the various topics to be covered during the interview

Provides structure in order to arrive at a judgement for each discussed topic by:

– sharing findings (from portfolio and interview) with the co-assessor

– comparing them with the assessment standard and

– arriving at a joint and substantiated judgement

Result: Both students and assessors know at all times 'where' they are in the interview and which topic is being discussed. This provides guidance and is also a tool for assessors to keep control of the interview.

3. Applies different questioning and interviewing techniques at the right time

Behaviour Listens actively

Uses appropriate interview techniques, including in any case STARRT(T), with in-depth questions to gather information about concrete behaviour and thought and decision processes

Has a clear idea at any time during the interview of exactly what they want to know (= what information they are looking for)

Explores the scope (transfer and opposite)

Maintains control throughout the interview (listening, summarising, asking follow-up questions)

If necessary, applies metacommunication techniques at the level of relationship and content

Is consistent with the student in terms of language and level of communication

Result: An interview with depth: the information required to arrive at the judgement is obtained.

14

4. Relates the experiences, working method and evidence provided by the student to assessment criteria

Behaviour Assesses the evidence for authenticity, relevance, topicality, variety and quantity

Applies the assessment triangle, in which evidence, professional activities and assessment criteria are structurally linked

Recognises products (professional and otherwise) as usable evidence

Relates evidence to the right professional activities and assessment criteria (classification)

Establishes links between evidence, professional activities and criteria

Records findings in an accessible way in the protocol provided

Result: Findings from the portfolio and interview are related to the assessment standard (= professional activities and assessment criteria) and laid down in a protocol.

5. Reaches a substantiated judgement and conveys this in a convincing manner

Behaviour Applies the appropriate assessment techniques in a structured manner

In doing so, uses the assessment triangle: assessment standard (in terms of activities and criteria) and evidence

Substantiates the judgement for each topic with examples of competent behaviour from the portfolio and interview related to the assessment standard

Arrives at a judgement and coordinates it with the co-assessor for a generally valid judgement

In the event of a difference of opinion regarding the judgement: looks at the findings from the portfolio and interview in a structured manner (= per subject) in relation to the assessment criteria. Uses facts from the portfolio and interview to reach a consensus

Informs the student about the judgement and the substantiation, using accessible language

Result: A uniform and transparent working method on which the judgement is based, with a generally valid judgement as the end result.

6. Gives constructive feedback

Behaviour Applies the model and rules for giving feedback

Result: Students receive feedback. It is clear to them which skills they have demonstrated (and to what extent), and what they still have to work on to achieve the required level.

7. Records the judgement in an accessible manner

Behaviour Records on the assessment form the extent to which professional competences have been demonstrated in relation to the assessment standard, after consultation with the co-assessor

Substantiates the judgement with findings from the portfolio (evidence) and interview (explanation of evidence, behaviour, thought and decision processes, reflection, transfer, etc.)

Writes in an accessible manner: coordinates written language with that of the student

Result: The assessment form transparently records the generally valid judgement in a way that can be understood by 'third parties'. The substantiation of the judgement is based on findings from the portfolio and interview for each subject discussed. It is also clear how the final judgement was based on this (e.g. with decision guidelines).

8. Handles any objections adequately and in a customer-oriented manner

Behaviour Uses appropriate techniques for dealing with objections, such as ACSIC

Refers to the objection procedure, if necessary

Result: Students feel that their objection has been heard. They have been listened to and, where possible, given information in order to remove any ambiguities, misunderstandings, etc. They know where they can find the objection procedure.

15

Appendix 3: Assessor evaluator profile Core responsibilities of the assessor evaluator

1. Observes the assessor when conducting an assessment (= aptitude test). 2. Observes and records their findings during the aptitude test. 3. Assesses whether the assessor meets the behavioural indicators in the assessor profile. 4. Communicates the judgement orally to the assessor immediately after the aptitude test, supported by observations

and findings from the test. 5. Records the judgement and the substantiation of it in a written report.

Basic competences of the assessor evaluator11 - Is an expert in the field of HBO assessments. - Has extensive experience in training HBO and other types of assessors and/or in the professional practice of

assessment & development and/or in conducting out HBO assessments. - Has been educated to Master's level and preferably holds a Master's degree. - Shows empathy and involvement with the person, but is businesslike in their judgement. - Communicates in an open and convincing manner, both verbally and in writing. - Works accurately and observes agreements. - Reflects on own performance and is focused on professional development as an assessment expert.

Qualities of the assessor evaluator The assessor evaluator is able to link observed behaviour to the behavioural indicators in the assessor profile. Based on their observations, they make a well-founded judgement and provide convincing feedback to the assessor immediately after the assessment has been completed. To substantiate their judgement, they use examples from the assessment with regard to applied techniques, skills and behaviour. They record the whole process in an accessible manner in a concise report.

Behavioural indicators: - Applies the technique of observing, recording and assessing adequately. - Is able to link observed behaviour to the behavioural indicators in the assessor profile. - Makes a well-founded final judgement by appraising observed behaviour in relation to the behavioural indicators. - Conveys the judgement to the assessor in a convincing and constructive manner. - Gives development-oriented feedback immediately after the aptitude test. - Records the judgement in an accessibly written report. - Handles any objections to the judgement adequately.

11 The basic competences are designed to assist with the selection of assessor evaluators.

16

Appendix 4: Retention of assessor certification form (example)

Name: _________________________________ Date: __________________________ Assessor for the programme(s): ______________________________________________

A. Overview of assessments conducted

Name of student Name of co-assessor (if applicable) Assessment

date Programme, year 2, 3 or 4

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

B. Other activities that contribute to your expertise as an assessor

Activity Date Number

of sessions

Explanation of activity and your own contribution

Evidence available? Y/N,

namely… Training and/or refresher training for assessors

Feedback received from an assessment expert while conducting an assessment

Feedback given to a fellow assessor on the basis of the assessor profile

Ongoing professional development meeting for assessors attended, namely 0 Peer review; 0 Calibration

session; 0 Other, namely…

Workshop followed for the improvement and deepening of my questioning, interviewing & assessment techniques

Attended a conference about assessments, programme-independent assessment, etc.

Other, namely…

Other, namely…

Other, namely…

Please submit the completed form to your supervisor prior to your annual appraisal/performance review.

17

Explanatory notes for completing the retention of assessor certification form Introduction For your certification as an assessor, you have demonstrated in an aptitude test that you have the required expertise and skills to conduct a high-quality assessment, in which the student's talents are done full justice. In order to be able to guarantee that you continue to deliver the required quality, it is necessary to maintain your expertise. By completing this form, you can map out how you have worked on your professional development as an assessor over the past year. The form is divided into two parts: A. An overview of the assessments you have conducted in the past year, and B. An overview of the activities you have undertaken in order to work on your professional development. Completion instructions 1. Please state your name, the date and the programme for which you are conducting assessments at the

top of the form.

Part A: Overview of assessments conducted 2. Use this table to provide an overview of the assessments you conducted in the past year by entering the

requested data. Although there is room for 10 assessments, you can mention more or fewer. In the latter case, you can expand the overview.

3. In the last column, you should indicate what kind of assessment it was, for example an assessment for the third year internship. Part B: Overview of other activities to promote expertise

4. In the first column, mention the activity in which you participated in order to maintain or increase your expertise as an assessor. Here you can mention any activity that supports you in your role as an assessor. A number of examples have been provided to help you get started. Examples are also given in Table 1 in Section 3.1.2 of this manual.

5. For each activity, indicate the date on which it took place in the second column. 6. Enter the number of sessions spent on the activity in the third column. 7. In the fourth column, explain the content of the activity. Be as specific as possible: mention titles/names

of conferences, workshops, etc., with topics/themes that were discussed and names of people/organisations that organised the activity. If you made a contribution yourself, state this explicitly as well.

8. In the last column, you should indicate whether you have evidence of participation in the activity and, if so, what this evidence is. Examples of evidence are a conference/training course programme with participant list; a PPT file of a presentation you gave at an activity; a report of feedback you received or gave; or a report of a peer review meeting in which you participated. You do not need to enclose the evidence with this form, but you must be able to provide it on request. It is useful to create a folder/file for this.

9. Once you have completed the form, you should hand it in to your supervisor prior to your annual appraisal/performance review.