www.aqu.cat

ENQA TRAINING OF AGENCY REVIEWERS

Focus on the role of the secretary and agency view

Núria Comet, Bern 12-13 May 2014

ENQA Review : a long way

Agency PanelPanel

AgencyAgency

Report

ENQA

SERENQA

~ 6m – 12 m 2m 2 days 5 years

First stage

The point of view of the Agency

The point of view of the Agency

The Self evaluation report (SER) is the basic sourceof information for the panel and has to provideclear information, sufficient reflections, critiqueand analysis.

Its contents must be corroborated with evidences.

The reports allows the review team to prepare lines of enquiry in advance of the site visit.

The point of view of the Agency

Some recommendations for drafting the self-evaluation report:

careful study of other agencies’ self-evaluations

meetings with other agencies

make a team responsible (need of human resources)

define the structure of the SER

evidence for each criteria had to be collected in one

document/ links (and translated to English)

draft SER should circulate throught steering bodies for

comments

The point of view of the Agency

SER- strengths:

occasion for self-reflection on strengths and

weaknesses (SWOT analysis)

learning process

engenders improvement mechanisms

“team building“, participation

transparency (SER is public)

consultative process overall acceptance

The point of view of the Agency

SER- weakness

heterogeneous style, repetitions

different level of interpretation of standards

continuous fine-tuning distance from initial

focus

“politically correct” in detriment to self-criticism

weak analytical approach

Second stage

The role of the secretary

The agency will publish the completed self-evaluation document on its website and submit it to the review secretary.

Role of Secretary

Member

Logistics

Communications

Panel

Chair

Agency

ENQAReport

ScheduleBefore

During

After

The secretary has a special role

Role of Secretary

• To get to know the ENQA Guidelines well• Read other ENQA Review Reports

• Read and analyse SER

• Review documentary evidence • Look at Agency website• Examine national documents

Maintain accurate list from the outset of all documents received and considered

As member of the panel:

Role of Secretary

• Time-line for whole review: prior meetingsite visitreport

• Manage logistical arrangements: • Travel arrangements: arrival and departure

times• Hotel booking (+ room for the prior meeting)• Evening meals

As responsible for the logistic:

Role of Secretary

As responsible for communication

• Act as the single conduit between the Panel, ENQA and the Agency throughout the review process.

• Make early introductions to the Institutional Contact (Agency), the Chair, and the Panel.

• Hold telephone meetings/Skype with the Chairregularly (2-3 recommended)

• Establish clear deadlines• Keep team on track

Role of Secretary

As responsible for the preparation of site-visit

Agree visit programme with the Chair and the Agency.

Pay attention to:

. the length of each interview

. assure that all the interest group are interviewed. the people (names/responsability) interviewed. break-times. arrange interpreter support (only if necessary). be sensitive towards the additional request

Role of Secretary

Type of interviews the ENQA review panel required to conduct

• executive board (first – last meeting)

• resp. of quality assurance/ of accreditation department

• chairs (and/or members) of permanent commissions;

• staff

• representatives of the Ministry / Government

• external evaluators

• representatives of public/private universities/ inst. of technology/

• students (preferably who participated in evaluations)

• stakeholders (e.g. employers)

Role of Secretary

As responsible for the preparation of site-visit:things to sort out during the prior meeting

The members must have a clear idea of their role during the site visit

• Produce clear agendas for the meetings

• Prepare a check-list

• Ensure what the panel need to clarify, which information they need, or which evidences need to find

Role of Secretary

During the site visit

• Use time wisely• Use the checklist • Keep notes of formal and informal discussions• Check that sufficient evidence has been

gathered at the end of each session and at the end of each day

• Keep the chair advised of any shortfalls or if sufficient evidence has been gathered

• Use the last internal meeting to agree:• the findings to be presented to the Agency

on the final day• next calendar

Role of Secretary

The final report It’s mainly your task as secretary !!

Start drafting the report as soon as possibleYou must make time available Block out one /two solid weeks

• Review notes, check records of documentation• Summarise team’s analysis and comments on

SER

• Consult all of team for additions and changes• Final consultation with Chair

• Send to the Agency to review accuracy• If necessary, rewrite the Report

Role of Secretary

The final report must

• Accurately reflect the findings of the panel • Provide evidence, analysis and conclusions against

each ENQA membership Criterion• Ensure the requirements of the exercise are met: Type

A or Type B • Reflect the national idiosyncrasy

Provide an clear account of the evidence collected by the team for the ENQA Board to make its decision on membership

Provide a useful tool for the ongoing development of the Agency itself

Third stage

The point of view of the Agency

Feedback of the Agency

The agency welcomed panel members who can understand their HE system and provide

additional insights

who are highly professional and with impartial analysis which include a national panel member

About the site -visit: practical problems with foreign reviewers and

stakeholders’ participation in the site visit

necessary to improve stakeholders’ involvement

Feedback of the Agency

The final meeting : whilst Panel had acquired a global image… the agency

ignores Panel conclusions

it’s a short meeting, not a list of degree of compliance not the final decision

The Report The agency expects that the report includes strengths,

weaknesses and recommendations

Feedback of the Agency

The agencies think that

It’s an European recognition of the Agency

It’s a way to gain legitimacy towards HEIs/ goverment

It’s a way to continue the work of improving the

quality of its evaluations

Thanks for your attention !

Núria Comet

ncomet@aqu.cat