1

DoQupTraining Seminar

Alicante, 24-26 April 2013

Organisation for the Quality Assurance of Institutions and Study Programmes

Guillermo Bernabeu, University of Alicante

The contextSpain: QA for HEA shared view on QA

The Context

From Berlin CommuniqueWe adopt the standards and guidelines for quality assurance in the European Higher Education Area as proposed by ENQA.

We commit ourselves to introducing the proposed model for peer review of quality assurance agencies on a national basis, while respecting the commonly accepted guidelines and criteria.

We underline the importance of cooperation between nationally recognized agencies with a view to enhancing the mutual recognition of accreditation or quality assurance decisions.

Spain:QA for HE

The Spanish Context: Timing

Initial steps in 2003 with a position paper from the Ministry of Education, Science and Sports (six different ministers ...).

ECTS credits and Diploma Supplement introduced very early (2003).

QA for Spanish HE (or ANECA) defined in 2002.

A modification of the Spanish Universities Law introduced in 2007.

(Too) long discussion about the BMD structure: Catalogue versus Register, 3 or 4 years for Bachelors, ... .

Different Royal Decrees regulating the BMD structure … were … published in 2005, 2007, 2010 and 2011.

Degree Structure and Study Programmes “kick-off” ended in 2010.

THE SPANISH AGENDA

The Spanish Context: Space

Public universities funding in Spain is regional based while HE regulation is a competence of the national authorities.

University autonomy is in the Spanish Constitution.

78 universities (50+28), 17 autonomous regions, 1.500.000 students at Spanish Universities, ...

ANECA, 4 QA regional agencies in ENQA&EQAR, 6 QA regional agencies, … .

THE SPANISH AGENDA

EVALUATION EX-ANTE: VERIFICATION

FOLLOW-UP PROCEDUREPublic informationOutcomesindicators

ACCREDITATION (after 4 or 6 years)Fulfillment

Higher Education Spanish System

RUCTSIIU

ACREDITAA

Quality Assurance

To verify that the design of the study programme includes the criteria established by the Ministry whose interpretation and application is made by ANECA. A public report is delivered by ANECA.

Ex-ante accreditationSTAGE 1

Verificación

Accreditation System in Spain

Higher Education Spanish System

1. DESCRIPTION OF THE DEGREE

2. JUSTIFICATION

3. LEARNING OUTCOMES

4. STUDENT ENTRY AND ADMISSION

5. PROGRAMME PLANNING

6. ACADEMIC STAFF

7. RESOURCES AND SERVICES

8. ANTICIPATED OUTCOMES

9. INTERNAL QUALITY ASSURANCE SYSTEM

10. IMPLEMENTATION AGENDA

BMEX-ANTE ACCRED.

Accreditation Process in Spain

RD 1393/8 (Modif. 861/2010) sets out how the report shlod be requested for the verification of Official Degrees

1. DESCRIPTION OF THE DEGREEEach degree design must include a description that is adequate and consistent with the level and/or academic validity so there is no confusion concerning its content or, where applicable, qualification for professional purposes.

2. JUSTIFICATIONEach degree design must be relevant in terms of the corresponding learning and/or research experience, be appropriate to the academic field that it refers to and/or comply with other similar existing studies.

• In the case of the Master’s, external equivalents with studies in other countries may be provided.

• In the case of the Master’s, justification must be provided for the research, academic or professional orientation of the programme.

• In the case of a Master’s with a professional or research orientation, the design must refer to the situation of R+D and innovation in the professional sector.

Evaluation Criteria

3. COMPETENCES The competences to be acquired by students must be consistent with those required in order for the degree to be awarded and the qualifications stablished in the EHEA. Consequently, these competences must be assessable and consistent with the corresponding level of the Bachelor or Master’s degree.

4. STUDENT ENTRY AND ADMISSION.The degree programme must have accessible systems that regulate and provide clear information on the different means of entry, admission and new student orientation.

• In the case of Master’s degrees, in addition to the envisaged entrance exams, state the body that is responsible for admissions, together with the applicable procedures and requirements.

Evaluation Criteria

5. PROGRAMME PLANNING.The programme of studies must constitute a proposal for study that has been designed in a coordinated way and takes into consideration the student’s dedication within a given period of time and the competences to be acquired in the degree proposal.

In the case of a Master’s degree with a professional orientation, the approach given to adequate professional practice will be of special importance, as well as collaboration agreements with private enterprise and other institutions for this practice.

6. ACADEMIC STAFF.Teaching staff and other support human resources in each degree must be adequate in order for the general aims and competences envisaged in the degree design to be achieved.

7. MATERIAL RESOURCES AND SERVICES.Material resources and services that are necessary for the development of the envisaged activities must be adequate for the envisaged aims and competences in the programme of studies to be achieved.

Evaluation Criteria

8. EXPECTED RESULTS

The degree design must include the anticipated outcomes in connection with the degree's efficiency and the general mechanisms for assessing the learning outcomes of the students.

Evaluation Criteria

GRADUATION RATE: the percentage of students who complete the course in the time envisaged in the programme of study (d) or take one year longer (d+1) in relation to their entrant cohort. This gives a measure of overall academic performance.

DROP OUT RATE: the percentage relation between the total number of students in a new entry cohort who should have obtained the degree the year before who neither registered this year nor the year before. In the case of one year Master’s programmes, the following definition shall be applied: percentage relationship between the total number of students in a new entry cohort that should have obtained their degree the previous academic year and that did not enrol in either that academic year or the subsequent one.

EFFICIENCY RATE: the percentage relation between the total number of credits in the programme of study and the total number of credits in which the group of graduated students in a particular academic year had to enrol in throughout their studies.

Evaluation Criteria

9. QUALITY ASSURANCE SYSTEMThe degree design must include an internal quality assurance system that ensures the control, review and continuous enhancement of the programme.

Specify the body or unit in charge of the quality assurance system for the programme of study (structure and composition), together with the internal rules of procedure. Details must be provided in this section on how participation in this body by teaching staff, students, academic managers, support staff and external stakeholders is organized.

Establish how the development of the programme of study is to be reviewed (aims, competences, planning, etc.) through the application of adequate mechanisms and procedures applied periodically for gathering and analysing information on:•The quality of the programme and the teaching staff. •The quality of placement/work experience and mobility programmes. •Graduate employment and degree satisfaction.•The satisfaction of the different groups involved (students, academic staff and administration and services staff, etc.) and the attention paid to suggestions and complaints.

Evaluation Criteria

AUDIT

QUALITY ASSURANCE SYSTEM

Have adequate and systematic mechanisms and procedures for decision- making that ensure the enhancement of the programme of study. These should include:

The quality of the programme and the teaching staff.The quality of placement/work experience and mobility programmes.

These mechanisms and procedures must at least provide for who is in charge, the way that decisions are made and follow-up of decisions that are made.

Define the criteria that establish the limits whereby the degree is finally suspended. A procedure needs to be established beforehand that describes the system set up to define these criteria in the institution/university, together with its review, approval and periodic updating.

Establish mechanisms to publish the information on the programme of study, its development and outcomes, and for this to be sent to all those involved and/or interested (students, teaching staff, support staff, prospective students, external stakeholders, etc.).

Evaluation Criteria

Evaluation Criteria

10. SCHEDULE FOR IMPLEMENTING THE DEGREE

The process of introducing new degree programmes must be planned in terms of a schedule, and there will need to be a mechanism, where applicable, to help students from existing courses to adapt to the new programme of study.

Evaluation Procedures

Evaluation ex-ante (verification)

•Evaluations Commissions for Bachelor/Master/Doctorate and for

the five tematic areas: Arts and Humanities, Sciences, Health, Social

Sciences and Law and Engineering and Architecture.

•Global Commissions for Bachelor/Master/Doctorate

•Cooperation with regional agencies (those in ENQA)

Evaluation Procedures

Evaluation ex-ante (verification)

•Evaluations Commissions for Bachelor/Master/Doctorate and for

the five tematic areas: Arts and Humanities, Sciences, Health, Social

Sciences and Law and Engineering and Architecture.

•Global Commissions for Bachelor/Master/Doctorate

•Cooperation with regional agencies (those in ENQA)

To check, by non-intrusive means, that the institution is implementing the study programme according to the approved design.

Follow-up procedureSTAGE 2

Seguimiento

Accreditation System in Spain

"Once initiated the implementation of programmes related to the official degrees of the University Register, and RUCT degrees, ANECA or assessment bodies that the autonomous communities determine, the project monitoring will be carried out by the University Council. "

"To this end, the ANECA and corresponding assessment bodies, in collaboration with the Ministry of Education and the corresponding Autonomous Communities jointly develop a protocol that will include defining minimum common core criteria and indicators for monitoring procedure curricula. "

25

Legal Framework

RD 1393/2007 (modif. RD 861/2010), art. 27.2 dice:

AVAP which implements the university quality assessment in the Valencian Community, is in charge of performing the functions described in Article 27.2 of Royal Decree 861/2010 cited above, to develop a protocol that will include defining a minimum common core criteria and indicators for the monitoring procedure curriculum.

Resolution of December 15th 2010, the president of the AVAP, by adopting the Track Protocol for official degrees in the Valencian Community.

Completed by Document Development Monitoring Protocol

26

Legal Framework

Curriculum Development

Acreditation

Verification

Quality Assurance System

Implantation

DEGREES

Sistema de Garantía Interna de Calidad

27

A general view

Tracking

AVAP

PROTOCOL

continuous improvement

- Understand the importance of tracking the official titles of the University

- Analyze the aspects involved in monitoring

- Understand the importance of having unified all the information involved in the Monitoring

28

29

2008, one center participated in the AUDIT program, ANECA (Faculty of Philosophy and Letters)

ANECA Certificate 16/09/2009 Implanted in the Centre during the year 2009/2010

2010, participe with the rest of the UA Centres ANECA Certificates 14/01/2011 Implanted in the Centre during the year 2011/2012

Prepare reports to be sent to verify all degrees and masters of the UA

Including in the corresponding paragraph 9 of the AUDIT program procedures

30

AVAP presents its protocol for monitoring the qualifications (year 2010/11)

Experimentally participated with two masters of the Faculty of Philosophy and Letters

2011/2012 we participated with 5 and 2 Masters Degrees and the internal track of 2 Masters that participated the previous year

Set the framework for quality of the centers, whose composition and functions are set out in Chapter 3 of SGIC:

1.Center Management Team2.Faculty or School Board3.Quality Commission4.Quality Coordinator5.Improvement Groups 6.Qualification Commission (Document support for application verification degrees (Masters) in UA)

31

32

AUDIT Review

A support tool for universities main mission is that universities incorporate continuous improvement strategies, develop and control their actions, revised and redefined to achieve some purposes consistent with the requirements of quality that marks the European Higher Education Area

AIM: to support and enhance development and implementation of Internal Quality Assurance Systems in all faculties and universities in Spain.

What is AUDIT?

AUDIT

33

34

OBJETIVES:

Facilitate the development and implementation of quality systems

Implement a procedure leading to the recognition of such systems

Contribute to the improvement of the quality of academic programs

Give a step towards the construction of a European Quality Space

AUDIT Review

35

Quality Manual

Procedures Manual

Other documents(DOCENTIA, POI, PEUA, etc.)

Records

tells us what to dotells us what to do

tells us how to do ittells us how to do it

AUDIT Review

CONTENTS OF A QUALITY MANUAL

STRATEGIC PROCESSES: PE

36

AUDIT Review

KEY PROCESSES : PC

SUPPORT PROCESSES : PA

MEASUREMENT PROCESSES : PM

37

What is AUDIT?

• December 2010, publication of the resolution establishing what will follow in the universities of the Valencian Community

38

AVAP Monitoring

OBJETIVES:

Ensure effective implementation of the teaching following the curriculum title.

Ensure public availability of information to different stakeholders. Identify any weaknesses in the development of lessons and analyze

the actions taken for correction. Provide recommendations for improvement in the implementation of

the curriculum. Demonstrate progress in the development of the title for the Internal

Quality Assurance System (SIGC) as regards both the review of the implementation of the curriculum and the proposed actions to improve its design and implementation.

Identify best practices for dissemination in the university system

39

AVAP Monitoring

STRUCTURE OF PROCEDURE:

1.Universities elaborate an annual report tracking each new title implanted.2.AVAP constitutes evaluation committees (integrated by renowned academics, EHEA experts and university students) to analyze the reports and other public information.3.AVAP issues individual reports (biennial) to be sent to the universities.4.Universities, if necessary, claim for interim reports.5.AVAP issues final reports tracking each title to be sent to the universities, Ministry and University Council.6.AVAP will publish a global report of the total qualifications tracking and universities of the Valencian Community

AVAP Monitoring

40

CRITERIA FOR MONITORING:

1.Information for the society and future students

2.Information on the development and operation of title

3.Upgrades / modifications of the curriculum

4.Recommendations made in the external evaluation reports

5.Assessing the implementation of the Internal Quality Assurance System (SIGC)

6.Evaluation of indicators

AVAP Monitoring

41

a) Description of the title, general and specific skills e.b) Training schedule.c) If present, specify the profession of the degree

(only in the case of a degree linked to a regulated profession)

d) Regulations for permanency.e) Rules of recognition and credit transfer.f) Admission Profile (access and admission

requirements for the degree, special entrance tests)

g) Pre-registration information (documents to be submitted, deadlines, etc.)

h) ANECA verification report and AVAP report and recommendations.

AVAP Monitoring

F03-PC05F04-PC05

Information for the society and future students Aspects to evaluate

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

a) Availability and adequacy of the guidelinesb) Scheduling of teaching including the structure of

the curriculum, schedules, classrooms.c) List of staff teaching subjects / courses indicating

the teaching category.d) Form in which takes place the end of the old plan

and the implementation of adaptive courses.e) External information practices and the end of work

studies f) Student mobility (mobility organization by title,

centers, exchange programs)g) Commitments to obtain title competence

(qualification procedure in order to assess the progress and results of student learning).

AVAP Monitoring

F03-PC05F04-PC05

Information on the development and operation of titleAspects to evaluate

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

a) The amendments are based on objective and previously analyzed information by the management bodies of the title.

b) The substantial changes that have occurred have been reported to the University Council for assessment.

c) The information that changes as a result of the amendments to the title has been implemented in the corresponding website.

AVAP Monitoring

F03-PC05F04-PC05

Upgrades / modifications of the curriculum Aspects to evaluate

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

a) Actions that have been developed with the recommendations proposed in the Verification Report and where issued, if the AVAP.

b) Actions that follow the recommendations of the monitoring reports of AVAP.

AVAP Monitoring

F03-PC05F04-PC05 Recommendations made in the

external evaluation reportsAspects to evaluate

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

a) SIGC Development, in relation to the qualification monitored and report submitted to the program

b) Development of the mechanisms used to:c) improving the quality of teaching and teachers.d) internships and mobility programs.e) measuring the labor market and the satisfaction of

graduates.f) measure the satisfaction of the various

stakeholders involved in the titleg) attention to suggestions and complaints.h) Compliance with the criteria for termination of title.i) Improvement actions developed from the analysis

of the results.j) Making decisions on curriculum development

based on SGIC.

AVAP Monitoring

F03-PC05F04-PC05

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

Internal Rating System of Quality Assurance Aspects to evaluate

a) Rate of Return of Title (CURSA)b) Title Dropout rate (CURSA)c) Efficiency rate of graduates (CURSA)d) Graduation rate title (CURSA)e) Enrollment ratio (AVAP)f) Rate of supply and demand (AVAP)g) PDI rate with a doctoral degree (AVAP)h) Full time PDI Rate (AVAP)

AVAP Monitoring

F03-PC05F04-PC05

Criterion 1

Criterion 2

Criterion 3

Criterion 4

Criterion 5

Criterion 6

Indicator evaluation

Aspects to evaluate

AVAP Monitoring

F03-PC05F04-PC05

Criterion 1: Information for the society and future students

Criterion 2: Information on the development and operation of title

Criterion 3: Upgrades/ modifications of the curriculum

Criterion 4: Recommendations made in the external evaluation reports

Criterion 5: Internal Rating System of Quality Assurance

Criterion 6:Indicator evaluation

F01-PC05F02-PC05

F01-PM01F02-PM01

SGIC

Degree verification memory

Claims / recommendations

AVAP

Performance reports

Satisfaction reports

Service satisfaction reports

Interest groups satisfaction reports

External evaluations reports

Indicators evaluation

Recommendations

Modify the PC05. Monitoring and improving qualifications

OBJETIVE

Establish the process by which the Centers of the UA held the annual monitoring of official titles, according to memory and verified with respect to the monitoring requirements for proper accreditation according to RD 1393/2007, RD 861/2010 and Resolution of 15 December 2010 of the AVAP, to prepare the annual monitoring report for each of their titles.

49

MonitoringPC05 modification

50

Set different tasks involved in tracking and qualifications in the review and improvement of the system of internal quality assurance

Qualification Coordinator

Quality Center Coordinator

UTC

Vice President with responsibility for Quality

Governing Council

AVAP (established by monitoring protocol)

Monitoring Workflow

51

It is a computer application for sustainable monitoring:

Verified memory AVAP / ANECA arguments or recommendations Changes to verified memory AVAP tracking reports Monitoring reports of the site visit Performance reports to qualifications Employability reports Reports of services satisfaction Reports of stakeholders satisfaction Report complaints and suggestions AVAP indicators System Monitoring of Title Quality Assurance

Proceedings of the Quality Assurance Committee of the Centre

Monitoring. AstUA

52

Monitoring. AstUA

53

Thank you for your attentionThank you for your attention