The Diploma in Technical and Vocational Education

Tracer Study

November 2008

Research Team:Margaret Nare, FCTVE

Montlenyane Selma-Robertson, FCTVETebogo Angel Moreetsi, FCTVE

George Herd, TAKlavs Dahl Christensen, TA

Candy Mbongwe, TASepako Seosenyeng, TA

Table of Contents1 LIST OF ABBREVIATIONS............................................................................................................................... 4

2 ACKNOWLEDGEMENTS................................................................................................................................. 5

3 INTRODUCTION............................................................................................................................................ 6

3.1 THE DIPLOMA IN TECHNICAL AND VOCATIONAL EDUCATION (DTVE).............................................................................63.1.1 Aims of the programme............................................................................................................................73.1.2 Outcomes of the Programme...................................................................................................................83.1.3 The DTVE programme structure...............................................................................................................93.1.4 The facilitation of the programme..........................................................................................................10

4 POLICY BACKGROUND................................................................................................................................ 11

4.1 RESEARCH TEAM................................................................................................................................................11

5 METHODOLOGY.......................................................................................................................................... 12

5.1 PROJECT IMPLEMENTATION PLAN (PIP)..................................................................................................................125.2 PURPOSE AND OBJECTIVES OF THE SURVEY..............................................................................................................12

5.2.1 The research areas.................................................................................................................................125.2.2 Target Groups for the Data Collection....................................................................................................135.2.3 The research questions...........................................................................................................................145.2.4 Detailing the research questions............................................................................................................15

5.3 CONTACT DETAILS OF THE FORMER DTVE STUDENTS.................................................................................................155.4 DEVELOPMENT OF DATA COLLECTION INSTRUMENTS.................................................................................................165.5 DEVELOPMENT OF TEMPLATES FOR REGISTRATION OF DATA.......................................................................................175.6 DATA COLLECTION PROCEDURES............................................................................................................................17

5.6.1 Data from DoE lecturers.........................................................................................................................175.6.2 Data from former DTVE student-teachers not employed in brigades/TCs..............................................175.6.3 Data from brigades/TCs.........................................................................................................................17

5.7 QUALITY ASPECTS...............................................................................................................................................185.7.1 Size of Samples.......................................................................................................................................195.7.2 Reliability................................................................................................................................................215.7.3 Validity...................................................................................................................................................21

6 DATA PRESENTATION................................................................................................................................. 23

6.1 QUESTIONNAIRE FOR DOE LECTURERS. (ANNEX D)...................................................................................................236.1.1 Relevance of the content of the DTVE....................................................................................................236.1.2 The structure of the DTVE programme...................................................................................................256.1.3 Portfolio Assessment..............................................................................................................................27

6.2 QUESTIONNAIRE FOR FORMER DTVE STUDENT-TEACHERS (ANNEX C)...........................................................................296.2.1 Relevance of the content of the DTVE and its use by DTVE graduates....................................................296.2.2 The structure of the DTVE programme...................................................................................................336.2.3 Mode of facilitation used in the DTVE programme.................................................................................356.2.4 Quality of facilitation by DoE lecturers...................................................................................................376.2.5 DTVE student-teacher perception on portfolio assessment....................................................................396.2.6 Effectiveness of the DoE support system for the DTVE programme........................................................40

6.3 COMPARING RESPONSES FROM DOE LECTURERS WITH RESPONSES OF THE DTVE STUDENT-TEACHERS................................446.3.1 DTVE programme content......................................................................................................................44

TABLE 7 AVERAGE RELEVANCE RATINGS FOR CONTENT OF MODULES DN01, DP01 AND DP02 BY DOE LECTURERS AND DTVE STUDENT-TEACHERS......................................................................................................................................................44

6.3.2 DTVE programme structure....................................................................................................................466.3.3 DTVE portfolio assessment system.........................................................................................................476.3.4 DTVE programme support system..........................................................................................................48

6.4 STRUCTURED FOCUS GROUP INTERVIEW: MANAGEMENT AND SUPERVISORS OF DTVE GRADUATES.....................................48

DTVE Survey Page 1

6.5 STRUCTURED FOCUS GROUP INTERVIEW: LEARNERS TAUGHT BY DTVE GRADUATES..........................................................506.6 COMPLETING THE DTVE PROGRAMME...................................................................................................................516.7 CONCERNS RAISED..............................................................................................................................................54

7 CONCLUSIONS AND RECOMMENDATIONS...................................................................................................58

7.1 CURRICULUM.....................................................................................................................................................58INTEGRATED KEY SKILLS (KS)..........................................................................................................................................58FACILITATION/FACILITATORS...........................................................................................................................................59THE PROGRAMME STRUCTURE/PORTFOLIO ASSESSMENT.....................................................................................................60LEARNING SUPPORT TO STUDENTS...................................................................................................................................61TEACHING PLACEMENT (TP)...........................................................................................................................................62

8 LITERATURE REFERENCE.............................................................................................................................. 64

8.1 EXIT SURVEY – DTVE DIPLOMA IN TECHNICAL AND VOCATIONAL EDUCATION PROGRAMME............................................648.2 BOTA SURVEY...................................................................................................................................................648.3 CONSULTANCY TO CONDUCT A TRACER STUDY OF GRADUATES OF THE BOTSWANA TECHNICAL EDUCATION PROGRAMME, 2007

65

DTVE Survey Page 2

Table of tables

Table 1: DTVE Enrolment 2001 - 2008-11-24...............................................................................................................7Table 2: DTVE modules and credit values....................................................................................................................9Table 3: Response rate...............................................................................................................................................19Table 4: Representation of categories among respondents.......................................................................................20Table 5: Representation of cohorts among respondents...........................................................................................20Table 6: Frequencies of responses on item 2.16........................................................................................................36Table 7: Average relevance ratings for content of modules DN01, DP01 and DP02 by DoE lecturers and DTVE student teachers.....................................................................................................................................................................44Table 8: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers..............................................................................................................................................45Table 9: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers..............................................................................................................................................46Table 10: Comparing ratings of related to programme structure by DoE lecturers and DTVE graduates..................47Table 11 Comparing of average ratings on items related to portfolio assessment by DoE lecturers and DTVE graduates...................................................................................................................................................................47Table 12 Colleges and brigades visited and number of respondents in the focus group interviews with management / supervisors of the DTVE graduates............................................................................................................................48Table 13: Colleges and brigades visited and number of learners participating in the focus group interview.............50Table 14: Data on withdrawal, supplementation, pass and fail. DTVE cohorts 1-4....................................................52Table 15: Reasons for withdrawing from DTVE programme......................................................................................52Table 16: Pass/fail rates as percent of completers.....................................................................................................53

Table of diagrams

DIAGRAM 1: BAR CHART ILLUSTRATING THE RELEVANCE RATING (SCALE 1 TO 5) BY DOE LECTURERSRS.............................................24DIAGRAM 2: RELEVANCE OF MODULE CONTENT AS PERCEIVED BY DOE LECTURERS....................................................................25DIAGRAM 3: RELEVANCE OF MODULE CONTENT AS PERCEIVED BY DOE LECTURERS....................................................................26DIAGRAM 4: STRUCTURE OF THE DTVE PROGRAMME.............................................................................................................27DIAGRAM 5: PORTFOLIO ASSESSMENT MODE AND PROCESS AS PERCEIVED BY DOE LECTURERS........................................................29DIAGRAM 6: RELEVANCE AND USE OF MODULES COMPARED.....................................................................................................31DIAGRAM 7: MODULES CONTENT RELEVANCE AND USE AS PERCEIVED BY DTVE GRADUATES..........................................................32DIAGRAM 8: MODULES CONTENT RELEVANCE AND USE AS PERCEIVED BY DTVE GRADUATES..........................................................33DIAGRAM 9: STRUCTURE OF THE DTVE PROGRAMME VIEWS OF DTVE GRADUATES.....................................................................35DIAGRAM 10: FACILITATION MODES USED IN THE DTVE PROGRAMME RATING OF........................................................................36EFFECTIVENESS AND FREQUENCY OF USE................................................................................................................................36DIAGRAM 11: PIE CHART ILLUSTRATING VIEWS ON VARIETY IN FACILITATION................................................................................37DIAGRAM 12: PIE CHART DISPLAYING STUDENT-TEACHERS VIEWS ON CONSTRUCTIVIST,..................................................................38LEARNER-CENTRED FACILITATION MODES USED BY DOE LECTURERS.............................................................................................38DIAGRAM 13 FORMER DTVE STUDENT-TEACHERS VIEWS ON THE QUALITY OF THEIR FACILITATORS..................................................39DIAGRAM 14: BAR CHART OF AVERAGE AGREEMENT RATINGS ON ITEMS ON PORTFOLIO ASSESSMENT MODE NAD PROCESS AS PERCEIVED BY

THE FORMER DTVE STUDENT-TEACHERS......................................................................................................................40DIAGRAM 15: BAR CHART OF AVERAGE AGREEMENT RATINGS ON SUPPORT RELATED ISSUES IN THE DTVE PROGRAMME AS PERCEIVED BY

FORMER DTVE STUDENTS.........................................................................................................................................42DIAGRAM 16: AVERAGE AGREEMENT RATINGS ON ITEMS RELATED TO SUPPORT DURING SUPPLEMENTATION AND SUPPORT RO DISTANCE

LEARNERS AS PERCEIVED BY FORMER DTVE STUDENTS...................................................................................................44

DTVE Survey Page 3

1 List of Abbreviations

APP Annual Performance PlanATTC Automotive Trades Technical CollegeBOTA Botswana Training AuthorityBTEP Botswana Technical Education ProgrammeCTVE College of Technical and Vocational EducationDoE Department of EducationDTVE Diploma in Technical and Vocational EducationDTVET Department of Technical and Vocational Education and TrainingFCTVE Francistown College of Technical and Vocational EducationGTC Gaborone Technical CollegeICT Information and Communication TechnologyJTeC Jwaneng Technical CollegeMTC Maun Technical CollegeNCC National Craft CertificateNCQF National Credit and Qualification FrameworkNDP # National Development Plan #ODeL Open, Distance and eLearningOBE Outcomes Based EducationPaTeCo Palapye Technical CollegePiP Project Implementation PlanRNPE Revised National Policy on EducationSPTeCo Selebi Phikwe Technical CollegeTA Technical AssistantTC Technical CollegeTEC Tertiary Education CouncilTP Teaching placementTS Tracer StudyTVET Technical and Vocational Education and Training VBA Visual Basic Applications

DTVE Survey Page 4

2 Acknowledgements

The tracer study group would like to express sincere gratitude to everyone who contributed to the collection of data for this study. We are highly indebted to the Brigade Coordinators and Principals of the Technical Colleges (TCs) who assisted by facilitating and organising the visits of the data collectors to their institutions. We especially appreciate all learners and staff who contributed by either completing the lengthy questionnaire - the former students on the Diploma in Technical and Vocational Education (DTVE), or who participated in a focus group interview - institutional managers and learners. We thank them all for their time and input.

DTVE Survey Page 5

3 Introduction

The main objective of this Tracer Study (TS) is to provide data and information to support a review of the Diploma in Technical and Vocational Education (DTVE) which is the responsibility of the Department of Technical and Vocational Education and Training (DTVET) and the management of the Francistown College of Technical and Vocational Education (FCTVE). The study is timely, in view of the significant changes envisaged for the Technical and Vocational Education and Training (TVET) sector of Botswana.

DTVET will become responsible for the management and funding of the 41 Brigades to support the expansion and standardisation to the Botswana Technical Education Programme (BTEP) qualification framework. A significant staff development requirement will arise since many of the brigade instructors do not have formal teacher training qualifications and are unfamiliar with the BTEP.

The Tertiary Education Council (TEC) is in the process of taking management and funding responsibility for the TVET sector with significant operational implications for the TCs

The National Credit and Qualification Framework (NCQF) is under development in an attempt to define and standardise the qualification levels of all education sectors in Botswana.

National Craft Certificates (NCC) units are under review Proposals for a certificate and an advanced certificate in Teacher training, within the BTEP

framework, have been presented to DTVET.

The study involved the tracing of former students of the DTVE programme from 2001 onwards to gather data on their experiences, both during the programme, and their subsequent involvement within the TVET sector. This report presents the data gathered, and suggests conclusions and recommendations.

3.1 The Diploma in Technical and Vocational Education (DTVE)

The DTVE is an 18 month, full-time teacher training programme offered by the Department of Education (DoE) at FCTVE. The entry requirement is normally a diploma or first degree in a vocational field. The programme is modularised, credit and outcomes-based.

Until the end of 2007 The DTVE programme was originally offered by the autonomous institution – The College of Technical and Vocational Education (CTVE) located in Gaborone. The first cohort of students was enrolled in 2001. At the end of 2007, the college relocated to Francistown to become the DoE at FCTVE.

The primary aims of the original college were defined by the RNPE Government Paper Number 2 of 1994 recommendation 62 (a), (see NDP 8 chapter 15, paragraph 15.138) as:

to provide pre-service training for teachers / instructors in the TVET sector to provide in-service ,staff-development training for the persons working in the TVET sector

NDP 9 in section 15.122 describes the major task of the DoE at FCTVE as to provide lecturers with professional teaching skills using both full-time course and distance/e-

learning delivery

Currently, the only programme offered by the DoE is the DTVE full-time programme in affiliation with the University of Botswana (UB).

DTVE Survey Page 6

Since 2001, the enrolment numbers are as follows:

Table 1: DTVE Enrolment: 2001 - 2008

Sept 2001 - May2003Cohort 1

July2003 - Dec2004Cohort 2

Jan2005 – July2006Jan 2005 – May 2008

Cohort 3

Sept2006 - June2008Cohort 4

TOTAL

Full time programme Full time programmeFull time Block

release

Full time programme

Pre-service

In-service

Pre-service In-service

Pre-service

In- service

12 3 27 18 28 Pre23 In

15 33 17

15 45 66 50 176*

The cohort of 80 students enrolled in 2008 and currently studying is not included in the table and was not used to gather information for this TS.

A total of 174 students, in 4 cohorts have started the DVET programme.

*The above total is 176 - 2 students starting in cohort 2 left due to illness and continued in cohort 3.

3.1.1 Aims of the programme

The DTVE curriculum aims to provide opportunities for student-teachers to:

a) become reflective and critical professional practitioners in the vocational education and training sector;

b) acquire knowledge of relevant educational theory related to the system in which they already work or plan to enter following the programme;

c) develop a wide range of strategies, tactics and expertise necessary for planning, preparing, implementing, assessing and evaluating teaching and learning sessions for the subjects and groups of learners or trainees with which they will be working;

d) integrate in the facilitating of learning, studies relating to inter-personal relationships and the development of key skills associated with their intended or actual work role;

e) acquire knowledge and skills to respond to the needs of their learners;

f) identify, select and use a range of resources to support learning sessions, including ICT;

g) acquire confidence in their professional knowledge and problem-solving skills, developing their own personal philosophy of education and commitment to and critical awareness of their professional situation;

h) commence the development of a positive and professional attitude to the philosophy of life long learning and professional development;

DTVE Survey Page 7

i) gain access to a ladder of awards, each of which matches their training and professional needs and those of their employers and learners;

j) identify barriers faced by learners in education and training such as disability, age, race and gender and promote professional practice that does not exclude groups of learners but recognises and values their diversity;

k) study teaching and training situations within the social, psychological and philosophical context and to relate these to changes in industry and to Government policy.

3.1.2 Outcomes of the Programme

On successful completion of the DTVE Programme the student-teacher will:

a) exhibit non-discriminatory beliefs, values and behaviour, in respect to disability, race, gender, sexual orientation, culture or ethnicity in all written work, teaching practice, work based experience and their day to day involvement with the Programme tutors, peers, learners, and others with whom they work or study;

b) have developed skills and understanding in relation to the use and application of information and communication technologies, sufficient to carry out their intended or expected role in technical/vocational education and training;

c) have demonstrated appropriate skills in researching, analysing and using information gathered from a variety of sources such as libraries, internet, intranet, world wide web, CD-ROMs, newspapers, journals and educational texts such as syllabuses and unit specifications;

d) be able to design, plan, implement, assess and evaluate learning programmes which take into account the diverse needs of the vocational learner or trainee and the specialist area in which they will teach;

e) demonstrate appropriate skills and competence in relation to supporting their own learners, monitor progress and provide effective guidance which enhances learning;

f) evaluate their own teaching effectiveness and establish their own development targets;

g) understand the role of assessment in relation to the evaluation of learning, the vocational curriculum and the award of national qualifications related to their own teaching area;

h) understand the factors which influence the development and structure of the vocational education and training curriculum and play an effective part in the planning implementation and evaluation of this curriculum, in relation to their own specialist teaching field;

i) understand the role of vocational education and training in relation to the Botswana Government, economic, social and cultural policy, in relation to their own professional practice;

j) be able to plan and execute work based projects aimed at enhancing the quality and effectiveness of the vocational curriculum, through appropriate levels of enterprise and collaboration with peers, mentors and managers;

DTVE Survey Page 8

k) evaluate their own learning during the programme and contextualise this with their own values and beliefs in relation to their own professional role as a teacher or trainer;

l) develop the knowledge, understanding and skills, needed for effective and unsupervised work in a wide range of contexts in vocational education and training.

3.1.3 The DTVE programme structure.

To achieve the above stated aims and outcomes an 18 month modular programme was designed and piloted with a group of 15 student teachers beginning September 2001. Teaching Placement (TP) at an vocational institution i.e. attachment to a brigade or TC offering BTEP, forms an important, integral part of the programme of which 60% is college based and 40% is TP

The programme structure consists of 10 modules. Each module has a credit value which is an indication of the time an average student-teacher will need to complete the module.1 credit corresponds approximately to 30 hours of study time. This includes contact hours with the tutor, with peers, study in library, work in computer room, individual study, etc. The table below gives an overview of the modules in the programme.

Table 2: DTVE modules and credit values

Module Credit valueDN01 Assess learners’ needs 6DP01 Plan for effective learning 6DP02 Prepare resources to support learning 6DF01 Facilitate learning 8DS01 Provide learners with support 4DA01 Assess learning 4DA02 Evaluate learning sessions 6DE01 Reflect upon roles of a teacher 6DE02 Improve own professional practice 6DE03 Undertake an action research 8TOTAL 60

The complete programme carries a credit value of 60, being equivalent to approximately1800 hours of study. The following “key skills” or generic skills are fully integrated into the 10 modules:

personal and interpersonal skills; information gathering and processing skills (Information and communication technology, ICT); communication skills; numeracy skills; quantitative data processing skills (numeracy); improving own professional performance.

These skills are developed by ensuring they are applied to activities in each and every module. The Key Skill level of each student is assessed at start of the programme and, whenever necessary, student-teachers are assisted and provided with (online) resources to practice and upgrade their skills.

DTVE Survey Page 9

3.1.4 The facilitation of the programme.The DTVE is a practical, activity-based programme founded on the general principles of adult education. Module activities and tasks are related the vocational area of specialisation of each student-teacher where possible. It is intended to be facilitated in a mode exemplary of that specified by the BTEP i.e. learner centred, based on a constructivist view of learning, supported by appropriate use of Information and Communication Technology (ICT).

The evidence for the outcomes, again similar to some BTE programmes, requires portfolio building. The portfolios are assessed in line with the Quality Assurance and Assessment Unit (QAA) following the procedure: unit assessment, internal verification and external verification. The only difference from DTVE is that the external verification is not done by QAA but by UB. In line with all BTE programmes the DTVE programme has assessed work placement periods. DoE refers to these work placement periods as Teaching Placement (TP

The DTVE programme is flexible and individualised. Student-teachers, within limits, can determine the pace of their own progress and the pace at which evidence are gathered for each module portfolio. For this to be successful, an effective support and mentoring system is in place. A college based personal tutor is assigned to each student teacher to provide support and guidance the student-teacher throughout the programme. During TP each student-teacher is attached to a mentor from the institution concerned.

DTVE Survey Page 10

4 Policy background

The Government is committed to providing quality technical and vocational education and training NDP9, section 15.112). The emphasis is on equipping learners with the knowledge, skills and attitudes to contribute to the socio-economic and technological advancement of the country. The Government in its drive to diversify the economy seeks to create a competent, innovative and internationally competitive workforce. To provide quality education one needs quality teachers / instructors.

DTVET’s Strategic Plan (2005 – 2009), FCTVE’s Strategic Plan (2007 – 2009) and FCTVE’s Annual Performance Plan, (APP) 2008-2009incorporated the Government’s objectives mentioned above. One of the initiatives mentioned in the APP, as a quality improvement initiative, is to conduct a tracer study of DTVE graduates. The DTVE programme has been running, with few changes, since its inception in 2001. A review of the programme is required to establish whether the stated aims and objectives are being achieved, and are still relevant to the current needs of DTVET and the vocational sector as a whole.

In a meeting with DoE in February 2008 called by the FCTVE Management attended by the Technical Assistance (TA) Team, the FCTVE Management instructed DoE to run a tracer study (TS) of former students of the DTVE programme with the purpose of informing the review of the programme, which has been offered unchanged since September 2001. The DoE was instructed to present the TS report not later than 30 June 2008. At the same meeting the TA Team committed itself to support DoE in its undertaking of the TS by developing a plan for implementation of the TS including survey instruments and assistance with analysis and reporting. The TAs estimated the DoE staff input – in order to meet the deadline stated by the FCTVE Management - to the TS to be 2 lecturers ½-time for 3 months. It was also underlined by the FCTVE Management that the TA assistance should be seen as capacity building of DoE staff with the objective of enabling DoE staff to undertake future similar surveys. Carrying out surveys of tertiary education institutions, like FCTVE, is an expectation that has been expressed by Tertiary Education Council (TEC) members during their visit to FCTVE in relation to accreditation of the college and its programmes

4.1 Research Team

Following the meeting with the FCTVE Management, DoE nominated the following lecturers from their Department to be responsible for the planning and carrying out of the tracer study:

Margaret Nare Montlenyane Selma-Robertson Tebogo AngelaMoreetsi

To assist DoE the following TAs were nominated:

Jan Deurwaarder George Herd Klavs Dahl Christensen

The three DoE lecturers and the three TAs formed the TS task force. The task force was in September 2008 joined by the following short term TAs:

Candy Mbongwe SepakoSeosenyeng

DTVE Survey Page 11

5 Methodology

This tracer study used quantitative as well as qualitative data. The main study is based on a survey questionnaire to two groups: 1. former DTVE student-teachers and 2.lecturers in the DoE at FCTVE. Qualitative data were collected through structure interviews with two groups 3.management and supervisors of the DTVE graduates employed in brigades and TCs and 4.Learners now being taught by the graduates. Interview guidelines were developed for groups 3 and 4.

The established TS task force had its first meeting in March 2008. Following that, several meetings were planned. However, some meetings had to be cancelled due to high workload of the DoE members.A separate folder was created on the FCTVE Moodle platform to cater for the communication among the task force members. All reference material, ideas, suggestions and decisions were uploaded to the platform.

Due to a high workload on the part of the DoE members of the TS task force, nearly all the work was carried out by the TAs. This meant that the request formulated by the FCTVE Management about capacity building among the DOE members of the TS task force did not occur.

5.1 Project Implementation Plan (PiP)

The first output from the task force was a PiP (Annex A), outlining tasks to be undertaken, responsibility and timeframe. The PiP took its point of departure in the request formulated by FCTVE Management in February 2008 that the survey report should be presented by June 30, 2008. This was due to the inability of DoE members of the task force being unable to meet the various deadlines and contributions stated in the PiP, resulting in a delays in completion of the TS report.

5.2 Purpose and Objectives of the Survey

Following the development of the PiP, the task force focused on defining the purpose and objectives of the survey. The purpose of the survey was already defined by FCTVE Management as:

Provision of information for review of the DTVE programme

5.2.1 The research areas

In order to meet the stated purpose the task force identified the areas that should be the target for the survey.

The relevance of the content of the DTVE programme for the teachers / instructors in order to competently deliver their vocational subject area should obviously be addressed in the survey. But the content should not be assessed in isolation. The content only becomes meaningful if the graduates are able and willing to implement the knowledge, skills and attitudes developed during their study. Consequently, the survey would also investigate to what extent the DTVE graduates were actually applying the skills and knowledge in their daily work as lecturers or instructors.

The quality of any face-to-face programme as perceived by the learners also depends on the quality and mode of facilitation. This is especially the case, when the objective of the programme is to equip the student teachers-with planning, learning resource development, facilitation, assessment and evaluation skills, using an outcome based, and learner centred approach. The mode of facilitation of the DTVE programme should be exemplary (do what you preach).

DTVE Survey Page 12

The learning resources and facilities available to the student-teachers are important in order to enhance their learning in a technological enhanced environment. Then, when in employment in colleges, they can act as examples of good teaching practice to colleagues on the use of technology in learning and facilitation.

The assessment mode of the DTVE programme is through continuous portfolio building for each of the learning modules. Portfolios contain the evidence of having achieved the learning outcomes and document the process towards their achievement. This mode of assessment is unfamiliar to many of the DTVE student-teachers more used to end examinations. It is a mode of assessment that requires continuous support from the DoE lecturers and which serves at the same time as an example to them, as they must be able to use portfolio assessment when they join TCs teaching BTE programmes at advanced and diploma level.

The support system in place for student-teachers was also identified as an area for evaluation, being a key aspect of any educational programme. The views of student teachers on the support system of the DTVE programme form an important aspect of the data collection. Views of those that completed the programme successfully (obtaining their DTVE), those that left before the normal end of the programme, and of those that completed the programme but failed to achieve the DTVE are all relevant for evaluation of the support system.

The final research area identified, focused on what happened to learners on the DTVE programme after they completed or left the programme i.e. post course destinations. Did the newly qualified student-teachers find employment? What happened to those who failed or left the programme early?In conclusion it was decided by the TS task force that in order to meet the purpose of the survey that data should be gathered in the following seven areas:

1. Content, its relevance and extent to which it is used by graduates, and programme structure 2. Mode of facilitation of the programme3. Quality of facilitation4. Learning resources and environment5. Mode of assessment6. Support system7 Post course destinations

However, an issue was raised concerning item 4 above. With the exception of the groups, cohort 4 and cohort 3 (block release), none of the other graduates of the programme would have experience of FCTVE facilities, its learning resources and environment, since they completed their programme at CTVE in Gaborone. Cohort 3 (block release) and cohort 4 were at FCTVE for a relatively few months only, and during a time when the campus was not fully operational. Therefore, it was decided to exclude the above item 4 from the survey as student-teachers would not be in a position to give views on the facilities and resources a full operational FCTVE would offer.

5.2.2 Target Groups for the Data Collection

Discussions in the TS task force indicated that there was a need to collect data not only from those who completed the DTVE programme successfully – the graduates, but also those who did not. Consequently, the student-teacher population targeted consisted of all student-teachers enrolments to the DTVE programme since its inception. In this document this group is referred to, not quit appropriate, as DTVE graduates.

DTVE Survey Page 13

Another group targeted to provide data, were current and former lecturers in the DoE (formerly CTVE). With their experience in facilitating the programme as well as their theoretical background they would be in a position to provide useful information related to the relevance and usefulness of programme content, its mode of facilitation and assessment, and the student support system.

The purpose of the DTVE programme is to produce skilled teachers / instructors for the TVET system in Botswana. It can therefore be expected that former DTVE graduates will be working in a brigades, government TCs or other vocational training institutions as instructors/lecturers1. In order to establish whether DTVE graduates use the knowledge, skills and attitudes acquired in the DTVE programme two groups were identified to provide data.

a. learners in brigades/TCs currently taught by the DTVE graduates b. the management of the institution in which the graduates are working, including their immediate supervisors

This data would focus on the relative performance of DTVE graduates compared with teaching staff with a non DTVE background.

In summary, the target groups for the survey were:

1. Former students enrolled on the DTVE programme since its inception (referred to as DTVE graduates)

2. Lecturers from DoE (current and/or former)3. Learners in brigades/TCs currently taught by the graduates4. Supervisors in brigades/TCs of the graduates

5.2.3 The research questions

The TS posed the following the questions:

1. a. How relevant is the content of the DTVE programme to the work of facilitators of BTEP/ other vocational programmes within the vocational sector of Botswana?b. Do the graduates apply the skills and knowledge which they have developed?c. How do graduates perceive the structure of the programme (Duration, workload, level, sequencing)and the content of modules?

2. Where the modes of facilitation used in the programme conducive to learning?

3. b. How do those taught by the graduates rate the quality of their facilitation?c. How do managers/supervisors in brigades and TCs rate the quality of the graduates?

4. a. How do graduates perceive the portfolio assessment system?b. How do DTVE facilitators perceive the portfolio assessment system?

5. What are graduates views on the support received during / after the programme?

6. a. What is the completion rate / pass rate?b. What are reasons for dropping out from the programme before completion?

1 The data did confirm this - 116 (67%) of the 174 graduates identified were employed in either a brigade or TC

DTVE Survey Page 14

c. Where are graduates employed after completion?d. What is the time gap between completion and employment?

5.2.4 Detailing the research questions

For each of the research questions a key performance indicator (KPI) with a standard was formulated. The data source / respondents to supply data and the data collection instrument to be used to collect the data were identified. The person(s) responsible for collecting these data were identified.

To ensure that the questions included in the data collection instruments address the objective a pro forma document “justifying the questions” was designed (Annex B). The DoE members were to complete these ‘justification’ documents with the Department as a form of validation that the correct questions were asked and ‘nice to know’ questions, not related to the set objectives were not included. Time constraints did not allow this part of validation of the questions to take place.

5.3 Contact details of the former DTVE Students

The records of former students of the DoE were requested. However, the lists did not include the latest contact details for graduates or their completion status (passed with or without supplementation, failed). There was also no indication of DTVE student teachers that withdrew from the course before the normal end date.

As graduates tend to be mobile, contact details current when the graduates left the DoE become quickly outdated. Those not in employment within the TVET system were therefore particularly difficult to trace. Considerable energy and time was therefore needed to trace each individual. Strategies for generating a complete list of DTVE graduates were as follows:

Records of the DoE on the 4 cohorts were the starting point, though they were found subsequently to be incomplete. Each student teacher admitted to the DTVE programme receives a unique ID number. The list of former student-teachers submitted to the tracer study group had some IDs omitted.

Admission records for cohorts 1 to 3 from a former DTVE lecturer were examined to supplement, where appropriate, the records by DoE management. The Deputy Principal resources at FCTVE (former HoD in the DoE) provided a fairly comprehensive for the student teachers in cohort 4.

The above strategy produced a list of 174 graduates. The list has a few ‘missing IDs’ i.e. ID numbers without a name attached. However the missing IDs could be accounted for as it was found that for some cohorts (especially 3 and 4) IDs were given to all student-teachers offered admission to the DTVE programme. Some having been offered a place on the programme did take up the offer i.e. never started the programme resulting in ‘empty’ IDs. The TS working group is confident that all DTVE graduates were however properly identified. The next task was to trace the current location and contact details of the graduates.

a. Strategy for tracing graduates working in the TCs or brigades Staff records from the TCs and the database containing the data of all persons working in the

brigades were used to identify the graduates currently working in these institutions. This allowed 116 (66%) of the graduates to be located, 77 in brigades and 39 in TCs.

DTVE Survey Page 15

A snowball technique was used whereby graduates who had been located were asked for assistance in tracing other graduates from their year or other cohorts. A further 36 graduates were traced.

22 graduates (13% of the total) could not be traced.

In summary, a total of 152 (87%) graduates were traced and contacted.

5.4 Development of Data Collection Instruments

The following data collection instruments were developed|:

a. A questionnaire for former DTVE student-teachers (graduates). (Annex C)b. A questionnaire for former and/or current lecturers in the DoE facilitating the DTVE programme. (Annex D)c. Guidelines for a structured focus group interview with managers / supervisors of DTVE graduates. (Annex E)d. Guidelines for a structured focus group interview with learners currently taught by DTVE graduates. (Annex E)

The steps in developing the instruments were the following:

a. Documents detailing the research questions in each of the six areas included a section with draft / suggested questions that should be answered by the identified populations in order to collect data for answering the research question. . These 6 documents were made available to members of the TS workgroup for scrutiny and review, together with a pro forma checklist to justify each question (See annex B) suggested for inclusion in the data collection instruments. The workgroup members based in the DoE were tasked with presenting the documents within the Department and include Departmental feedback and review. This is a low level form of establishing the validity of the data collection instrument. No feedback was received and hence it was assumed that the documents could be used for the next stage without any change.

b. The suggested questions in the 6 documents were collated into 4 separate documents: the first draft of the 2 questionnaires and the 2 guidelines for the structured focus group interview. These draft documents were made available, through the Moodle platform to the members of the TS workgroup for review and dissemination in the DoE, together with two checklists. The first checklist was to be used to check each questions, whether it was correctly formulated, short, simple language, not a leading question, etc. The second checklist addresses the layout and structure of the whole questionnaire e.g. where similar questions grouped together, does it start with relatively straight forward questions, etc. No feedback was received from the DoE, but feedback was received from TA members in the TS workgroup.

c. The TA members of the group produced, based on the feedback, a final version of the documents. These were presented to the principal FCTVE for approval, and to the HoD of the DoE for discussion within the Department and approval. Approval was received from principal and HoD of DoE.

DTVE Survey Page 16

5.5 Development of Templates for Registration of Data

Once the questionnaires were developed, templates for entering of the quantitative data were developed. In order to construct the two dimensional XY (Scatter) charts a Microsoft Visual Basic Applications (VBA) macro was developed.

5.6 Data Collection Procedures

Four of the TS workgroup members were involved in the collection of the data working in pairs. To ensure that data were collected in the same way by each pair of data collectors, guidelines for the structured focus group interview with management/supervisors and learners of the DTVE student-teachers were developed (Annex E). Data collection visits to TCs and brigades were structured in the same way to increase the reliability of data collected.

5.6.1 Data from DoE lecturers

The developed and approved questionnaires (Annex D) were distributed to all 14 lecturers in the DoE at FCTVE. Five former lecturers were contacted by e-mail and requested to complete the questionnaire.Of the 14 DoE lecturers 9 (75%) returned completed questionnaires. Three completed questionnaires were received from former DTVE lecturers. Relatively, this is a fair representation of DTVE lecturers involved in facilitating the DTVE programme.

5.6.2 Data from former DTVE student-teachers not employed in brigades/TCs

58 former student teachers (33%) are not working in the TVET system (TC or brigade). Of these 58 former DTVE student teachers 22 could not be traced (13% of the total 174 former DTVE student-teachers). The 36 former DTVE student teacher traced were approached by phone and/or e-mail and questionnaires were dispatched to them through fax, e-mail or by hand. All 36 received the questionnaire. 22 (61%) were received back, some after numerous follow ups through telephone and/or e-mail. A few were collected by visiting the persons at their workplace / house. The collection of these 22 questionnaires turned out to be a tedious and time consuming exercise.

5.6.3 Data from brigades/TCs2

A data collection plan was drawn up by the TS task force, including visits to all TCs and brigades. There are 20 brigades with 39 DTVE graduates. 13 (85%) were visited and data collected from the former DTVE student-teachers management / supervisors and learners. A time plan, scheduling visits to two institutions per day was produced. A standard programme3 for the visits was drawn up. All institutions were contacted by telephone, informed about the purpose of the survey and the programme for the visit. Once the details had been agreed on, the programme was faxed to the institution. Generally, all institutions contacted were very cooperative, and interested in receiving the data collection teams.The 7 brigades not visited by the data collection teams were contacted by phone and fax. Questionnaires for the former DTVE student-teachers were supplied through fax. The completed questionnaires were to be faxed back. A total 34 (87%) completed questionnaires were obtained from DTVE student-teachers in the brigades and 49 (64%) from DTVE graduates working in the colleges.

2 List of all institutions visited is found in Annex G3 The programme is found in the Annex H

DTVE Survey Page 17

Former DTVE studentsAt the brigades/TCs the graduates of the DTVE programme were addressed as a group. A brief introduction explaining the purpose and the objectives of the data collection exercise was given. Following that, the questionnaires were handed out to the instructors/lecturers for completion. The time for completing the questionnaire was approximately 1 – 1½ hour. This is generally too long for a questionnaire, however as the DTVE former student-teachers were sitting together, discussing issues, recalling good and bad experience during their stay at the college and given the time by management to complete the questionnaire in all but one instance this worked well. In GTC the former DTVE student-teachers refused to complete the questionnaire as they had, according to them, raised issues with DTVET to which no response was received and hence they felt that there was no need for them to complete the questionnaire as ‘nobody will attend to what we say’ and ‘we completed a questionnaire when exiting the college, so this is a duplication’.With 49 (64%) of the 77 former DTVE student-teachers currently employed in the TCs completing the questionnaire, sufficient data were collected. After, and during completion of the questionnaires the respondents were encouraged to raise issues not covered in the questionnaire. A number of concerns related to recognition of the DTVE, BTEP phase training, the BTEP, continuous staff development, and other were brought forward. These are presented in section 7 of this report.

Management, supervisors of the instructors/lecturers in the brigades/TCsWhile the instructors/lecturers completed their questionnaires, the research team interviewed the supervisors. In the brigades the supervisors comprised the brigade coordinator, the training coordinator and in some cases HoDs. In the TCs the supervisors generally were the HoDs. In several instances members of the brigade/college management participated, e.g. the centre contact, deputy principal academic. The structured interview was conducted, following the guidelines produced (Annex E). One member from the data collection team would read the questions and both researchers would write down answers for later comparison and compilation. Purpose of the interview was explained to the managers / supervisors emphasising that in answering the questions they should make comparison between those of their staff who had followed the DTVE programme and those who had not. The interviews lasted approximately 45 minutes.

Learners in the brigades/TCs currently being taught by DTVE graduatesAfter interviewing the supervisors, learners taught by the DTVE graduates were gathered and interviewed. On average 3-4 learners of each DTVE graduate were requested to participate in the focus group interview. In institutions with many graduates the number of learners per. graduate could be smaller to keep the total number of learners in around the 15 range for a manageable group and ensure participation of all the learners were briefed on the purpose of the interview and its confidence. The names of the DTVE graduates were mentioned to them and they were requested in their answers to keep these instructors / lecturers in mind and compare them with those of their instructors/lecturers who had not completed the DTVE programme. During the interview learners raised issues not covered in the guidelines and/or not directly related to the facilitation of the DTVE graduates. Yet some of the concerns are relevant to the TVET system in general and hence included in section 7 of this report.

5.7 Quality Aspects

In this section issues of sample sizes, reliability and validity will be discussed

DTVE Survey Page 18

5.7.1 Size of Samples

The sample sizes measured by respondents as compared to the populations are as follows:

Table 3: Response rate

Target group Population size Number of respondents %

DoE lecturers Lecturers in DoE: 14

Former lecturers traced 5

9

3

75%

60%Former DTVE students

174 105 60%

Learners taught by DTVE graduates

Not known 195 -

Management / Supervisors of DTVE graduates

Not known 59 -

A 60% response rate from all the DTVE former students and a 69% response rate from those traced and contacted is high for tracer studies. The data collected are more than sufficient to draw conclusions as they may be considered representative for the whole population.

The tables 4 and 5 present the data on how the different categories in the population of 174 DTVE former students are represented in the sample of 105 respondents.

DTVE Survey Page 19

Table 4 Representation of categories among respondents

Population Number Number respondents

Male 94 58 (62%)Female 80 47 (59%)

In service 75 56 (75%)Pre service 99 49 (60%)

Full time 159 93 (58%)Block release 15 12 (80%)

In the total population of 174 DTVE graduates three categorisations can be used By gender: male or female By background: in service (employed in the TVET sector and send for teacher training) or pre

service (mainly following completion of diploma course in a vocational area but without work experience in the TVET system)

By mode of studying: in full-time face to face mode or in block release mode

Full-time face to face student-teachers train for 18 months at the DoE. The block release student-teachers (all in-service) train at DoE for one academic term (with a front load of two terms) and work at their institution the next term. It takes these student-teachers 3 years to complete the programme.Table 4 gives the data on the number of respondents in each category as a percent of that category in the population. With a representation of about 60% to 70% in the respondent groups all categories are well represented.

Table 5 Representation of cohort among respondentsCohort Number in population Number

respondingC1 15 5 (33%)C2 43

(actual 45, but2 transferred to C3)24 (56%)

C3 51 32 (63%)C3 block 15 12 (80%)

C4 50 32 (64%)Total 174 105 (60%)

Table 5 gives the percent of each cohort among the 105 respondents. Cohort 1 is not surprisingly least represented as they are most difficult to trace. 6 (40%) could not be traced of the 9 traced and supplied with the questionnaire 5 (56%) responded. The other cohorts have a 60% representation, with the block release group 80%. As this group left in May 2008 and an exit structured focus group interview was carried out with this group, data on the group were available. In addition all student-

DTVE Survey Page 20

teachers in this group were in service and hence could be easily contacted in their workplace in a brigade or a TC.

From the above data one might conclude that the 105 respondents are a good representation of the population of 174 on the variable mentioned.

5.7.2 Reliability

A reliable data collection instrument is one that would give the same result if repeatedly used with the same or an identical group.In order to assess whether the data collection instruments were reliable the data collection instruments were piloted at Palapye Technical College and Selebi Phikwe Technical College. The pilot did not reveal any conceptual problems with the instruments. When the data collection instruments were rolled out to other colleges and brigades, no significant difference between the responses from the two colleges and the responses from the two pilot colleges were observed.More systematically the internal reliability of the two questionnaires was estimated by applying Cronbach´s as an estimator4 that quantifies the reliability of a score of several questions in a questionnaire. The estimator will take values between 0 and 1. In most cases an acceptable minimum score is 0.7. The estimator was calculated for all sections in the two questionnaires with numerical data. The estimator provided values to underline that the two questionnaires can be regarded as having acceptable reliability.

For the calculated means (average rating) of the scores on a nominal 1 to 5 scale the 95% confidence intervals were computed. These intervals will with 95% certainty contain the true value in the population.

5.7.3 Validity

It is only after establishing the reliability of the instrument that validity is to be considered.Validity of a data collection instrument refers to the question “Is the instrument or measure used measuring what it is intended to measure?” To assess the validly of the data collection instruments the method of “consensual members checks” was used.

The data collection instruments were presented by the DoE members of the task force for feedback, comments and alterations to all DoE staff members, together with “the question justification document”(Annex B) Each question in the instrument was to be justified: why is the question asked? To what research question is it attempting to find an answer to? As no feedback was received it was assumed that the instruments were valid. As professional educators staff in the DoE wants to collect valid data from the DTVE student teachers in order to review content, mode of facilitation, assessment and support of the programme offered. The lecturers in DoE being the experts on the DTVE programme are therefore the most knowledgeable persons to assess the content validity of the data collection instruments.The data collection instruments do allow to collect data on the relevance and use of the DTVE programme content, its mode of facilitation, its facilitators, its mode of assessment and the support provided as perceived by stakeholders.

4 Detailed description is found in Annex I

DTVE Survey Page 21

This is one aspect of the DTVE programme. The aspect NOT addressed is how the DTVE programme compares to vocational teacher training programmes offered in the region and internationally. Is the programme in line with current developments and thinking related to the training of teachers for the TVET sector in the 21st century? This is an important but different aspect. This however, raises the question whether the survey actually is providing answers to all the relevant questions. An independent researcher with relevant background in pedagogical training of vocational lecturers might need to look at the DTVE programme from these different angles. This is further elaborated under Recommendations.

DTVE Survey Page 22

6 Data presentation

In this section the data collected are presented.

6.1 Questionnaire for DoE lecturers. (Annex D)

The facilitators in the DoE have on average 3.3 years of experience with the programme. 33% of the lecturers have facilitated 6 or more of the 10 modules in the DTVE programme.

6.1.1 Relevance of the content of the DTVE

On a scale from 1 (very irrelevant) to 5 (very relevant) the relevance of each content aspect of the programme has an average rating from 4.3 to 5.0. As key performance indicator was set the value 4. Hence the content of the DTVE programme is considered to be relevant for student-teachers to become competent and efficient facilitators.The average relevance rating for the 10 modules is in the range 4.6 to 5.0. Lecturers view the 10 modules as very relevant.The lowest overall rating is scored on the integrated key skills (average score 4.3, closer to the ‘partly relevant’ score of the relevant rating scale.

Diagram 1 illustrates lecturers´ ratings of the relevance of each of the 10 modules and of the incorporated key skills (KS)

Diagram 1: Bar chart illustrating the relevance rating (scale 1 to 5) by DoE lecturers

1.01.52.02.53.03.54.04.55.0

Relevance of the 10 Modules as percieved by DoE Lecturers n = 12

Ratin

g on

1 to

5 sc

ale

rs

DTVE Survey Page 23

The relevance rating of the elements of content of each of the modules, and of integrated key skills is presented in diagram 2 and diagram 3

Diagram 2: RELEVANCE of module content as perceived by DoE lecturers

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Content of DN01, DP01, DP02, DF01 and DS01; its RELEVANCE as percieved by DoE Lecturers n = 12

Ratin

g on

1 to

5 sc

ale

DTVE Survey Page 24

Diagram 3: RELEVANCE of module content as perceived by DoE lecturers

1.01.52.02.53.03.54.04.55.0

Content of DA01, DA02, DE01, DE02; DE03 and KS; RELEVANCE as percieved by DoE Lecturers n = 12

Ratin

g on

1 to

5 S

cale

The lecturers identified in their individual comments 4 content areas that, in their opinion, need to be included and/or strengthened in the programme

Integrating of HIV/AIDS related issues Use of modern technology in facilitation and learning (e-learning) Underpinning knowledge related to adult learning principles and psychology of learning Aspects of school management and administration (professional conduct, academic rules)

6.1.2 The structure of the DTVE programme

Lecturers consider the DTVE programme as a very relevant teacher training programme. However, as for the structure of the programme on several statements the key performance indicator level of 4 (on the 1 to 5 scale) was not attained. The average rating on the 1 to 5 agreement scale (from strongly disagree to strongly agree) are displayed in diagram 4

DTVE Survey Page 25

Diagram 4: Structure of the DTVE programme

2.01 Enjoy f

acilita

tion

2.02 Programme w

ell plan

ned

2.03 Sequen

cing O

K

2.04 Content le

vel O

K

2.05Time t

o complet

e OK

2.06 Workl

oad f.

learners

OK

2.07 Workl

oad f.

Lecturer

s OK

2.08 High qualit

y progra

mme

2.09 Well

integrati

on betwen

n colleg

e and W

P

2.10 Studen

t well

prepare

d f. WP

2.11 Flexib

le pro

gramme

2.12 Adequate

time for T

P

2.13 Well

integrat

ed co

ntinous assm

ent

2.14 Adult Prin

ciples

of Teac

hing applie

d

2.15 Effecti

ve fac

ilitati

on

2.16 Efficie

nt facili

tation

2.17 Efficie

nt support s

ystem

s

2.18 Effecti

ve support t

o noncomplet

ers1.01.52.02.53.03.54.04.55.0

DoE lecturers perception of the structure of the DTVE programme n= 12

Ratin

g on

1 to

5 S

cale

Just half of the responding lecturers feel that the programme is well planned. The weakness of the programme reported is time related or support related.

In relation to time the majority of lecturers stated that Time to complete the programme by learners is inadequate Workload for learners and lecturers is too high Time for teaching placement is insufficient

In relation to support the majority of lecturers feel that the support for student teacher on the programme is inefficient failing the programme (and having to complete in distance mode) is not effective

The items related to the above all score 3 or below. Time is a major issue in the programme (work overload) and there is need when reviewing the programme to look for ways to make the programme more ‘time’ friendly.

DTVE Survey Page 26

6.1.3 Portfolio Assessment

The views of the 12 DoE lecturers related to the portfolio assessment mode and process indicate that the lecturer consider the portfolio assessment system to be (average scores 4.1 to 4.8)

the most appropriate for OBE encouraging self directed learning and reflective practice motivating valid fair in demonstrating the progress towards the achievement of theLOs transparent to student-teachers

Diagram 5 illustrates the agreement rating on the statements in the questionnaire on the 1 to 5 agreement scale.

DTVE Survey Page 27

Diagram 5: Portfolio assessment mode and process as perceived by DoE lecturers

3.01 Appropriate

3.02 Eviden

ce for a

ll LOs

3.03 Self d

irecte

d Learn

ing

3.04 Demonstr

ates p

rogre

ss toward

s Los

3.05 Encourag

es refl

ection

3.06 Monito

ring t

ool

3.07 Valid

3.08 Fair

3.09 Motiva

ting

3.10 Reliable

3.11 Ensures

studen

t Authen

ticity

3.12 Tran

spare

nt

3.13 Encourag

es cri

tical th

inking

3.14 Not extr

a time c

onsuming1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Portfolio Assessment evaluated by DoE lecturers n = 12Ra

ting

on 1

ro 5

Sca

le

The majority consider the portfolio assessment mode more time consuming than other more traditional ways of assessment of learning. About one third of the lecturers doubt whether the assessment is facilitator independent and can ensure authentic of the work produced by learners.

The above responses are amplified in the responses to the open questions in this section of the questionnaire. The motivating aspect, quality assurance (IV/ EV) process and the learner centred nature of portfolio assessment are mentioned as the strength of this form of assessment.

As weaknesses are mentioned that portfolio assessment is time consuming, brings in issues related to authenticity of the documents placed in the portfolio (plagiarism) by the student-teachers, and for it to function w needs lecturers very familiar with OBE and its assessment.

To combat the weaknesses it is suggested that more, well trained teacher trainer, familiar with OBE should be employed and staff development workshops should be facilitated for new staff not familiar with OBE portfolio assessment.

DTVE Survey Page 28

6.2 Questionnaire for former DTVE student-teachers (Annex C)

104 former DTVE student-teachers returned the questionnaire which was made available to 154 (80%) of the DTVE former student that could be traced from 174. The questionnaire was lengthy and as a result most of the questionnaires returned missed responses to some of the questions. The data as collected are summarised in annex C .

6.2.1 Relevance of the content of the DTVE and its use by DTVE graduates.

The content of the DTVE programme was rated by respondents on its relevance and its use in the work situation. As one might expect the ”USE” ratings are all lower than the ”RELEVANCE” ratings. The content of the DTVE programme is considered by DTVE graduates as (highly ) relevant all content being rated above the performance indicator (4) set. The relevance rates are in the range 4.2 – 5.0.The use frequencies were rated on a nominal scale from never used (1) to very often (5) and fall in the range from 2.5 to 4.8. Diagram 6, 7 and 8 display relevance and use ratings of content

DTVE Survey Page 29

Diagram 6: Relevance and use of modules compared.

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.01.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

DN01

DP01

DP02

DF01

DS01

DA01

DA02

DE01

DE02

DE03

KS

1. Content, its RELEVANCE and FREQUENCY of USE as percieved by former DTVE students n = 105

Relevance rating

Ratin

g fr

eque

ncy

of u

se

Diagram 6 clearly illustrates the lower rating on use than on relevance of the content of the 10 modules and the in-cooperated keys kills.

The DTVE graduates ratings on relevance and use of content was analysed across the following three categorisations of the population

gender: male or female background: in service (employed in the TVET sector and send for teacher training) or pre

service (mainly following completion of diploma course in a vocational area but without work experience in the TVET system)

mode of studying: in full-time face to face mode or in block release mode

DTVE Survey Page 30

Average ratings for each group in each category were computed in Excel and a t-test to test for significant difference at the 95% confidence level was carried out (Annex H). No significance differences were found. The expressed views and given ratings on aspects of the DTVE programme are independent of gender, mode the programme was taken by the student-teacher or whether the student teacher was in service or pre service.

Diagram 7: Modules content relevance and use as perceived by DTVE graduates

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Content of DN01, DP01, DP02, DF01 and DS01; RELEVANCE and FREQUENCY of USE as percieved by former DTVE students n = 105

Relevance Frequency of Use

Ratin

g on

1 to

5 S

cale

DTVE Survey Page 31

Diagram 8: Modules content relevance and use as perceived by DTVE graduates

1.31 Different fo

rmati

ve as

sessm

ent a

ctiviti

es

1.33 Keeping r

ecord

of learn

ers ac

hievem

ents

1.36 Evalu

ate own fa

cilitati

on

1.38 Evalu

ate lea

rning reso

urces

1.41 Keeping p

ersonal

develomen

t log

1.44 Effecti

ve worki

ng rela

tionships

1.46 Championing t

eaching a

nd learn

ing meth

ods

1.48 Proactivit

y on own fu

rther

developmen

t

1.52 Using W

ord

1.54 Use power

point

1.56 Use Web

searc

h

1.58 Use disc

ussing s

kills (

particip

ating)

1.60 Use pres

entati

on skills

to au

dience

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Content of DA01, DA02, DE01, DE02, DE03 and KS; RELEVANCE and FREQUENCY of USE as percieved by former DTVE students n = 105

Relevance Frequency of Use

Ratin

g on

1 to

5 S

cale

The USE of following content was rated 3 or below use of manipulatives in learning sessions use of PowerPoint presentations keeping of personal developmental log action research

The low use of PowerPoint presentations is mainly due to lack of the appropriate technology in the TVET institutions, as reported during interviews.

Content that was mentioned as to be included or strengthened can be place in the following categories

DTVE Survey Page 32

Underpinning theory / knowledge Curriculum development Psychology (adult educational) Learning theories Botswana’s educational laws and policies

Key skills Stress management Time management (2) Data analysis (2) Presentation skills ICT

o Software use such as PowerPoint, Excel (6)o Internet use and search (3)o Video productiono Introducing and training of vocational area specific software (e.g.

AutoCad)Management and administration in education

Principles of educational administration and management Guidance and counselling School resource management

Use of modern technology in education E-learning Video conferencing E-assessment

BTEP phase 1 - 3 Full coverage (and recognition by DTVET) of BTEP phase training

Three of the above areas, underpinning knowledge, use of modern technology and management/administration in education, were also mentioned by the DoE lecturers

6.2.2 The structure of the DTVE programme

The responses to the items in the questionnaire (item 1.63 – 1.78) related to the structure of the DTVE programme indicate that the respondents agree that the programme is enjoyable, interesting, motivating, well planned, of high quality. The average score on the agreement scale on these items was in the range 3.2 to 4.3. Diagram 9 illustrated the strength of agreement to the items in the questionnaire.

Diagram 9: Structure of the DTVE programme views of DTVE graduates

DTVE Survey Page 33

1.63 Enjoye

d Programme

1.64 Programme w

ell plan

ned

1.65 Good sequen

zing o

f Units

1.66 Appropriate

Content

1.67 Programme in

teresti

ng

1.68 Time f

or Completi

on adeq

uate

1.69 Workl

oad ac

ceptab

le

1.70 Opportunities

for own Crea

tivity

1.71 Programme o

f high

Qualit

y

1.72 Programme s

atisfi

ed Ex

pectati

ons

1.73 College W

ork an

d TP well

integrat

ed

1.74 Programme w

ell str

uctured

for T

P

1.75 Sessi

ons cove

ring e

ach Unit w

ell plan

ned

1.76 Programme m

otivating

1.77 Programme c

onsidere

d my P

L and Ex

perien

ce

1.78 Programme fl

exible

1.01.52.02.53.03.54.04.55.0

Statements on the structure of the DTVE programme rated by Former DTVE Students n= 105

Ratin

g on

1 to

5 S

cale

Two items score below the neutral point of 3. Both are time related. Workload is seen as not acceptable and time to complete LOs as insufficient. In the general comments (question 1.79) this is amplified. None of the cohorts completed in the 18 months scheduled and a number of months were added to each course.

In the general comments, remarks related to the structure of the programme restate in many different forms the two low rated statements

The workload is too heavy and Time is insufficient

it can argue that both statements express the same. The time is insufficient to Fully benefit from TP as TP is too much evidence collection focussed instead of developing

classroom management and facilitation skills in a range of situations To ‘learn’, the focus is on evidence collections and document production with insufficient

time to ‘digest’ the underpinning theory. This results in copying / plagiarism / ‘buying’ of evidence from other groups and/or cohorts

One suggestion forwarded was to make all the learning resources for each module available online (e-learning) so student-teachers can work at own pace.

6.2.3 Mode of facilitation used in the DTVE programme.

DTVE Survey Page 34

The items 2.01 to 2.17 requested the former student teachers to give their opinion on the facilitation mode used. The DTVE programme is designed to do what it preaches: using learner centred, blended approaches to supporting student-teachers to acquire knowledge, skills and attitudes. Items 2.01 to 2.11 ask learners to rate effectiveness of various methods used and to rate the frequency these methods are used.Diagram 10 shows a bar chart displaying the average ratings on effectiveness and frequency of use.

Diagram 10: Facilitation modes used in the DTVE programme rating of effectiveness and frequency of use

2.01 lectu

re

2.02 whole

class

discussi

on

2.03 indep

enden

t learn

ing

2.04 role p

lay

2.05 web

based lea

rning

2.06 video

sessi

on

2.07 demonstr

ation

2.08 group w

ork & pres

entati

ons

2.09 PowerPoint s

ession

2.1 coopera

tive lea

rning

2.11 experi

ential l

earn

ing1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Effectiveness and frequency of use ratings of faciltation mode in the DTVE programme n = 105

Effectiveness rateFrequency of use rate

Ratin

g on

1 to

5 S

cale

The methods mentioned are all considered to be (very) effective with ratings in 4.1 to 4.6 range. The uses of the various methods range from 2.8 to 4.2. Least used (rate 2.8) are video based sessions. As the method used will depend on the content it is not surprising that certain methods are used less frequent than others.

DTVE Survey Page 35

Average ratings across gender, programme mode, and in/pre service entry were checked for significant differences using t-test. No significant differences at the 95% confidence level were found.Respondents prefer mainly co-operative learning approaches and dislike teacher centred, lecure type of facilitation mode.The variety in facilitation mode is found to be fine by 82% of the respondents. The pie chart displays the responses.

Diagram 11: Pie chart illustrating views on variety in facilitation

8%

82%

11%

2.15 The variety in the different Facilitation Modes in the DTVE programme as experienced by former Students

n = 105

Too LittleJust FineToo much

As for the mode of facilitation catering for the preferred learning style of the student-teacher, the majority (about 60%) feels that that is true to some extent. The results are in table

Table 6: Frequencies of responses on item 2.16

Mode of facilitation catered for preferred learning style

Frequency Percent

Not at all 2 2%To some extend 61 59%Very well 40 39%

DTVE Survey Page 36

The mode of facilitation used by DoE reflects very well (70%) a constructivist, learner centred approached according to the student-teachers. Diagram 12 illustrates in a pie chart student-teachers views.

Diagram 12: Pie chart displaying student-teachers views on constructivist, learner-centred facilitation modes used by DoE lecturers.

3%

27%

70%

2.17 Facilitation Modes used in the DTVE programme reflected a constructivist, learner centred Approach

n = 105

Not at allTo some extentVery well

Views of former DTVE student teachers on variety in facilitation, link between mode of facilitation and own preferred learning style and whether facilitators use a learner centred constructivist approach do not differ significantly across gender, learning mode (FT or BL) or across in/pre-service student teachers.

6.2.4 Quality of facilitation by DoE lecturers.

Section 3 of the DTVE graduates questionnaire collected data on the views of the former students as to the quality of the facilitation they received. ‘Customer’ satisfaction surveys are important for the service provider in order to find out whether or not customers are satisfied. The information obtained can guide well informed improvement plans. In the educational setting the student-teachers are the customers and the lecturers the service providers.

The rating on a scale from very poor (1) to very good (5) are displayed in the bar chart in diagram 13.

DTVE Survey Page 37

Diagram 13 Former DTVE student-teachers views on the quality of their facilitators

3.01 Approachab

le durin

g sess

ions

3.02 Contacted

after

sessio

ns

3.03 communica

ting enthusia

sm

3.04 Approaching s

tudents

w. respect

3.05 Prepare

d for s

essions

3.06 Sound kn

owledge

about to

pic

3.07 Constructive

feed

back on as

signmen

ts

3.08 Return

ing assi

gnmen

ts with

in reaso

nable

time

3.09 Encourag

ing indep

enden

t and cr

eative

thinking

3.10 Using w

ell prep

ared te

aching a

ids

3.11 Using ti

me ava

ilable f

or sess

ions effecti

vely

3.12 Open to

sugg

estions f

rom studen

ts

3.13 Using v

ariaty

of motiva

tional ski

lls

3.14 Continously su

pport stu

dents

3.15 Explai

ning evid

ence

require

ments

clearl

y

3.16 Organisin

g sess

ions in an

inter

esting w

ay

3.17 Planning u

nit/module w

ell

3.18 Starti

ng sess

ions on time

3.19 Main

taning p

ositive

learning a

tmosphere

3.20 Roile model

for learn

er cen

tred ap

proaches

3.21 Encourag

ing the u

se of m

odern te

chnology

3.22 Open to

discussi

ons of o

ther vie

wpoints

3.23 Receptive

to constr

uctive

criticis

m

3.24 Providing a

dequate

guidan

ce ab

out evid

ence

3.25 Fair a

nd unbiased

in as

sessin

g

3.26 Demonstr

ating p

rofes

sionali

sm at

all ti

mes1.01.52.02.53.03.54.04.55.0

Facilitators as percieved by former DTVE Studentsn =105

Ratin

g on

1 to

5 S

cale

Student teachers´ average rate on each item is in the adequate to very good range. Student teachers generally express the opinion that their lecturers are doing a professional job. Student teachers are least satisfied with the turn over time of assignments (rated 3.6). Across the categories used in this survey the average ratings are not significantly different (Annex H).In the open question space student-teachers were positive about the facilitation of their lecturers. Positive critical observations were clearly (i) referring to isolated cases (ii) aimed to improve the learning environment and interaction between student-teachers and lecturers. Issues mentioned by more than one student-teacher Give prompt feedback *6 Ensure timely feedback – well before submission deadline *2 Give constructive feedback, not destructive *3 Treat student teachers with respect / as adults *5 Employ qualified lecturers familiar with the DTVE programme / OBE *7 Employ DTVE graduates as SDFs *6 Lecturers need to come to sessions well prepared *4 Lecturers should be familiar with / inducted into the DTVE programme *7 Lecturers should be open to criticism *2 Inconsistency in assessment among lecturers need to be addressed *3

DTVE Survey Page 38

Lecturers should also be specialist in a particular vocational area *4 Address authenticity / plagiarism *2 Act professional, avoid bias, favouritism

6.2.5 DTVE student-teacher perception on portfolio assessment

DTVE is assessed continuously through portfolio building. Particularly for teacher trainin programmes, portfolio assessment is very appropriate, provided it is used effectively and efficiently. The DTVE portfolios should contain the evidence for the learning outcomes and document the process that lead to the final evidence. Student-teachers, were to rate their agreement with statements related to portfolio assessment. Diagram 14 illustrates the average rating on the portfolio assessment related items

Diagram 14: Bar chart of average agreement ratings on items on portfolio assessment mode nad process as perceived by the former DTVE student-teachers

4.01 Most a

ppropria

te for O

BE

4.02 Ensures

eviden

ce for a

ll Los

4.03 Encourag

es sel

f dire

cted le

arning

4.04 Demonstr

ates p

rogre

s toward

s iden

tified Lo

s

4.05 Encourag

es refl

ectice

practice

4.06 Provision of s

ummative

and su

mmative

opportunities

4.07 High Vali

dity

4.08 Fair

4.09 Motiva

ting

4.10 Reliable

4.11 Ensures

authen

ticity

4.12 Effecti

ve

4.13 Tran

spare

nt

4.14 Allows d

emonstr

ation of sk

ills an

d knowled

ge

4.15 Greatly

enhan

ces cr

itical th

inking

4.16 NOT more

time consu

ming than

other modes

of asse

ssmen

t1.0

2.0

3.0

4.0

5.0

Portfolio Assessment mode and process as percieved by former DTVE udents n =105

Ratin

g on

1 to

5 S

cale

Student teachers were in agreement (score 4 of above) with the statements that portfolio building as used in the DTVE programme is most appropriate, ensures all learning outcomes are covered, gives responsibility to learners for own learning, enhances reflective practice and critical thinking, is motivating, transparent and effective to monitor own progress. These aspects are mentioned in the open items 4.17 and 4.19 where as strength of the portfolio assessment system were mentioned:

DTVE Survey Page 39

Continuous nature of portfolio assessment, the cycle: submit evidence – receive feedback – implement feedback – resubmit

Portfolio assessment (PA) allows to monitor growth and progress PA encourages reflective and critical thinking PA places responsibility for learning and evidence collection with the student teacher PA ensures all LOs are covered PA process is transparent as the portfolio verification document describes in detail the

evidence to be submitted

The below 4 agreement ratings scores are on exactly the same aspects as identified by the DoE lecturers:

Reliability Authenticity Time consuming mode

The three aspects being problematic are described in a variety of ways in item 4.18

Reliability of assessmento Marking is ‘subjective’o Different lecturers use different criteria (You ‘pass’ with lecturer A, but will have to

resubmit with lecturer B) Authenticity of submitted evidence

o Copying from other students (as work will be submitted to different lecturers, hence copying / plagiarism goes unnoticed in nearly all instances)

o Plagiarism from webo ‘buying’ portfolios from previous cohortso Focus on evidence production / assessment – at the cost of learning – encourages

copying due to time pressure and deadlines to be met Time consuming nature of PA

o Too much work within a short timeo ‘not achieved’ for minor short comings [could be resolved using oral evidence as in

BTEP]

The expressed views of the DTVE student teachers on portfolio building are independent (at 95% significance level) of gender, whether they are in- or pre-service student teachers and whether they studied in the full time face to face mode or block release mode (See annex H).

6.2.6 Effectiveness of the DoE support system for the DTVE programme

In all modes of programme delivery the support available and given to learners to achieve the learning outcomes is crucial. One might produce the most engaging, motivating learning materials but without effective and efficient support to learners the materials by itself will not produce the desired outcome i.e. that learner will learn and achieve the set learning outcomes. Within the DTVE programme the student teacher may aspect support and guidance from (i) the module facilitator on

DTVE Survey Page 40

academic aspect and evidence production for the module (ii) his/her personal tutor to receive guidance and support in a holistic way. The (weekly) meetings with the personal tutor will discuss not just a single module but look at the overall picture of the programme and set attainable targets on weekly basis. Action planning is a key feature, as is the addressing of (personal) issues that might negatively impact on achieving set goals. (iii) The TP mentor, the experienced lecturer / instructor in the institution where the student teacher is placed for TP. The mentor inducts supports and assists the student teacher throughout the TP period by observing the student-teacher, giving feedback and supporting in evidence collection.

The questionnaire items 5.01 – 5.12 addressed the above issues. Diagram 14 displays in a bar chart the average agreement ratings on the items related to the availability and receiving of support.

Diagram 15: Bar chart of average agreement ratings on support related issues in the DTVE programme as perceived by former DTVE students

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0Support systems in place as percieved by former DTVE Students

n = 105

Ratin

g on

1 to

5 S

cale

Average agreement ratings on the item seeking view on the support available and given to the DTVE student teachers falls within the 3.4 – 4.2 range. The lowest average agreement rating (3.4) is on the item relating to a personal guidance/counseling, a system to address personal problems negatively affecting the performance of the student teacher. This is not surprising as the DoE has for long been depending on guidance and counselling officers NOT located within the Department. With DoE at FCTVE the current student-teachers can make use of the Guidance and Leaner support system in place within FCTVE.

DTVE Survey Page 41

Overall student teachers perceive the support system available and the support given by their lecturers as adequate for their needs. Among strengths are mentioned (item 5.19):

Facilitators are very supportive throughout the programme *18 Facilitators are committed and readily available for support *3 Strong support given before and during TP Close cooperation between student-teachers and facilitators, and constructive feedback

ensures effective evidence collection *2 Counselling available when needed (‘if the going was tough’) *2 Personal tutor system effective to overcome academic problems Learning resources accessible and available *6

As weaknesses in the support system student-teachers mentioned:

Facilitators are unskilled/less experienced in OBE *6 Feedback provided

o Not timely *8o Inconsistent across facilitators (biased /reflecting favouritism of lecturers) *6o Plagiarism / copying goes undetected *2

Insufficient briefing and support for student-teacher placed in ‘distance learning mode’

Supplementation rates of DTVE students are high. Few pass after final submissions of their portfolios. The supplementation rate for the first cohort was 33%, for the second cohort 10%, while for cohort 3 and 4 the supplementation rate was 100% - no student teacher passed ‘on first attempt’. Those who supplement are provided with extensive feedback on what is to be done in order to meet the evidence requirements; this is part of the support system. Those failing the programme are referred to ‘distance mode’ – they can complete the programme in distance mode. For those having failed and continuing in distance learning mode a support system is crucial, without it they will not be in a position to successfully complete the programme. Diagram 16 is a bar chart displaying the average agreement rate of DTVE former students that were supplementing and/or referred to the distance learning mode.

DTVE Survey Page 42

Diagram 16: Average agreement ratings on items related to support during supplementation and support ro distance learners as perceived by

former DTVE students

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Support to students who had to supplement and to those who did not obtain diploma

If supplemented If no diploma obtained

Ratin

g on

1 to

5 S

cale

The average agreement ratings by supplementing DTVE students on the questionnaire items 5.13 – 5.15 are between 3.5 and 4. This is in the partly agree region. It suggests that the support system during supplementation needs to be critically reviewed to make it more effective and efficient.This corresponds to the rating of the DTVE facilitators on the statement “2.17. The programme has efficient support systems in place” with an average agreement rating of 3.0 – half of the responding facilitators (strongly) disagreed with the statement.

DTVE student-teacher that failed the programme and were placed in ‘distance mode’ rated the items related to support in the range 1.8 to 2.0. In other words they perceive that there is no effective operational system in place to support them. 50% of the facilitators concur with that perception (item 2.18 in the lecturers’ questionnaire score average rating of 2.7)

The expressed views of the DTVE student teachers on the DTVE programme support system are independent (at 95% significance level) of gender, whether they are in- or pre-service student teachers and whether they studied in the full time face to face mode or block release mode (Annex H).

DTVE Survey Page 43

6.3 Comparing responses from DoE lecturers with responses of the DTVE student-teachers.

The questionnaire for lecturers and for DTVE graduates had many identical items. This allows the perceptions of the two groups to be compared and tested for significant differences. (Annex H)

6.3.1 DTVE programme content

The average relevance ratings of content and modules in general by DoE lecturers and DTVE graduates were compared. A t-test was applied at 95% confidence level – no significant difference were found. The ratings of DoE lecturers and of DTVE student-teacher are not in any way different. Table 7 compares the average relevance rating by DoE lecturers and DTV graduates.

Table 7 Average relevance ratings for content of modules DN01, DP01 and DP02 by DoE lecturers and DTVE student-teachers.

DTVE graduates

DoE lecturers

1.01 Diagnose Learners´ Needs 4.6 51.02 Determine Learning Style 4.5 4.81.03 Methods for different needs and learning styles 4.7 51.05 Interpret LU specifications 4.6 51.06 Prepare scheme of work 4.8 51.07 Prepare session plans 4.7 51.09 Different activities to meet LOs 4.7 4.61.10 Written LM to support learning 4.9 4.91.11 Audio/visual LM to support LOs 4.7 51.12 Visual learning aids to support LOs 4.8 4.81.13 Manipulatives to support Los 4.5 4.31.14 OHT to support Los 4.6 4.51.15 PowerPoint to to support LOs 4.6 4.51.16 Internet/Web based resources to support LOs 4.7 4.6

The differences in ratings are very small and not significant at 95% confidence level (Annex H) The content of the modules is rated as (very) relevant by both student-teachers and DoE lecturers. Diagram 16 in a double bar chart illustrates that ratings of the two groups are extremely close to each other or identical in some instances.

DTVE Survey Page 44

Table 8: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers.

DTVE graduates

DoE lecturers

1.18 Group discussions in facilitation 4.8 4.81.19 Cooperative learning spproaches 4.6 51.20 Experimental learning approach 4.7 51.21 Individualised learning approach 4.6 4.81.22 Range of motivational techniques to support LOs 4.7 51.23 Manage conflict situations 4.7 4.71.24 Effective questioning techniques 4.8 4.91.25 Promotion of equal opportunities 4.8 4.51.27 Induction programme 4.8 4.91.28 Support to meet LOs 4.7 4.61.29 Personal support and guidance 4.6 4.51.31 Different formative assessment activities 4.8 51.32 Feedback to support learning 4.9 51.33 Keeping record of learners achievements 4.9 4.91.34 Fair and unbiased assessment 4.8 51.36 Evaluate own facilitation 4.6 4.91.37 Evaluate teaching methods 4.6 4.91.38 Evaluate learning resources 4.6 4.91.40 Being a reflective practitioner 4.2 4.61.41 Keeping personal develoment log 4.0 4.41.43 Evaluate own practice using action plans 4.4 4.51.44 Effective working relationships 4.7 4.51.45 Adhering to high professional standards 4.8 4.51.46 Championing teaching and learning methods 4.7 4.91.47 Enterprise activities with institutions 4.2 4.71.48 Proactivity on own further development 4.7 4.81.50 Undertake action research 4.4 4.91.52 Using Word 4.8 4.71.53 Use speadsheet, e.g. graphing 4.8 4.61.54 Use power point 4.8 4.51.55 Use data analysis techniques 4.8 4.61.56 Use Web search 4.7 4.51.57 Use APA referencing 4.5 4.51.58 Use discussing skills (participating) 4.8 4.51.59 Use discussion skills (leading) 4.8 4.41.60 Use presentation skills to audience 4.9 4.7

Note that on the key skills content the average ratings by DoE are ALL less than the average ratings of the student teachers.

DTVE Survey Page 45

The overall relevance rating of each module by DoE lecturers and TVE student-teachers are displayed in table 9.

Table 9: Average overall relevance ratings of each module and the integrated key skills (KS) by DoE lecturers and DTVE student-teachers.

DTVE graduates

DoE lecturers

DN01 Assess Learners´ Needs 4.6 4.7DP01 Plan for effective Learning 4.8 5DP02 Prepare Resources to support Learning 4.8 4.9DF01 Facilitate Learning 4.8 4.9DS01 Provide Learners with Support 4.7 4.6DA01 Assess learning 4.7 5DA02 Evaluate learning Sessions 4.6 4.8DE01 Reflect upon Roles of a Teacher 4.4 4.7DE02 Improve own professional Practice 4.7 4.7DE03 Undertake Action Research 4.3 4.9Key Skills 4.8 4.3

The two differences, although not significant, to note are on DE03 and KS. Student-teacher are not strongly convinced about the relevance (and use) of module DE03, the action research module, while lecturers consider it highly relevant. For the integrated key skills it is the other way around. Student teachers find key skill very relevant and mentioned (see section 5.2.1) that more attention and support should be given to key skills.

6.3.2 DTVE programme structure

Identical questions in the DoE lecturers and the DTVE graduates questionnaires allow to compare ratings on programme structures. The ratings and the items are tabulated in table 10.

DTVE Survey Page 46

Table 10: Comparing ratings of related to programme structure by DoE lecturers and DTVE graduates

Item

DTVE graduates

DoE lecturers

Enjoy programme 3.9 4.0Programme well planned 3.2 3.5Good sequencing of Units 3.3 4.0Appropriate Content 3.4 4.4Time for Completion adequate 2.6 2.0Workload acceptable 2.2 2.3High Quality programme 4.2 4.3Well prepared for TP 4.1 4.1Integration of TP and college work 4.1 4.4Programme is flexible 3.5 3.1

On most items DoE lecturers give a slightly higher rating. The problem of too heavy workload and the insufficient time given to complete the programme is shared between lecturers and student-teachers.

6.3.3 DTVE portfolio assessment system

Both DoE lecturers and DTVE student-teachers rated items related to the portfolio assessment. They are tabulated in table 10 with the average agreement rates. Diagram 20 illustrates the data in the table in a double bar chart. The difference in rating are not significant at 95% confidence level as checked by using a t-test (Annex H).

Table 11 Comparing of average ratings on items related to portfolio assessment by DoE lecturers and DTVE graduates

Portfolio assessment related itemsDTVE graduates

DoE lecturers

Most appropriate for OBE 4.3 4.6Ensures evidence for all LOs 4.4 4.5Encourages self directed learning 4.5 4.8Demonstrates progress towards identified LOs 4.3 4.3Encourages reflective practice 4.3 4.1Provides opportunities to demonstrate competence 4.3 4.1High validity 4.3 4.4Fair way to document progress 4.5 4.4Motivating 4.4 4.3Reliable method. Lecture independent 3.8 3.5Ensures authenticity of evidence 3.7 3.9Transparent method of assessment 4.1 4.1Encourages critical thinking 4.4 3.3NOT more time consuming than other modes of assessment 3.2 2.5

DTVE Survey Page 47

DoE lecturers and DTVE student-teachers share the perception that using portfolio assessment is more time consuming than other forms of assessment. They share the view that portfolio assessment is most appropriate for the outcome based DTVE programme, ensures all learning outcomes are covered, gives responsibility to learner for own learning, enhances reflective practice and critical thinking, is motivating, transparent and effective to monitor progress.

6.3.4 DTVE programme support system

In the student-teacher questionnaire a section was focusing on the support system of the DTVE programme. In the DoE lecturer questionnaire only two items (2.17 & 2.18) referred to the support system. The average agreement score on these items is low, 3.0 and 2.7 respectively. Both the support system during the running of the DTVE programme and the support system to support student-teachers that failed (and are moved to the distance learning mode) the programme are perceived as inefficient. The student-teachers share this opinion as outlined in section 6.2.6.

The DTVE programme structure, its content, portfolio assessment system and support system are perceived in the same way by DoE lecturers and student teachers. Any differences in average rating of corresponding items in the two questionnaires are not significant. Strengths and weaknesses are shared ground and hence form a solid basis for implementation of any change in the programme that might be needed. There is no disagreement between DoE lecturers and their former students as to the strong and weak aspect of the DTVE programme.

6.4 Structured focus group interview: management and supervisors of DTVE graduates

Structured focus group interviews were conducted with management/supervisors of the DTVE graduates with in 6 TCs and 13 groups in the Brigades. A total of 59 respondents were involved 27 in the TCs and 32 in the brigades. The details are in table 12.

Table 12 Colleges and brigades visited and number of respondents in the focus group interviews with management / supervisors of the DTVE graduates

Institution: College / Brigade

Management / supervisors

ATTC 4GTC 5JTeC 4MTC 7

PaTeCo 2SPTeCo 5

Total TCs 27Barolong VTC 2

KRDA 4Marapong 3Marobela 2

DTVE Survey Page 48

Naledi 6Ngethu 3

Okavango 2Ramatea 1Shashe 1

Tlokweng 1Tswelelopele 2

Tutume 5Total Brigades 32

The purpose of these structured focus group interviews (Annex E) with management / supervisors of the DTVE graduates was to find out: How do managers / supervisors of DTVE graduates rate the quality of their facilitation as compared to facilitators without this teacher training background? The summary of the responses of the 19 structured focus group interviews with management/supervisors of former DTVE students is found in Annex H. The most frequent mentioned strengths of the DTVE graduates were:

Competent and confident in learner centred methods Effective use of teaching/learning aids Effective use of a variety of teaching/learning methods Effective planning and preparation of lessons Preparing effective schemes of work Supporting management/Co operative Supporting learners in achieving the learning outcomes Demonstrating professional behaviour/ Good work relationship with learners Contribute ideas and take initiative in the institution Professional interaction with learners

The responses are based on classroom / workshop observations by management / supervisor taking place at least once every term and the level of participation in staff meetings observed. Management / supervisors in brigades reported a stronger impact of the DTVE graduates on the institution than management / supervisors in the TCs. DTVE graduates in the brigades on completion of the DTVE programme were given more responsibilities, promoted and/or given positions in the management. In 10 of the 13 interviews with brigade management / supervisors it was mentioned that the DTVE graduates had been running workshops for the instructors in the institution varying in length from 1 day to 2 weeks (during term break). This dissemination of knowledge and skills within the institution was NOT mentioned in any of the 6 TCs visited.

In discussion weaknesses and suggestions for improvement of the DTVE programme management and supervisors mentioned issues not directly related to the quality of facilitation of the DTVE graduates. These will be covered in the next section under concerns and issues raised

DTVE Survey Page 49

6.5 Structured focus group interview: learners taught by DTVE graduates

A structured focus group interview (Annex E) was carried out with learners currently taught by the DTVE graduates working in the TCs or Brigades. The procedure followed is described in section 5.6. Data were collected through interview sessions with 91 learners in 6 TCs and 104 learners in 13 Brigades. The details are in table 13.

Table 13 Colleges and brigades visited and number of learners participating in the focus group interview

Institution: College / Brigade

Learners

ATTC 10GTC 14JTeC 14MTC 17

PaTeCo 15SPTeCo 21

Total TC’s 91Barolong VTC 15

Ghanzi 6KRDA 8

Marapong 8Marobela 4

Naledi 5Ngethu 6

Okavango 8Ramatea 5Shashe 8

Tlokweng 6Tswelelopele 17

Tutume 8Total Brigades 104

The purpose of the interviews was to answer the research question: How do learners of graduates rate the quality of the facilitation by DTVE graduates? Learners took the opportunity provided to raise various issues related to their education and their educational environment. Some of these issues, reported in the section “Concerns raised”, did not immediately relate to the focus question: What is the quality of the facilitation of the DTVE graduates? A report was produced for each of the 19 focus interviews and these were summarised (Annex J). The data relating to the perceived quality of facilitation by the DTVE graduates by the learners is summarised here.

DTVE Survey Page 50

At the surface level the facilitation of DTVE graduates does not differ much from the facilitation of lecturers / instructors without DTVE. All lecturers in TCs and brigades facilitate, assess and support their learners. However by looking more carefully the learners mentioned differences. Learners in the brigades pointed out two areas in which the DTVE graduates differ from the non DTVE trained lecturers / instructors

Differences in practice. DTVE graduateso Use more variety in teaching methods. They ensure the teaching method used is

aligned to the learning outcome to be coveredo Predominantly use learner centred approaches. Cooperative learning methods,

especially small group work, are frequently used. Non teacher trained lectures / instructors tend to use teacher centred lecture methods.

o Plan each session carefullyo Scheme the whole course. In contrast non teacher trained lecturers / instructors are

poor in scheming or do not scheme. This leads to situations that work is unequally divided over the term, with extreme pressure towards the end or even having to extend the official set length of the course.

o Demonstrate a professional approach to their work Provide prompt and constructive feedback Set very regular formative tests Treat all learner equal

Differences in attitude. DTVE graduateso Demonstrate confidence. “They seem to know what they are doing”o Are less authoritative / more flexible in taking learners’ suggestion on boardo Are very supportive in assisting learners to achieveo Are approachable in and outside sessions

DTVE graduates in the brigades were singled out as good role models for other lecturers / instructors in the great majority of cases. The differences mentioned by learners are too a large extend similar to those reported in the previous section as the strengths observed by management / supervisors.

Learners in the TCs saw little or no difference in the practice of the DTVE graduates and their other lecturers. This is understandable as most staff in the TCs are experienced and a number of them have some teacher training background. In the brigades qualifications of instructors is lower and majority has not gone through any form of teacher training. The learners in the TCs did mention noticeable difference in attitude, similar to the once mentioned by learners from the brigades.

Learners in both brigades and TCs, although emphasising the positive aspects of their DTVE trained facilitators, did express that in their view ALL lecturers / instructors in their institution, including the DTVE graduates, did not approach them as adult learners. Learners mentioned numerous situations in which they felt that they were treated without sufficient respect, felt belittled, insulted. “They all treat us like kids”

6.6 Completing the DTVE programme

This section presents data to answer the research questions

DTVE Survey Page 51

a. What is the completion rate / pass rate?b. What are reasons for dropping out from the programme before completion?c. Where are graduates employed after completion?d. What is the time gap between completion and employment?

Data to calculate withdrawal, completion, pass and fail rates of student-teacher in the DTVE programme were collected from the DoE. Different documents give slightly different data for the 174 student teacher that went through (part of) the programme. Table 14 summarises the data as available to the TA tracer group and mainly based on the compiled student list (Annex K).

Table 14 Data on withdrawal, supplementation, pass and fail DTVE cohorts 1 to 4.

Cohort Pre service

In service

Withdrawn from course

Supplement ing

Passed Failed

C1 12 3 0 5 (33%) 9 (60%) 6(40%)C2 27 18 9 (20%) 4 (11%) 32 (71%) 4 (9%)

C3 FT 28 23 3 (6%) 48 (100%) 43 (84%) 5 (10%)C3

Block- 15 1 (7%) 14 (100%) 5 (33%) 9 (60%)

C4 33 17 10 (20%) 40 (100%) 33 (66%) 7 (14%)

Withdrawal rates for cohort 2 and 4 are 20% i.e. 1 out of each 5 student teacher that started the programme withdrew from the programme before completing. Reasons for withdrawing from the course or not completing were given by 15 respondents and can be categorized as in the table 15.

Table 15: Reasons for withdrawing from DTVE programme

Category Reason as stated in responseMedical (13%) Due to illness

Medical reasonsTaking up employment opportunity (20%)

I was appointed at Department of Supplies I found an employment opportunity and so I went for it Offered employment

Course related reasons (33%)

The programme was taking longer than the expected 12 months and was caught up in a financial crisis

Had not finished all the modules when the time elapsed(f=3)

There were many reasons, e.g. Files not being marked in time, Teaching Placement wrongly placed and facilitators personalizing things.

Personal reasons (33%) I had personal / family / business issues to take care of(f=5)

Supplementation rate All student-teachers in cohort 3 and 4 had to supplement i.e. nobody passed the programme on 1st submission of the portfolios. No data are available to explain the 100%

DTVE Survey Page 52

supplementation rate. It is clearly related to the way the programme is structured i.e. the workload and time pressure mentioned by both DTVE student-teachers and DoE lecturers.

Pass rates are calculated as a percent of those starting the course. Pass rate for the block release group is very low, only 1 out of every three student teachers passed in this mode. If one calculates the pass rate as a percent of those completing the course (leaving out the withdrawals) the percent pass / fail are as in table 16.

Table 16. Pass / fail rates as percent of completers

Cohort Passed FailedC1 9 (60%) 6(40%)C2 32 (89%) 4 (11%)

C3 FT 43 (90%) 5 (10%)C3

Block5 (36%) 9 (64%)

C4 33 (82.5%) 7 (17.5%)

Employment data for DTVE former student teachers are incomplete – data on the 22 non traceable previous students are missing. Assuming that the 22 not traced students are employed (if in the TVET system they would have been traced) then 8% of the former DTVE students are unemployed (all except 1, from the last cohort finishing in July 2008). Employed in the TVET sector (TCs or brigades) are 67% of the 174 DTVE student-teachers. Hence about 25% is employed outside the TVET system (some will be self employed or studying).

17 (16%) of the respondents did not find employment immediately after completion. Reported waiting times vary from a few months to a year.

Reasons stated for not being immediately employed are: Still waiting for DTVET to post me “They said I am a foreigner so I can only be employed when DTVET advertises, then I can

apply just like any other foreigner” Have not completed all modules (f=2) I completed my studies in 2006 and had to wait for DTVET to employ me. I got employment

in February 2007 with the Department. I don’t know why I had to wait so long to get employment looking at the fact that I was a qualified lecturer

As a pre-service trainee I was not automatically placed by DTVET I applied at various Institutions and did not get an offer(f=2) It took time for the Ministry/Department to allocate me to a college(f=5) I had to wait for 6 months reason being that DVET was still looking for posts No idea why DTVET did not employ me Took 3 months to find Temporary Full Time employment in a TC

It is made clear to student-teachers entering the programme as pre service candidates that employment on completion is not guaranteed. Yet with the shortage of trained vocational teachers,

DTVE Survey Page 53

one would expect that all successful graduates from the DTVE programme could be immediately absorbed into the TCs. It needs further investigation to find out why successful graduates from the DTVE programme are not immediately employed.

6.7 Concerns raised.

Inevitably, there were issues raised during the data collection that were outside the scope of the study. Listed here are those thought to be important for a future review of the programme, or a wider review of vocational teacher training.

DTVE graduatesRecognition of DTVE1. Graduates working in the TCs expressed disappointment that DTVET does not recognise the DTVE as worthy of a salary increment. They are at same scale point as those having a vocational diploma but with no teacher training qualification. This could act as a deterrent to some, since there is no financial incentive to attend or achieve.2. Comments were made regarding some college managers, not fully appreciating the value of DTVE and therefore reluctant to permit staff to enrol for the course. 3. Articulation routes are apparently not well defined. The lack of an internationally recognised qualification caused difficulty for some students attempting to enrol on Bed programmes abroad. This could be addressed by seeking formal arrangements through memorandums of Understanding between FCTVE and other institutionsBTEP training1. DTVE graduates are required to go through BTEP phase 1 / 2 /3 training when placed in colleges. The content of these training workshops is covered by the DTVE programme and was an unnecessary duplication. The DTVE graduates complained that they often knew more about BTEP, than those delivering the phase training!

DTVE facilitators at DoE1. It was mentioned that there has been ‘an inflow of secondary school teachers’ as facilitators of the outcome based DTVE programme. Graduates felt that the background of these lecturers made them unsuitable to deliver the outcome based DTVE programme without themselves first being thoroughly trained on OBE in general and the DTVE programme in particular and familiarising themselves with vocational training (work attachment in industry was suggested and/or attachment for at least one term as lecturer in a vocational area in a TC i.e. facilitating BTEP.

Staff development1. The graduates expressed concern that there was no programme for continuous professional

development upon completion of the DTVE. There would be a demand for: training in project supervision portfolio building (several BTEP units require learners to build portfolios) the use of technology (e-learning and eAssessment).

Gap between competencies achieved and what can be implemented

DTVE Survey Page 54

1. The lack of resources in some of the TCs, and in most of the Brigades was mentioned by graduates as a major problem. The lack of data projectors, and network connectivity in the classroom prevented the use of PowerPoint presentations and internet resources in the classroom.

Follow up of ‘non-completers’1. There are a number of former DTVE students that did not complete all the modules. They reported that they did not receive clear information as to how they could continue beyond the normal end of programme, to complete them. Some reported that attempting to clarify this with DoE had not been successful.

Feedback on outcomesMany respondents made the comment that after having giving their time and input to the tracer study they will never receive any feedback or information on the outcomes.

The Teaching Placement component of the DTVE1. It appears that students on TP are reluctant or unwilling to take responsibility for a group of learners on a timetabled basis. They want access to a class group only while being observed for external assessment.2. The consequence of the above: TCs are reluctant to employ DTVE graduates as they have not really seen the graduates ‘performing’ within the college.2. TP supervisors suggested student-teachers to (i) follow experienced teacher for about 1 week (ii) take full responsibility for a group of learners for the next weeks. This allows feedback from supervisors.3. The TCs and brigades are not sufficiently informed beforehand by DoE about the upcoming TP. When DoE lecturers are at the TCs and Brigades they don’t consult or interact with the lecturers/instructors of that institution. 4. External supervision (DoE at FCTVE) could be greatly reduced by training the workplace based mentors and, after training, allowing these to assess student on TP. Currently TP supervisors / mentors feel sidelined: their views are not considered nor asked for.

Communication DoE with TCs and brigades1. Management at both TCs and brigades expressed that “we do not know much about the content of the DTVE programme”. A need was expressed for workshops/ briefing meetings by DoE for management / HoDs on the programme.

Staff development (SD)1. Management at both TCs and at brigades identified the need for continuous staff development to upgrade staff and management. The question was raised in some of the brigades “What happened to the BRIDEC short training courses?” These were felt to have been very useful for instructors to improve their teaching skills. As these courses were offered (phase 1 -3) during term breaks they fitted well in the programmes of the Brigades. Similar short courses were suggested for staff development.2. SD activities related to using of modern technology in teaching, understanding blended/flexible learning approaches, setting formative assessments, classroom management skills – were mentioned as some areas a staff development unit in the DoE should provide.

DTVE Survey Page 55

3. Teachers should also be upgraded in vocational areas. Currently some of the graduates cannot teach all units in their vocational area as they lack the content knowledge for certain units. During the teacher training attention should be paid to the vocational content knowledge to ensure the graduate feels confident with all units at the foundation and certificate BTE programme level.

Learners taught by DTVE graduatesLearners are concerned about

a. the BTEP1. The assessment mode is not challenging, same assessment instrument(s) used repeatedly.2. The long time it takes to receive the BTEP certificate after completion. Learners in TCs stated that learners– since the start of the programme – had not received a Certificate (going back to 2004 / 2005).3. Employers do not value the BTEP as it shows A only (achieved, misinterpreted as a grade A).4. Learners prefer a system of assessment/reporting that does distinguish between learner levels of achievement, as they feel this could enhance employment opportunities.5. Lack of understanding of the BTEP by senior college management.6. Lack of understanding of BTEP by employers – who prefer the graded NCC.7. Poor planning of some of the programmes – a programme can be extended by some months because some LOs are not covered or they ‘forgot’ to offer an elective – leading to extension of the programme.8. Key skill departments in TCs received much criticism from learners. They felt the departments are poorly organised, lecturers authoritative and lacking knowledge. It was suggested by some learners that the lecturers in their vocational department ‘could do a better job’ and cover the key skills as integral part of other units.

b. Access to resources1. Opening hours, access to computers and library was felt to be insufficient.2. More computers and fast internet access was felt to be needed in connecting with project report research and writing3. Access to computers frequently only for learners in ICT programmes, access for other vocational areas was considered to be insufficient in relation to work they had to do on the computers (e.g. project work and reports)

c. Resources1. Brigade learners mentioned lack of resources: library books, computer (if present – outdated hard and software), data projector2. In some vocational areas it was said that the number of teaching staff was insufficient resulting in the few staff available being overloaded and having insufficient time to support learners individually.

d. Support1. Non ICT related vocational areas felt that they were not supported enough in ICT during the very limited time (at times 1 or 2 hours a week) they had access to computers. Nobody was around to give them support and they felt belittled by ICT lecturers who hardly tolerated them in “their” lab.

DTVE Survey Page 56

e. The learners´ views1. Learners in TCs and brigades are concerned about the weak vocational knowledge of many lecturers / instructors. It was stated that it is not uncommon that instructors / lecturers with an NCC are teaching NCC. Learners expressed that there is a need for vocational upgrading of lecturers / instructors.2. Due to lack of vocational knowledge instructors/lecturers tend to copy from books but fail to explain what they copied.3. Learners in TCs observed that the content presented frequently lacked relationship to the real work situations. In their opinion this was due to the lack of industrial experience of the great majority of lecturers. They, the lecturers, have a theory background but lack experience in industry.4. Learners in TCs observed that in their opinion there was no difference in approach used by lecturers when delivery foundation, certificate or advanced certificate BTEP. In their view the different levels need different approaches and ways of interaction with the learners.

DTVE Survey Page 57

7 Conclusions and Recommendations

The recommendations are based upon the combination of the data collected from the all the target groups:

7.1 Curriculum

Both DoE lecturers and graduates are satisfied with the curriculum content of the modules. The DoE lecturers are marginally more satisfied with the module content. Conversely, the graduates´ satisfaction rating of the integrated Key Skills was higher than the lecturers.

Some comments from the graduates, the DoE lecturers and management supervisors suggest that they would have liked additional input on the following topics:

psychology adult learning theories the context of vocation education in Botswana educational policy and management integrating of HIV/AIDS related issues use of modern technology in facilitation and learning (e-learning)

The content of the DTVE has not changed significantly since 2001 during which time there have been major changes in the vocational sector, with more to come over the next few years. While there is a general satisfaction with the content of the DTVE, from the surveyed stakeholders, a wider stakeholder review e.g. including TEC, BOTA and private colleges, could provide useful input to a review of the design and content of the programme. Equally, comparison with similar programmes from other countries, both regionally and further afield, could be considered.

Integrated Key Skills (KS)The DTVE programme was designed to include key skills at BTEP level 4. There are no stand alone modules; all of the desired competencies are included in the other individual modules. This is in line with current international trends ensuring that students develop and improve key skills throughout all levels study. The concern is however, that while the criteria for achievement of the key skill is embedded, how is the skill to be evidenced, how is it to be taught and by whom? The average rating of the relevance of key skills by DoE lecturers (4.3) - lower than the average rating (4.8) of the graduates. This is not statistically significant in itself. However when combined with comments from the graduates concerning the strengthening of key skills in the programme (see section 6.2), this may indicate that the teaching practices of DoE lecturers should be reviewed for effectiveness as far as the key skills component is concerned. Very few of the current DoE staff were involved in the original design of the programme, and some of the original aims and objectives may need to be re-emphasised, especially in view of the later recommendation concerning new staff induction.

DTVE Survey Page 58

Recommendation 1That DTVET initiate a review of the DTVE curriculum by external consultants and with the involvement of a wider set of stakeholders. This should include comparisons with similar programmes offered both regionally and internationally.

Further, very few of the current DoE lecturers have previous experience with an outcomes based programme and the integration of Key Skills.

Facilitation/FacilitatorsGenerally speaking the DTVE graduates are satisfied with the mode of facilitation used to deliver the DTVE. They are also positive about the general facilitation of the programme by the DoE lecturers who are considered professional in their approach and demonstrate good quality in their work. However, an observation made by some graduates indicated that when being taught by staff new to DoE, it was obvious that they had not been sufficiently well inducted to the programme facilitation and assessment procedures. This was confirmed by informal comments by DoE staff, who recently joined the department. It is good practice to ensure that new staff are properly inducted to both the institution and the specific department in which they will work. It is usually carried out by a group of experienced practitioners, each contributing a particular area of expertise to the induction programme. It should involve appropriate verifiable activities.

Some comments indicated that the real-world business and industry knowledge of some lecturers could be improved. As is the case for vocational lecturers, short industrial placement is an important and appropriate way to keep up-to-date with modern practices and trends and could be considered by DoE management and staff alike.

DTVE Survey Page 59

Recommendation 2It is recommended that FCTVE management initiate an internal review of the practices being adopted by staff to teach the embedded key skills components in the taught modules. The review should consider the effectiveness of the practices and consider ways to enhance and improve student opportunities that enable them to practice their key skill competencies to the required level. The review should further take into consideration the diverse background of students by acknowledging prior learning. Sharing of identified best practice could be included in new staff induction processes.

Recommendation 3That the FCTVE management initiate an internal review of the continuous professional development of DoE staff. This should include vocational placements and the induction processes for new teaching staff. The following should be considered:

Include an initial training needs analysis for new teaching staff in relation to the content, facilitation and assessment of the DTVE programme

Ensure that appropriate induction and continuous professional development activities are created to meet individual staff needs. The use of ICT could usefully be explored here e.g. online induction courses

Create opportunities e.g. short industrial placement, or attachment to vocational departments in TCs, for staff to become familiar with the vocational (BTEP) areas of their students in order to contextualise DTVE content to student teachers vocational areas

The Programme Structure/Portfolio AssessmentThere is general agreement that the structure and sequencing of the programme is effective. The portfolio assessment is considered appropriate, fair and valid. However, concerns were raised by both DoE teachers and the graduates.

The overall workload is too heavy for the time available Insufficient time available to develop and practice new skills Issues concerning plagiarism Assessment reliability Gathering evidence is time-consuming

A heavy workload can have the effect of reducing the quality of facilitation e.g. delays in providing feedback, the quality of feedback. To save time, students may be tempted to copy evidence generated by other students (plagiarism). One frequently suggested way to address this issue was to increase the number of staff in the DoE. This might address the workload of the DoE staff, but is not guaranteed to resolve the issue for the students. It might be expected that the curriculum review recommended earlier would consider this issue in some depth; however, there may be some useful efficiencies that could be considered by the department.

integrating related content from different modules so that it is taught once and the same evidence used across different module portfolios

investigate other possibilities for cross referencing of evidence across module portfolios to reduce the amount of evidence which needs to be gathered

review timetabled staff allocations e.g. a single lecturer teaching the same module to all student groups in a single cohort. This would reduce preparation time and reduce the time required for internal verification (since the overall number of portfolios to be internally verified would be reduced). It could also be expected to improve the reliability of assessment.

use more flexible / blended learning approaches to reduce face-to-face contact time Consider stricter use of deadlines and the use of student progress files introduce Accreditation of Prior Learning (APL) to recognise the prior experience and

knowledge of students, many of whom already have extensive teaching experience. This could lead to exemption from certain modules and/or learning outcomes.

investigate how modern technology can be used to make more effective and efficient use of the time available for the programme e.g. electronic submission and tracking of evidence

The issue of plagiarism is a growing concern internationally. Student use of ICT makes the copying and sourcing of others work, much easier than before. It is obviously a concern that individual students obtain the diploma by their own efforts and study and not through some form of academic deceit. However, the issue is arguably more serious from an institutional point of view. Educational institutions have a duty of due diligence to ensure that the status or perceived value of its qualifications remains high. A qualification which may be obtained simply by purchasing a portfolio from a previous student, is of little value. Any institution which ignores the trends is failing in its duty.

DTVE Survey Page 60

This is an urgent issue which needs to be given a high priority by the DoE. Effective approaches could be:

Consider the use of computer software, or subscription to online plagiarism detection services

Regularly change the nature or form of evidence to be gathered across student cohorts and groups

Student induction activities to emphasise the nature of plagiarism as academic fraud and clearly outline the measures taken by the department to combat it, and the penalties attached

FCTVE Plagarism Policy made available to students during induction

Learning support to studentsAn important component of an educational training programme is the learning support system available to the learners. This becomes even more important, for the new flexible and blended learning approaches expected to be developed at FCTVE using e-learning and distance learning. The support available and given to student teachers during the DTVE programme is rated by student-teachers as adequate in meeting their needs. However, both DoE lecturers and DTVE graduates feel that it is inefficient. As might be expected, amongst those who needed support beyond the normal end of the programme, the satisfaction rating of the support system is very low (see section 6.2).

A good departmental learner support system will generally include:

An extensive programme of induction Assigning a named support/programme tutor First line pastoral care Appeals and other support for student advocacy Regular measure of student satisfaction across a range of indicators Regular in-course progress guidance using progress files Exit guidance at normal end of programme Post-course guidance when required

DTVE Survey Page 61

Recommendation 4While awaiting the outcome of Recommendation 1, the DoE should internally review its assessment practices, timetabling and the use of modern technology to seek efficiencies which reduce the work load of staff and students.

Recommendation 5That FCTVE management initiate an investigation to determine the extent of plagiarism on the DTVE programme and to propose effective methods and procedures which may be applied to combat it.

Published post-course reviews and performance indicators

The following table is copied from section 6.6

Cohort Pre service

In service

Withdrawn from course

Supplement ing

Passed Failed

C1 12 3 0 5 (33%) 9 (60%) 6(40%)C2 27 18 9 (20%) 4 (11%) 32 (71%) 4 (9%)

C3 FT 28 23 3 (6%) 48 (100%) 43 (84%) 5 (10%)C3

Block- 15 1 (7%) 14 (100%) 5 (33%) 9 (60%)

C4 33 17 10 (20%) 40 (100%) 33 (66%) 7 (14%)

Learning support is now internationally recognised to be a holistic strategy aimed at ensuring a quality experience for learners in support of their optimal achievement. The cost to government of providing financial support to learners, maintaining and staffing institutions is high. Retention and pass rates are an important key performance indicator which helps to assess the cost effectiveness of educational policy and government spending. Institutions need some means by which they can be assessed for cost-effectiveness and performance against national benchmarks. Quality frameworks and measures of performance are readily available for comparison from other countries and equivalent teacher-training programmes.

Teaching Placement (TP)TP is an integrated part of the DTVE programme. The students go for 3 months on TP during their study. In-service student teachers return to their institutions for TP while for the pre service student teachers a place is found in a TC or a brigade that offers programmes in the vocational area of specialization of the DTVE student. During TP the student-teachers collect evidence for various portfolios. Some of the evidence is obtained through observation – DoE lecturers use observation checklists during classroom sessions.

Some concerns about TP were raised during interviews with the management/supervisors group and relate mainly to the communication, coordination and effectiveness of TP. Mentioned were:

No regular consultation or meetings with institutions to coordinate TP, leading sometimes to inconvenience and inappropriate timing.

Student teachers concentrate on evidence collection and consider the placement period too short to effectively practice their facilitation skills

DTVE Survey Page 62

Recommendation 6It is recommended that DoE initiate a review of the DTVE learner support system currently in place. Best practice from similar regional and international institutions should be consulted. It would also be appropriate to maintain records of student retention and pass rates for current and future student cohorts.

Student teachers should fully participate in the institution during TP, taking over the full responsibilities of a lecturer / instructor under close supervision of the mentor

Limited or no resources for student teachers to use, but needed for evidence collection e.g. use of data projector, OHT etc.

With training, institution based mentors could be used for some observational assessment activities thus reducing the workload of the DoE lecturers and reducing costs overall.

With the relocation of the DoE to FCTVE new opportunities for student-teachers to collect evidence at the college itself e.g. establishing learners´ needs, giving academic support to learning, running a PowerPoint session with a group of on-campus learners could all be done on campus. Each student teacher could be assigned to a lecturer in the vocational Departments. Opportunities for the collection of portfolio evidence could be identified e.g. teaching single topics to groups of learners. This would create time for ‘full-time facilitation’ during TP at other institutions.

DTVE Survey Page 63

Recommendation 7It is recommended that the planning, coordination and arrangements for TP are reviewed, taking into consideration that the DTVE is now delivered within a technical college with a wide range of vocational departments. This could permit regular interaction with the vocational departments, its lecturers and learners. The review should consider the feasibility of:

Involving experienced instructors and lecturers, after training in the assessment of the student-teachers

Gradually giving a full observation/teaching load to the student teacher during TP under the supervision and guidance of the institutional based mentor.

8 Literature Reference

8.1 Exit Survey – DTVE Diploma in Technical and Vocational Education Programme

The survey was carried out in January 2008 by the TA Team with a group of 8 DTVE block release students shortly before their completion of the programme. Block release DTVE programme mode implied that the student-teachers would be full time on campus for the first two trimesters (July – December 2005) and after that alternate between working as instructors in their Brigade for one trimester (started first trimester 2006) and being full time student-teachers on campus for the following semester. As the DTVE programme is a 18 month programme for full time student-teacher, this made the programme for the block release group to last 3 years. The purpose of the survey was to gain understanding of the learning experiences of the participants in the block-release DTVE programme

The main findings were The content of the programme was seen as relevant and useful (applicable) to the work as

an instructor in a brigade. The portfolio assessment mode was seen as an appropriate way to assess the learning

outcomes Great dissatisfaction with the quality of the facilitators of the programme Support perceived to be of good standard during the initial block deteriorated later in the

course (progress grids, personal tutor system cancelled; great delays – up to two years – in feedback)

Lack of communication between DoE and brigade management on the nature, content and mode assessment of the DTVE programme

For the full report see Annex L

The findings of the exit survey are in line with and support the following findings in this report The content of the DTVE programme is relevant and applicable in the work situation Portfolio assessment is an appropriate assessment mode for the DTVE programme Weaknesses in the portfolio assessment system (time consuming, authenticity, reliability of

assessment, turn over time and nature of feedback) need to be addressed The need for strengthening the underpinning theories and philosophies . The support available and received from DoE lecturers is of good quality. Some weaknesses in the support system need to be addressed e.g. support to student-

teachers placed in the distance learning mode

8.2 BOTA survey

Bota carried out a tracer study for vocational training graduates (phase 1) in 2005. The main aim of the survey out this was to track down vocational training graduates in order to identify and address mismatches between former learner’s jobs and the training received.

DTVE Survey Page 64

The main conclusion from the tracer study is that the vocational training system does not produce readily employable people mainly because of the absence of practical experience in the training programme.

This conclusion is identical to concerns expressed by learners, DTVE graduates and management. This relates to confusion over the purpose of the Foundation and Certificate programmes: are these programme an exit or bridging qualification. With the Government Grant Loan Scheme now applicable to learners in the TVET system for certificate – advanced certificate and diploma level, the three perceived as an entity ,it can be deduced that exit into industry / self employment should be at diploma level. However with no BTEP advanced certificate and/or diploma holders produced as yet it is still to be established whether BTEP effectively prepares learners for the world of work i.e. whether or not they are (self) employable. T he delays in offering the BTEP Advanced Certificate and Diploma levels contributed to negative perceptions of BTEP among learners, management and DTVET graduates. The stronger work experience component, which is a feature of both the Advanced Certificate and DiplomaBTEPs, will also help to combat the negative image of the BTEPs.

8.3 Consultancy to Conduct a Tracer Study of Graduates of the Botswana Technical Education Programme, 2007

The overall objective of this tracer study on BTEP graduates (2002 to 2006) was to find out about the graduates1. employability in terms of course relevance and quality for the world-of-work; 2. employment or self employment.

Some of the findings were lack of BTEP awareness by employers who may be prefer to employ an NCC or other better known

qualification. There is need to re-consider the programme structure of BTEP, especially with regard to the balance

between theory and practical assignments as both employers and graduates identified this as an area of weakness

Respondents also identified problems with the quality and education levels of instructors. A serious concern for BTEP is that half of the total graduate respondents are still unemployed or

economically inactive.

These findings are identical to the concerns mentioned in section 6.7

DTVE Survey Page 65