University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The...

25
University of the Southern Caribbean School of Education & Humanities Department of Graduate Education Studies Maracas Royal Road, St. Joseph, Trinidad. WI Faculty’s Usage of Technology-Based Assessment: An Analysis of Perceived Barriers in Three Universities. A Research Project Paper Presented in Partial Fulfillment Of the Requirement for the Degree Master of Arts in Educational Psychology By Xxxxxxxxxxxx Xxxxxxxxxxx June 2014

Transcript of University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The...

Page 1: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

University of the Southern Caribbean

School of Education & Humanities

Department of Graduate Education Studies

Maracas Royal Road, St. Joseph, Trinidad. WI

Faculty’s Usage of Technology-Based Assessment: An Analysis of Perceived

Barriers in Three Universities.

A Research Project Paper

Presented in Partial Fulfillment

Of the Requirement for the Degree

Master of Arts in Educational Psychology

By

Xxxxxxxxxxxx Xxxxxxxxxxx

June 2014

Page 2: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION i

University of the Southern Caribbean

School of Education & Humanities

Department of Graduate Education Studies

Maracas Royal Road, St. Joseph, Trinidad. WI

The undersigned certify that they have read, and recommend to the School of Education &

Human Sciences for acceptance, a research project titled, faculty’s usage of technology-

based assessment: an analysis of perceived barriers in three universities, by Xxxxxxxxxx

Xxxxxxxxxx, in partial fulfilment of the requirement for the degree MASTER OF ARTS IN

EDUCATIONAL PSYCHOLOGY

………………………………………. …………………………………….

Chair: Vishranti Eton, Ph.D Member: Franklyn N. Baldeo, Ph.D

……………………………………………………………

Dean: School of Education & Human Sciences, (Ag.)

Joseph Ragoonanan, MA

……………………………………

Approval Date

Page 3: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION ii

Table of Contents

Page

Abstract ..................................................................................................................................... ii

Dedication ................................................................................................................................ iii

Acknowledgements ................................................................................................................... v

List of Tables ........................................................................................................................... vi

List of Figures ......................................................................................................................... vii

List of Abbreviations ............................................................................................................. viii

Chapter 1 ……………………………………………………………………………………...1

Background to the Study ........................................................................................................... 1

Statement of the problem ....................................................................................................... 11

Purpose/objectives ........................................................................................................ 12

Significance of the study ............................................................................................... 13

Definition of terms ........................................................................................................ 13

Information and communication technology (ICT) ……………………………………7

Technology-based assessment………………………………………………………….7

Understanding technology-based assessment ............................................................... 14

Twenty-first century skills ............................................................................................ 14

Ways of thinking. .............................................................................................. 14

Ways of working. ............................................................................................. 14

Tools for working ............................................................................................. 14

Skills for living in the world ............................................................................. 15

Organization of the study .............................................................................................. 15

Page 4: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION iii

References …………………………………………………………………………………...50

Appendix ................................................................................................................................ 56

A. IRB Approval .................................................................................................................... 56

B. ICT Usage Survey ............................................................................................................. 57

C. Letter of Introduction ........................................................................................................ 64

D. Acceptance Letter from one University ……………………………………………....... 65

E. Informed Consent ..................................................................................................................

F. Demographic and Personal Characteristics of Sample ..........................................................

G. Proficiency Level for Technology Hardware/Software Tools ..............................................

H. Perceptions of TBA ...............................................................................................................

Page 5: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION iv

Abstract

This study was conducted to determine university faculty’s usage of technology-based

assessment (TBA) and to explore their perceptions of barriers to successful integration of

TBA. An online survey was disseminated to lecturers in the Faculty of Education in three

major universities in Trinidad. Most participants spent more than 4 hours daily on the

computer, were highly computer literate and had very little professional training in the use of

TBA. Results revealed positive relationships between TBA usage and lecturers’ daily use of

computers, their proficiency with ICT applications, and their perceptions of TBA. Analysis

showed no significant relationship between usage and perceived barriers, but identified the

four top barriers as time to prepare assessments, lack of training, poor technical

infrastructure, and access to ICT hardware and software. Recommendations include more

targeted and subject-compatible professional development to increase competence and

confidence in using TBA, and more efficient technical support when ICT problems occur.

Key words: technology-based assessment, higher education faculty, technology integration,

barriers, perceptions, ICT.

Page 6: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION v

Acknowledgements

I would like to express my gratitude to Dr. Eton and the School of Education faculty

for generously sharing your knowledge and guidance over the past two years.

Sincerest appreciation to Dr. Brathwaite, my research supervisor, for your insightful

feedback and support throughout my research journey and for demystifying SPSS!!

To Ms. Holder for your advice and mentorship in providing not only the “what,” but

the “how” of being a competent assessment practitioner – warmest gratitude.

To Dr. Baldeo – for your “ready ear” always and your upbeat encouragement right

down to the finish line – thank you.

To Dr. McGarrell – who re-awakened my love for writing and lit the research fire in

me. Your class and your gentle spirit were gifts to me and truly epitomized the integration of

faith and learning. Thank You!

To Dr. Elvin Gabriel – your faith in me and my abilities, and your unconditional

support in all my academic and professional pursuits, no matter how busy you are – have

been the wind beneath my wings for close to twenty years – sincerest gratitude!

To my dear friend Shalla – the instant connection we felt on the first day has

blossomed into “friend for life” status. I am forever grateful.

My deepest appreciation and love for my most ardent supporters - husband, Bunny,

and kids, Gi and Gin. You walked every step of this degree with me. Thank you, we did it!

And to God – thank you for being my Source always. Indeed, You do not call the

qualified, but You qualify the Called. Thank you for this call. It has been a life-changing

experience for which I am deeply grateful and I will strive to honour You by blooming

wherever I am planted.

Page 7: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION vi

List of Tables

Table Page

1. Percentage of TBA Usage per Semester………………….…………………………… 25

2. Perceived Barriers………………………………………………………….………..... 28

3. Difference in TBA Usage Based on Literacy Levels…………………………………... 30

4. Difference in Perceptions Based on Literacy Levels …………………………………. 32

5. Difference in Perceptions Based on Age Groups………………………………………34

6. Relationship Between Usage and Six Independent Variables………............................ 35

7. Multiple Regression: Model Summary……………………………................................ 36

8. Multiple Regression: Coefficients………………………………………………… ...... 37

9. Multiple Regression ANOVA………………………………..……………………… ... 37

Page 8: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION vii

List of Figures

Figure Page

1. Unified theory of acceptance and use of technology …………………………………..17

2. Mean usage of TBA with the increase of age ………………………............................ 29

3. Mean usage of TBA with years of experience...…………..………..….……………… 30

4. Mean usage of TBA with increased computer literacy………………...……………… 31

5. Mean increase in perceptions with increased computer literacy ..….…........................ 33

6. Mean increase in perceptions with increased age……..…………….…........................ 33

Page 9: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION viii

List of Acronyms

TBA – Technology-based assessment

ICT – Information and Communication Technologies

CRESST – Centre for Research, Evaluation, Standards and Student Testing

US – United States

T&T – Trinidad and Tobago

TVET – Technical and Vocational Education and Training

UTAUT – Unified Theory of Acceptance and Use of Technology

UWI – The University of the West Indies

USC – The University of the Southern Caribbean

UTT – The University of Trinidad & Tobago

ANOVA – Analysis of Variance

Ph.D – Doctor of Philosophy

M – Mean

S.D. – Standard Deviation

Page 10: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 9

Chapter 1

Background to the Study

“Technology in education is neither a novelty nor is it a fad. It is a part of the

modern world, and is becoming more and more ubiquitous in our lives every year” (Saba,

2009, p. 9). This unavoidable phenomenon in education, more specifically, in higher

education, served as motivation for this study.

The irreversible impact of technology on tertiary level education cannot be ignored as

most of today’s college students have spent their entire lives using computers, cell phones,

and other digital media, and have integrated technology into almost everything they do

(Oblinger, 2008; Prensky, 2007 as cited in An & Reigeluth, 2011). In their 2010 CRESST

report, Behrens, Mislevy, DiCerbo, and Levy, concur that “the world in which learning and

assessment must take place is rapidly changing” and add that “traditional assessment

methods are inadequate for evaluating learning in our digital world” (p. 6). Valdez (2005)

underscores the need for authentic assessment based on his observation that “we are living in

an age of information overload with the expectation that students will learn high-level skills

such as how to access, evaluate, analyze, and synthesize vast quantities of information while

teachers are evaluated by their ability to have students pass tests that often give no value to

these abilities” (p. 3).

Studies show that far too many students are bored, unmotivated, and uninvolved, that

is, disengaged from the academic and social aspects of school life (Appleton, Christenson, &

Furlong, 2008), partially because of irrelevant curriculum and assessment strategies

(Ralabate, 2011). Atkins, Bennett, Brown, Chopra, Dede, and Fishman, (2010) observe that

“outside school, students are free to pursue their own passions and at their own pace – the

Page 11: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 10

opportunities are limitless, borderless and instantaneous” … thus challenging the education

system to balance learning theory with modern technology to create for students, “relevant

and personalized learning experiences that “mirror their daily lives and the reality of their

futures” (p. x).

In addition to its effects on student engagement, research on technology-based

assessment (TBA) is increasingly concerned with the relevance of assessment to twenty-first

century skills and students’ preparedness for the world of work. The Partnership for 21st

Century Skills, a Washington-based group involving the US Department of Education and

founding members such as Apple Computer, Inc., and Cisco Systems, Inc., asserts that “high

stakes assessments alone do not generate evidence of the skill sets that the business and

education communities believe will ensure success in the 21st century” (p. 1). Likewise, the

Trinidad and Tobago’s (T&T) 2005 Draft Policy for Information and Communications

Technology (ICT) in Education acknowledges the role of ICT to “enhance human capacity,

dynamize the teaching-learning environment, and, …create an environment that encourages

creativity, critical thinking, and decision-making” (p. 3). Currently, however, while there

have been studies regarding technology integration in schools in Trinidad (Rampersad, 2011,

Ganpat et al., 2013), there is little data on the use of technology for assessment in T&T in

higher education.

TBA research is becoming more of an imperative as new technologies emerge and as

the competition among higher education institutions increases. Technology “makes it

possible to adopt new and arguably better approaches to instruction and/or change the content

or context of learning, instruction, and assessment” (Lawless & Pellegrino, 2007, p. 581).

Still, researchers caution that institutions should not rush into technology integration without

Page 12: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 11

first assessing the variables involved, such as the purposes of assessment and the contexts in

which these assessments will take place (van Merriénboer & van der Vleuten, 2012, p. 354,

Atkins et al., 2010, p. xi). Additionally, to derive the benefits of technology integration,

researchers must respond to teachers’ experiences, which are critical to the feedback,

planning and implementation stages of technology adoption efforts. Rotterham and

Willingham (2010), believe that we “waste a valuable resource when we don’t give teachers

time to share their expertise” and add that “for change to penetrate the classroom” (p. 19),

teachers’ voices must be acknowledged in meaningful ways – so that they can improve their

students’ learning experiences as well as their own professional practice.

Statement of the problem. The vast array of technological tools such as laptops,

iPods, iPads, and interactive whiteboards have altered the face of education and its delivery

in classrooms (Corrin, Lockyer, & Bennett, 2010; Gathercoal, 1999; Jamil & Shah, 2011)

prompting the need to change how we assess students in the classroom. Studies, (as cited in

Taylor & Parsons, 2011, p. 6) support this notion - “If we fail to change our pedagogy,

curriculum, and assessment strategies, we fail our students and jeopardize our own futures

(Willms, 2003; Robinson, 2009; Tapscott, 1998; Prensky, 2005; Gilbert, 2007).

In T&T, the 2005 Draft policy on ICT noted a “persistent mismatch between the

output of the education and training sector and the needs of an increasingly technologically-

oriented economy in terms of the relevance, quality, and quantity of skilled human

resources” (p. 13). The Taskforce agreed that technology integration is an essential

component in adapting to the needs of the “ever changing global environment” resulting in

the Ministry’s efforts to become more “student-centered and ICT driven” (pp. 3, 6). Still,

even with the proliferation of ICT in the Nation’s primary, secondary and tertiary-level

Page 13: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 12

classrooms, there is little information, at present, from the educators’ perspective regarding

the challenges of implementing technology-based assessment.

Twenty-first century skills, including innovation and critical thinking, are

progressively being emphasized as a necessity for success in higher education and in today’s

workplace. How do we know if we are producing this type of graduate? What are our

current assessment methods and are they measuring these skills? I concur with Schleicher’s

view that “you cannot improve what you cannot measure” (as cited by Molnar, 2010).

Purpose/objectives. The important role of teachers in classroom technology

implementation efforts has been documented in many studies (Georgina & Hosford, 2009;

Taiwo, 2009; Almekhlafi & Almeqdadi, 2010; Saavedra & Opfer, 2012; Khan, 2012) and

cannot be over-emphasized. Thus, this study sought to investigate the use of technology-

based assessment from the Faculty’s viewpoint and to explore their perceptions of

institutional, personal and professional barriers to successful integration of TBA. More

specifically, the study aimed at answering the following research questions:

1. To what extent Higher Education Faculty used technology-based assessment for

students?

2. What were the perceived barriers to using TBA?

3. Was there a relationship between participants’ demographic and personal

characteristics and their usage of TBA?

4. Were the perceptions of TBA affected by (a) computer literacy and (b) age?

5. Was there a relationship between usage of TBA and 1) computer literacy, 2) daily

use, 3) proficiency with hardware/software confidence, 4) perceptions of TBA 5)

confidence with applying TBA, and 6) perceived barriers?

Page 14: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 13

6. What were the factors that encouraged future use of TBA?

Significance of the study. In recognizing a need for a better-skilled and qualified

workforce, T&T has invested US $2 billion over the past decade in tertiary and technical and

vocational education (Policy on Tertiary Education, TVET, and Lifelong Learning in

Trinidad and Tobago, 2011, p. 13). With so much money being invested in technology

integration, there is very little understanding regarding the extent to, and manner in which

TBA is used in higher education in T&T. International research also shows that technology

is still under-utilized in assessment and feedback practices (Jisc, 2010, p. 8.) and that

institutions are not using “the full flexibility and power of technology to design, develop, and

validate new assessment materials and processes for both formative and summative uses”

(Atkins, et al., 2010, p. 25). As such, there is need to explore the challenges faced by those

who are already using TBA and the barriers that prevent inexperienced users from increasing

their use.

In identifying ways in which technology is used as an assessment tool at the tertiary

level, and in highlighting challenges to implementation, this study can assist teachers,

administrators, and policy makers to eliminate or reduce barriers to the adoption of TBA. It

can also inform the professional development gap which, according to Godzicki et al., (2013)

has been fraught with “serious problems, often being based on the “somewhat narrow view

that teachers need only technical skills and a good attitude” (p. 32). This is critical, given the

prediction that “assessment will continue to move towards technologically-supported

automation” (Redecker et al., 2011, p. 30).

Page 15: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 14

Definition of terms. The following are a list of terminologies and phrases that were

uses in this report of the study. It is intended to give the context in which these terms were

used.

Information and communication technology (ICT). Technologies used for the

handling and communication of information, specifically in education – including, but not

limited to, computers, audio visual systems, videodiscs, microcomputer-based laboratories,

the Internet, virtual learning centres, local and wide area networks, instructional software,

and educational television (Ministry of Education Draft Policy, 2005, p. 12).

Technology-based assessment. The application of ICT to assessment processes,

including delivery of tests, capture of responses, and marking by either computer or human

marker (Jisc, 2010, p. 56; Csapó, Ainley, Bennett, Latour, & Law, 2012, p. 192).

Understanding technology-based assessment. Technology can be used to manage

assessment information by presenting it in different ways to enhance the quality of

presentation or to improve the range and scope of the content (Mogey & Watt, 1996). It can

also be part of a “fully automated assessment system” when all aspects of the system is

technology-based, “from the assessment which the student completes to the processing and

administration of the marks, including the overall management of assessment information”

(Mogey & Watt, 1996, p. 0).

Twenty-first century skills. These are four broad categories:

1. Ways of thinking. Creativity, critical thinking, problem-solving, decision-

making and learning.

2. Ways of working. Communication and collaboration

3. Tools for working. Information and communications technology (ICT)

Page 16: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 15

and information literacy.

4. Skills for living in the world. Citizenship, life and career, and personal

and social responsibility.

Organization of the study. Following this presentation of the study, the remaining

chapters are organized as follows: Chapter 2 examines the current findings regarding

classroom technology integration and reviews literature pertaining to TBA as it relates to

educators and students, with an emphasis on the challenges to current TBA practices.

Chapter 3 gives details of the methodology used in collecting and analyzing the data, while

Chapter 4 covers analysis of the data along with presentation of the results and a summary of

the research findings. Chapter 5 includes a summary of the study with a discussion of the

major findings. It also provides limitations and includes recommendations for practice and

areas for further research.

Page 17: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 16

Chapter 4

Results

This chapter is based on several types of data analysis – descriptive statistics,

independent samples t-test, ANOVA, correlations and multiple regression - described in

detail in the Data Analysis section. Results are presented in response to the previously

identified research questions.

Sample description. Forty faculty members from the School of Education in three

major universities took part in this study. An analysis of the demographic and personal

characteristics, (see Appendix F), showed that 80% of the participants were females while

20% were males, and 67.5% were over the age of 50 years. Most of the sample (55%)

completed a Ph.D degree while 42.5% completed a Master’s degree, with only 2.5% holding

a Diploma. Over half of the participants (57.5%) had more than 20 years working in

Education. Nearly all the participants owned a computer at home (97.5%) and had access to

their own computers at school (90%). Daily computer usage ranged from more than 6 hours

(42.5%) to less than 1 hour (2.5%) while most of the respondents reported high to very high

computer literacy (82.5%). Approximately half of the participants (52.5%) had “a fair

amount” to “a lot” of professional training in the use of technology-based assessment (TBA)

while 47.5 % had very little to no professional training.

Technology proficiency among faculty was examined using descriptive statistics

(Appendix G). Results show the distribution of lecturers’ level of proficiency on 13

technology tools and applications using a Likert-type scale (1-None, 2-Little, 3-Moderate, 4-

High). In observing individual mean scores, more than half of the lecturers reported

“high” proficiency in using word processors (M=3.93, SD=.35), electronic mail (M=3.83,

Page 18: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 17

SD=.45), presentation software (M=3.65, SD=.58), search engines(M=3.40, SD=.87), and

computer projector systems (M=3.38, SD=.93). Over 50 % of lecturers reported “moderate”

to “high” proficiency in the use of spreadsheets (M=2.75, SD=.87), discussion lists (M=2.63,

SD=.1.13), chat forums (M=2.78, SD=1.1), instructional film (M=2.98, SD=1.07) and social

network tools (M=2.83, SD=1.11) while more than 50% reported little or no proficiency in

the use of instructional software (M=2.50, SD=1.09), drawing/graphics programs (M=2.13,

SD=1.09), or web page development (M=1.70, SD=.88).

Research question # 1. To what extent does tertiary level faculty use TBA for

students? Table 1 shows faculty’s percentage of per-semester usage of TBA. The

respondents’ Usage of TBA Scale scores ranged from 15-40, M=24.73, SD=5.71. The mean

score divided by the number of statements revealed a per question mean of 2.47, indicating

that the majority of respondents answered most statements as either “rarely” or “moderate.”

This result suggests that overall, there was low to moderate use of technology for assessment

purposes.

An analysis of individual activities revealed that faculty used technology mostly for

sending and retrieving assignments from students (M=3.38, SD=.71), preparing tests, quizzes

and assignments (M=3.05, SD=.82), scoring/managing assessment data/record keeping

(M=2.98, SD=.86), and for conducting teacher-made assessments (M=2.40, SD=1.00). The

least-used assessment activities were e-portfolios for students’ self-assessment/reflection

(M=1.83, SD=.98) and online tools to help students peer-assess (M=1.5, SD=.75).

Page 19: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 18

Table 1

Percentage of Usage per Semester of Technology-based Assessment Activities

Assessment Activity No response

No Usage

Rarely (once or twice per semester)

Moderate (several times per semester)

High (almost

weekly per semester)

Mean SD

Preparing tests, quizzes and assignments

30.0 35.0 35.0 3.05 .82

Use for Scoring/ Managing Assessment Data/ Record Keeping

2.5 30.0 35.0 32.5 2.98 .86

Use for Conducting Teacher-made Assessments

22.5 30.0 32.5 15.0 2.40 1.0

Use for Standardized Testing

30.0 30.0 32.5 7.5 2.18 .96

Use for Sending/Retrieving Assignments to and from Students

12.5 37.5 50.0 3.38 .71

Use for Discussion Forums/Wikis/Blogs to monitor Students’ Understanding, Give Feedback

22.5 - 37.5 22.5 17.5 2.35 1.03

Alternative Assignment Formats (eg. Podcasts, Multimedia Presentations)

12.5 30.0 35.0 22.5 2.68 .97

E-Portfolios for students’ self-assessment/ reflection

5.0 35.0 40.0 12.5 7.5 1.83 .98

Online Tools to Help Students Peer-Assess Eg Webpa

2.5 55.0 35.0 5.0 2.5 1.5 .75

Checking for Plagiarism in Assignments

17.5 42.5 22.5 17.5 2.4 .98

Research question # 2. What are the perceived barriers to using TBA?

Table 2 shows the distribution of faculty’s responses on perceived barriers to using

TBA. Cronbach’s alpha value of 0.751 for this scale showed an acceptable internal

consistency. The respondents’ Barriers to Using TBA scale scores ranged from 13-39,

Page 20: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 19

M=29.58, SD = 6.11, and a per question mean of 2.96. This indicated that most respondents

answered most statements as either disagree or neutral, suggesting that, in general, the stated

items were not seen as barriers to usage of TBA.

However, an analysis of individual items indicated that the top four barriers to TBA

were, “teachers often have low self-efficacy regarding their competence in using TBA”

(M=2.62, SD=.9); “faculty lacks knowledge in combining technology with learning theory

and assessment principles,” (M=2.38, SD=1.01); “inadequate school access to media for

effective use of TBA,” (M=2.35, SD=1.31); and “deficiency in technical support services

when hardware/software malfunction,” (M=2.15, SD=1.23). Based on the results, the

barriers with the least influence were, “TBA jeopardizes the security and integrity of student

data on the system,” (M=1.43, SD=1.01); and “TBA is often incompatible with my subject

area,” (M=1.18, SD=.97).

Page 21: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 20

Table 2

Percentage of Faculty’s Perceived Barriers to Using Technology-Based Assessment

Barriers to usage of TBA Strongly Disagree

%

Disagree %

Neutral %

Agree %

Strongly Agree

%

Mean SD

Existing technical infrastructure is limited

10.0 32.5 12.5 27.5 17.5 2.10 1.32

Inadequate school access to media for effective use of TBA

10.0 25.0 2.5 45.0 17.5 2.35 1.31

Deficiency in technical support when ICT malfunctions

10.0 22.5 25.0 27.5 15.0 2.15 1.23

Deficiency of professional development opportunities

7.5 45.0 20.0 25.0 2.5 1.7 1.02

TBA is often incompatible with my subject area

22.5 50.0 10.0 15.0 - 1.18 .97

Often insufficient time to prepare TBAs

5.0 30.0 30.0 25.0 10.0 2.05 1.09

Students have inadequate access to computers (at home, in school) for completion of assignments

5.0 40.0 27.5 27.5 - 1.78 .92

Teachers often have low self-efficacy regarding their competence in TBA

15.0 20.0 50.0 12.5 2.62 .91

TBA jeopardizes the security and integrity of student data on the system

12.5 52.5 20.0 10.0 5.0 1.43 1.01

Faculty lacks knowledge in combining technology with learning theory and assessment principles

25.0 25.0 37.5 12.5 2.38 1.01

Research question # 3. Is there a relationship between participants’ demographic

and personal characteristics and their usage of TBA?

Independent samples t-tests were used to determine whether the means of different

pairs of independent groups varied significantly on usage of TBA: gender, level of training,

school access to computer, and amount of daily usage of computer. The assumption of the t-

Page 22: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 21

test was met, with equal variance assumed in all cases, as determined by the Levene statistic,

p >.05, in other words, the variability of each group is approximately equal.

There was no significant difference in the mean score of usage of TBA between

faculty who had little to no training and those who had moderate to high amounts of

training, t(38) = -.187, p=.071. Additionally, faculty who had access to their own computer

at school scored higher (M=25.25, SD=5.74) than those who did not have access ((M=20.00,

SD=2.45), but this was not a statistically reliable difference, t(38) = 1.795, p=.081. Results

indicated however, that there was a statistically significant difference in mean usage of TBA

between faculty who used the computer for more than 6 hours daily and those who used

computers less than 6 hours daily, -4.98 (95%CI, -.8347 and -1.612), t (38) =-2.994, p=.005.

An ANOVA was conducted to assess the mean differences on the usage of TBA

across variables in the study with three or more groups: (1) age (N=4, 30-39, 40-49, 50-59,

>60) (2) level of education (N=4, Diploma, Associate/Bachelors, Graduate (Masters) and

Postgraduate (Ph.D), (3) years in education (N=5, 0-5 years, 6-10 years, 11-15 years. 16-20

years, >20 years); and (4) computer literacy level (N=4, low, moderate, high and very high).

With regard to age, there was a difference in means between all age groups with an

increase from 30-49 to 40-49 age group, followed by consecutive decreases in mean usage

from the 40-49, 50-59 and >60 age groups (Figure 2). However, these differences did not

reach statistical significance, (F(3, 39) = 1.328, p = .281). Similarly, the mean score usage of

TBA increased from the diploma level to graduate level to post graduate level of

qualification but this difference was not statistically significant (F(2, 37)=.659, p=. 523).

Page 23: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 22

Figure 2. Mean of TBA usage with the increase of participant’s age

Results for years of experience and usage revealed no statistical difference, F(4,35) =

.368, p=. 830, but showed increased usage in respondents with less than 10 years of service

and those with more than 20 years of service (see Figure 2).

Figure 3. Mean of TBA usage with the increase in years of experience.

Page 24: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 23

References

Albirini, A. (2006). Teachers’ attitudes toward information and communication technologies:

The case of Syrian EFL teachers. Computers & Education, 47(4), 373-398.

Almekhlafi, A. G., & Almeqdadi, F. A. (2010). Teachers' Perceptions of Technology

Integration in the United Arab Emirates School Classrooms. Journal of Educational

Technology & Society, 13(1).

An, Y. and Reigeluth, C. , 2011-11-08 "T153-TED Creating Technology-Enhanced,

Learner-Centered Classrooms: K-12 Teachers’ Beliefs, Perceptions, Barriers, and

Support Needs" Paper presented at the annual meeting of the AECT International

Convention, Hyatt Regency Jacksonville Riverfront, Jacksonville, FL. Retrieved on

June 2, 2013 from http://citation.allacademic.com/meta/p511789_index.html

Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school:

Critical conceptual and methodological issues of the construct. Psychology in the

schools, 45(5), 369-386. doi:10.1002/pits.20303

Atkins, D. E., Bennett, J., Brown, J. S., Chopra, A., Dede, C., & Fishman, B. (2010).

Transforming American education: Learning powered by technology. Learning, 114.

Bebell, D., Russell, M., & O Dwyer, L. (2004). Measuring teachers' technology uses: Why

multiple-measures are more revealing. Journal of Research on Technology in

Education, 37(1), 45-64

Beeput, R. (2012). Information and communication technology (ICT) in the primary school:

Perceptions of ICT and teacher training in Trinidad and Tobago. Abstract retrieved

from http://www.uwispace.sta.uwi.edu/dspace/handle/2139/11943a

Page 25: University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The irreversible impact of technology on tertiary level education cannot be ignored as most

TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 24

Appendix A

IRB Approval