University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The...
Transcript of University of the Southern Caribbean School of Education ... - apa/Sample Paper.pdf · The...
University of the Southern Caribbean
School of Education & Humanities
Department of Graduate Education Studies
Maracas Royal Road, St. Joseph, Trinidad. WI
Faculty’s Usage of Technology-Based Assessment: An Analysis of Perceived
Barriers in Three Universities.
A Research Project Paper
Presented in Partial Fulfillment
Of the Requirement for the Degree
Master of Arts in Educational Psychology
By
Xxxxxxxxxxxx Xxxxxxxxxxx
June 2014
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION i
University of the Southern Caribbean
School of Education & Humanities
Department of Graduate Education Studies
Maracas Royal Road, St. Joseph, Trinidad. WI
The undersigned certify that they have read, and recommend to the School of Education &
Human Sciences for acceptance, a research project titled, faculty’s usage of technology-
based assessment: an analysis of perceived barriers in three universities, by Xxxxxxxxxx
Xxxxxxxxxx, in partial fulfilment of the requirement for the degree MASTER OF ARTS IN
EDUCATIONAL PSYCHOLOGY
………………………………………. …………………………………….
Chair: Vishranti Eton, Ph.D Member: Franklyn N. Baldeo, Ph.D
……………………………………………………………
Dean: School of Education & Human Sciences, (Ag.)
Joseph Ragoonanan, MA
……………………………………
Approval Date
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION ii
Table of Contents
Page
Abstract ..................................................................................................................................... ii
Dedication ................................................................................................................................ iii
Acknowledgements ................................................................................................................... v
List of Tables ........................................................................................................................... vi
List of Figures ......................................................................................................................... vii
List of Abbreviations ............................................................................................................. viii
Chapter 1 ……………………………………………………………………………………...1
Background to the Study ........................................................................................................... 1
Statement of the problem ....................................................................................................... 11
Purpose/objectives ........................................................................................................ 12
Significance of the study ............................................................................................... 13
Definition of terms ........................................................................................................ 13
Information and communication technology (ICT) ……………………………………7
Technology-based assessment………………………………………………………….7
Understanding technology-based assessment ............................................................... 14
Twenty-first century skills ............................................................................................ 14
Ways of thinking. .............................................................................................. 14
Ways of working. ............................................................................................. 14
Tools for working ............................................................................................. 14
Skills for living in the world ............................................................................. 15
Organization of the study .............................................................................................. 15
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION iii
References …………………………………………………………………………………...50
Appendix ................................................................................................................................ 56
A. IRB Approval .................................................................................................................... 56
B. ICT Usage Survey ............................................................................................................. 57
C. Letter of Introduction ........................................................................................................ 64
D. Acceptance Letter from one University ……………………………………………....... 65
E. Informed Consent ..................................................................................................................
F. Demographic and Personal Characteristics of Sample ..........................................................
G. Proficiency Level for Technology Hardware/Software Tools ..............................................
H. Perceptions of TBA ...............................................................................................................
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION iv
Abstract
This study was conducted to determine university faculty’s usage of technology-based
assessment (TBA) and to explore their perceptions of barriers to successful integration of
TBA. An online survey was disseminated to lecturers in the Faculty of Education in three
major universities in Trinidad. Most participants spent more than 4 hours daily on the
computer, were highly computer literate and had very little professional training in the use of
TBA. Results revealed positive relationships between TBA usage and lecturers’ daily use of
computers, their proficiency with ICT applications, and their perceptions of TBA. Analysis
showed no significant relationship between usage and perceived barriers, but identified the
four top barriers as time to prepare assessments, lack of training, poor technical
infrastructure, and access to ICT hardware and software. Recommendations include more
targeted and subject-compatible professional development to increase competence and
confidence in using TBA, and more efficient technical support when ICT problems occur.
Key words: technology-based assessment, higher education faculty, technology integration,
barriers, perceptions, ICT.
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION v
Acknowledgements
I would like to express my gratitude to Dr. Eton and the School of Education faculty
for generously sharing your knowledge and guidance over the past two years.
Sincerest appreciation to Dr. Brathwaite, my research supervisor, for your insightful
feedback and support throughout my research journey and for demystifying SPSS!!
To Ms. Holder for your advice and mentorship in providing not only the “what,” but
the “how” of being a competent assessment practitioner – warmest gratitude.
To Dr. Baldeo – for your “ready ear” always and your upbeat encouragement right
down to the finish line – thank you.
To Dr. McGarrell – who re-awakened my love for writing and lit the research fire in
me. Your class and your gentle spirit were gifts to me and truly epitomized the integration of
faith and learning. Thank You!
To Dr. Elvin Gabriel – your faith in me and my abilities, and your unconditional
support in all my academic and professional pursuits, no matter how busy you are – have
been the wind beneath my wings for close to twenty years – sincerest gratitude!
To my dear friend Shalla – the instant connection we felt on the first day has
blossomed into “friend for life” status. I am forever grateful.
My deepest appreciation and love for my most ardent supporters - husband, Bunny,
and kids, Gi and Gin. You walked every step of this degree with me. Thank you, we did it!
And to God – thank you for being my Source always. Indeed, You do not call the
qualified, but You qualify the Called. Thank you for this call. It has been a life-changing
experience for which I am deeply grateful and I will strive to honour You by blooming
wherever I am planted.
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION vi
List of Tables
Table Page
1. Percentage of TBA Usage per Semester………………….…………………………… 25
2. Perceived Barriers………………………………………………………….………..... 28
3. Difference in TBA Usage Based on Literacy Levels…………………………………... 30
4. Difference in Perceptions Based on Literacy Levels …………………………………. 32
5. Difference in Perceptions Based on Age Groups………………………………………34
6. Relationship Between Usage and Six Independent Variables………............................ 35
7. Multiple Regression: Model Summary……………………………................................ 36
8. Multiple Regression: Coefficients………………………………………………… ...... 37
9. Multiple Regression ANOVA………………………………..……………………… ... 37
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION vii
List of Figures
Figure Page
1. Unified theory of acceptance and use of technology …………………………………..17
2. Mean usage of TBA with the increase of age ………………………............................ 29
3. Mean usage of TBA with years of experience...…………..………..….……………… 30
4. Mean usage of TBA with increased computer literacy………………...……………… 31
5. Mean increase in perceptions with increased computer literacy ..….…........................ 33
6. Mean increase in perceptions with increased age……..…………….…........................ 33
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION viii
List of Acronyms
TBA – Technology-based assessment
ICT – Information and Communication Technologies
CRESST – Centre for Research, Evaluation, Standards and Student Testing
US – United States
T&T – Trinidad and Tobago
TVET – Technical and Vocational Education and Training
UTAUT – Unified Theory of Acceptance and Use of Technology
UWI – The University of the West Indies
USC – The University of the Southern Caribbean
UTT – The University of Trinidad & Tobago
ANOVA – Analysis of Variance
Ph.D – Doctor of Philosophy
M – Mean
S.D. – Standard Deviation
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 9
Chapter 1
Background to the Study
“Technology in education is neither a novelty nor is it a fad. It is a part of the
modern world, and is becoming more and more ubiquitous in our lives every year” (Saba,
2009, p. 9). This unavoidable phenomenon in education, more specifically, in higher
education, served as motivation for this study.
The irreversible impact of technology on tertiary level education cannot be ignored as
most of today’s college students have spent their entire lives using computers, cell phones,
and other digital media, and have integrated technology into almost everything they do
(Oblinger, 2008; Prensky, 2007 as cited in An & Reigeluth, 2011). In their 2010 CRESST
report, Behrens, Mislevy, DiCerbo, and Levy, concur that “the world in which learning and
assessment must take place is rapidly changing” and add that “traditional assessment
methods are inadequate for evaluating learning in our digital world” (p. 6). Valdez (2005)
underscores the need for authentic assessment based on his observation that “we are living in
an age of information overload with the expectation that students will learn high-level skills
such as how to access, evaluate, analyze, and synthesize vast quantities of information while
teachers are evaluated by their ability to have students pass tests that often give no value to
these abilities” (p. 3).
Studies show that far too many students are bored, unmotivated, and uninvolved, that
is, disengaged from the academic and social aspects of school life (Appleton, Christenson, &
Furlong, 2008), partially because of irrelevant curriculum and assessment strategies
(Ralabate, 2011). Atkins, Bennett, Brown, Chopra, Dede, and Fishman, (2010) observe that
“outside school, students are free to pursue their own passions and at their own pace – the
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 10
opportunities are limitless, borderless and instantaneous” … thus challenging the education
system to balance learning theory with modern technology to create for students, “relevant
and personalized learning experiences that “mirror their daily lives and the reality of their
futures” (p. x).
In addition to its effects on student engagement, research on technology-based
assessment (TBA) is increasingly concerned with the relevance of assessment to twenty-first
century skills and students’ preparedness for the world of work. The Partnership for 21st
Century Skills, a Washington-based group involving the US Department of Education and
founding members such as Apple Computer, Inc., and Cisco Systems, Inc., asserts that “high
stakes assessments alone do not generate evidence of the skill sets that the business and
education communities believe will ensure success in the 21st century” (p. 1). Likewise, the
Trinidad and Tobago’s (T&T) 2005 Draft Policy for Information and Communications
Technology (ICT) in Education acknowledges the role of ICT to “enhance human capacity,
dynamize the teaching-learning environment, and, …create an environment that encourages
creativity, critical thinking, and decision-making” (p. 3). Currently, however, while there
have been studies regarding technology integration in schools in Trinidad (Rampersad, 2011,
Ganpat et al., 2013), there is little data on the use of technology for assessment in T&T in
higher education.
TBA research is becoming more of an imperative as new technologies emerge and as
the competition among higher education institutions increases. Technology “makes it
possible to adopt new and arguably better approaches to instruction and/or change the content
or context of learning, instruction, and assessment” (Lawless & Pellegrino, 2007, p. 581).
Still, researchers caution that institutions should not rush into technology integration without
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 11
first assessing the variables involved, such as the purposes of assessment and the contexts in
which these assessments will take place (van Merriénboer & van der Vleuten, 2012, p. 354,
Atkins et al., 2010, p. xi). Additionally, to derive the benefits of technology integration,
researchers must respond to teachers’ experiences, which are critical to the feedback,
planning and implementation stages of technology adoption efforts. Rotterham and
Willingham (2010), believe that we “waste a valuable resource when we don’t give teachers
time to share their expertise” and add that “for change to penetrate the classroom” (p. 19),
teachers’ voices must be acknowledged in meaningful ways – so that they can improve their
students’ learning experiences as well as their own professional practice.
Statement of the problem. The vast array of technological tools such as laptops,
iPods, iPads, and interactive whiteboards have altered the face of education and its delivery
in classrooms (Corrin, Lockyer, & Bennett, 2010; Gathercoal, 1999; Jamil & Shah, 2011)
prompting the need to change how we assess students in the classroom. Studies, (as cited in
Taylor & Parsons, 2011, p. 6) support this notion - “If we fail to change our pedagogy,
curriculum, and assessment strategies, we fail our students and jeopardize our own futures
(Willms, 2003; Robinson, 2009; Tapscott, 1998; Prensky, 2005; Gilbert, 2007).
In T&T, the 2005 Draft policy on ICT noted a “persistent mismatch between the
output of the education and training sector and the needs of an increasingly technologically-
oriented economy in terms of the relevance, quality, and quantity of skilled human
resources” (p. 13). The Taskforce agreed that technology integration is an essential
component in adapting to the needs of the “ever changing global environment” resulting in
the Ministry’s efforts to become more “student-centered and ICT driven” (pp. 3, 6). Still,
even with the proliferation of ICT in the Nation’s primary, secondary and tertiary-level
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 12
classrooms, there is little information, at present, from the educators’ perspective regarding
the challenges of implementing technology-based assessment.
Twenty-first century skills, including innovation and critical thinking, are
progressively being emphasized as a necessity for success in higher education and in today’s
workplace. How do we know if we are producing this type of graduate? What are our
current assessment methods and are they measuring these skills? I concur with Schleicher’s
view that “you cannot improve what you cannot measure” (as cited by Molnar, 2010).
Purpose/objectives. The important role of teachers in classroom technology
implementation efforts has been documented in many studies (Georgina & Hosford, 2009;
Taiwo, 2009; Almekhlafi & Almeqdadi, 2010; Saavedra & Opfer, 2012; Khan, 2012) and
cannot be over-emphasized. Thus, this study sought to investigate the use of technology-
based assessment from the Faculty’s viewpoint and to explore their perceptions of
institutional, personal and professional barriers to successful integration of TBA. More
specifically, the study aimed at answering the following research questions:
1. To what extent Higher Education Faculty used technology-based assessment for
students?
2. What were the perceived barriers to using TBA?
3. Was there a relationship between participants’ demographic and personal
characteristics and their usage of TBA?
4. Were the perceptions of TBA affected by (a) computer literacy and (b) age?
5. Was there a relationship between usage of TBA and 1) computer literacy, 2) daily
use, 3) proficiency with hardware/software confidence, 4) perceptions of TBA 5)
confidence with applying TBA, and 6) perceived barriers?
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 13
6. What were the factors that encouraged future use of TBA?
Significance of the study. In recognizing a need for a better-skilled and qualified
workforce, T&T has invested US $2 billion over the past decade in tertiary and technical and
vocational education (Policy on Tertiary Education, TVET, and Lifelong Learning in
Trinidad and Tobago, 2011, p. 13). With so much money being invested in technology
integration, there is very little understanding regarding the extent to, and manner in which
TBA is used in higher education in T&T. International research also shows that technology
is still under-utilized in assessment and feedback practices (Jisc, 2010, p. 8.) and that
institutions are not using “the full flexibility and power of technology to design, develop, and
validate new assessment materials and processes for both formative and summative uses”
(Atkins, et al., 2010, p. 25). As such, there is need to explore the challenges faced by those
who are already using TBA and the barriers that prevent inexperienced users from increasing
their use.
In identifying ways in which technology is used as an assessment tool at the tertiary
level, and in highlighting challenges to implementation, this study can assist teachers,
administrators, and policy makers to eliminate or reduce barriers to the adoption of TBA. It
can also inform the professional development gap which, according to Godzicki et al., (2013)
has been fraught with “serious problems, often being based on the “somewhat narrow view
that teachers need only technical skills and a good attitude” (p. 32). This is critical, given the
prediction that “assessment will continue to move towards technologically-supported
automation” (Redecker et al., 2011, p. 30).
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 14
Definition of terms. The following are a list of terminologies and phrases that were
uses in this report of the study. It is intended to give the context in which these terms were
used.
Information and communication technology (ICT). Technologies used for the
handling and communication of information, specifically in education – including, but not
limited to, computers, audio visual systems, videodiscs, microcomputer-based laboratories,
the Internet, virtual learning centres, local and wide area networks, instructional software,
and educational television (Ministry of Education Draft Policy, 2005, p. 12).
Technology-based assessment. The application of ICT to assessment processes,
including delivery of tests, capture of responses, and marking by either computer or human
marker (Jisc, 2010, p. 56; Csapó, Ainley, Bennett, Latour, & Law, 2012, p. 192).
Understanding technology-based assessment. Technology can be used to manage
assessment information by presenting it in different ways to enhance the quality of
presentation or to improve the range and scope of the content (Mogey & Watt, 1996). It can
also be part of a “fully automated assessment system” when all aspects of the system is
technology-based, “from the assessment which the student completes to the processing and
administration of the marks, including the overall management of assessment information”
(Mogey & Watt, 1996, p. 0).
Twenty-first century skills. These are four broad categories:
1. Ways of thinking. Creativity, critical thinking, problem-solving, decision-
making and learning.
2. Ways of working. Communication and collaboration
3. Tools for working. Information and communications technology (ICT)
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 15
and information literacy.
4. Skills for living in the world. Citizenship, life and career, and personal
and social responsibility.
Organization of the study. Following this presentation of the study, the remaining
chapters are organized as follows: Chapter 2 examines the current findings regarding
classroom technology integration and reviews literature pertaining to TBA as it relates to
educators and students, with an emphasis on the challenges to current TBA practices.
Chapter 3 gives details of the methodology used in collecting and analyzing the data, while
Chapter 4 covers analysis of the data along with presentation of the results and a summary of
the research findings. Chapter 5 includes a summary of the study with a discussion of the
major findings. It also provides limitations and includes recommendations for practice and
areas for further research.
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 16
Chapter 4
Results
This chapter is based on several types of data analysis – descriptive statistics,
independent samples t-test, ANOVA, correlations and multiple regression - described in
detail in the Data Analysis section. Results are presented in response to the previously
identified research questions.
Sample description. Forty faculty members from the School of Education in three
major universities took part in this study. An analysis of the demographic and personal
characteristics, (see Appendix F), showed that 80% of the participants were females while
20% were males, and 67.5% were over the age of 50 years. Most of the sample (55%)
completed a Ph.D degree while 42.5% completed a Master’s degree, with only 2.5% holding
a Diploma. Over half of the participants (57.5%) had more than 20 years working in
Education. Nearly all the participants owned a computer at home (97.5%) and had access to
their own computers at school (90%). Daily computer usage ranged from more than 6 hours
(42.5%) to less than 1 hour (2.5%) while most of the respondents reported high to very high
computer literacy (82.5%). Approximately half of the participants (52.5%) had “a fair
amount” to “a lot” of professional training in the use of technology-based assessment (TBA)
while 47.5 % had very little to no professional training.
Technology proficiency among faculty was examined using descriptive statistics
(Appendix G). Results show the distribution of lecturers’ level of proficiency on 13
technology tools and applications using a Likert-type scale (1-None, 2-Little, 3-Moderate, 4-
High). In observing individual mean scores, more than half of the lecturers reported
“high” proficiency in using word processors (M=3.93, SD=.35), electronic mail (M=3.83,
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 17
SD=.45), presentation software (M=3.65, SD=.58), search engines(M=3.40, SD=.87), and
computer projector systems (M=3.38, SD=.93). Over 50 % of lecturers reported “moderate”
to “high” proficiency in the use of spreadsheets (M=2.75, SD=.87), discussion lists (M=2.63,
SD=.1.13), chat forums (M=2.78, SD=1.1), instructional film (M=2.98, SD=1.07) and social
network tools (M=2.83, SD=1.11) while more than 50% reported little or no proficiency in
the use of instructional software (M=2.50, SD=1.09), drawing/graphics programs (M=2.13,
SD=1.09), or web page development (M=1.70, SD=.88).
Research question # 1. To what extent does tertiary level faculty use TBA for
students? Table 1 shows faculty’s percentage of per-semester usage of TBA. The
respondents’ Usage of TBA Scale scores ranged from 15-40, M=24.73, SD=5.71. The mean
score divided by the number of statements revealed a per question mean of 2.47, indicating
that the majority of respondents answered most statements as either “rarely” or “moderate.”
This result suggests that overall, there was low to moderate use of technology for assessment
purposes.
An analysis of individual activities revealed that faculty used technology mostly for
sending and retrieving assignments from students (M=3.38, SD=.71), preparing tests, quizzes
and assignments (M=3.05, SD=.82), scoring/managing assessment data/record keeping
(M=2.98, SD=.86), and for conducting teacher-made assessments (M=2.40, SD=1.00). The
least-used assessment activities were e-portfolios for students’ self-assessment/reflection
(M=1.83, SD=.98) and online tools to help students peer-assess (M=1.5, SD=.75).
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 18
Table 1
Percentage of Usage per Semester of Technology-based Assessment Activities
Assessment Activity No response
No Usage
Rarely (once or twice per semester)
Moderate (several times per semester)
High (almost
weekly per semester)
Mean SD
Preparing tests, quizzes and assignments
30.0 35.0 35.0 3.05 .82
Use for Scoring/ Managing Assessment Data/ Record Keeping
2.5 30.0 35.0 32.5 2.98 .86
Use for Conducting Teacher-made Assessments
22.5 30.0 32.5 15.0 2.40 1.0
Use for Standardized Testing
30.0 30.0 32.5 7.5 2.18 .96
Use for Sending/Retrieving Assignments to and from Students
12.5 37.5 50.0 3.38 .71
Use for Discussion Forums/Wikis/Blogs to monitor Students’ Understanding, Give Feedback
22.5 - 37.5 22.5 17.5 2.35 1.03
Alternative Assignment Formats (eg. Podcasts, Multimedia Presentations)
12.5 30.0 35.0 22.5 2.68 .97
E-Portfolios for students’ self-assessment/ reflection
5.0 35.0 40.0 12.5 7.5 1.83 .98
Online Tools to Help Students Peer-Assess Eg Webpa
2.5 55.0 35.0 5.0 2.5 1.5 .75
Checking for Plagiarism in Assignments
17.5 42.5 22.5 17.5 2.4 .98
Research question # 2. What are the perceived barriers to using TBA?
Table 2 shows the distribution of faculty’s responses on perceived barriers to using
TBA. Cronbach’s alpha value of 0.751 for this scale showed an acceptable internal
consistency. The respondents’ Barriers to Using TBA scale scores ranged from 13-39,
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 19
M=29.58, SD = 6.11, and a per question mean of 2.96. This indicated that most respondents
answered most statements as either disagree or neutral, suggesting that, in general, the stated
items were not seen as barriers to usage of TBA.
However, an analysis of individual items indicated that the top four barriers to TBA
were, “teachers often have low self-efficacy regarding their competence in using TBA”
(M=2.62, SD=.9); “faculty lacks knowledge in combining technology with learning theory
and assessment principles,” (M=2.38, SD=1.01); “inadequate school access to media for
effective use of TBA,” (M=2.35, SD=1.31); and “deficiency in technical support services
when hardware/software malfunction,” (M=2.15, SD=1.23). Based on the results, the
barriers with the least influence were, “TBA jeopardizes the security and integrity of student
data on the system,” (M=1.43, SD=1.01); and “TBA is often incompatible with my subject
area,” (M=1.18, SD=.97).
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 20
Table 2
Percentage of Faculty’s Perceived Barriers to Using Technology-Based Assessment
Barriers to usage of TBA Strongly Disagree
%
Disagree %
Neutral %
Agree %
Strongly Agree
%
Mean SD
Existing technical infrastructure is limited
10.0 32.5 12.5 27.5 17.5 2.10 1.32
Inadequate school access to media for effective use of TBA
10.0 25.0 2.5 45.0 17.5 2.35 1.31
Deficiency in technical support when ICT malfunctions
10.0 22.5 25.0 27.5 15.0 2.15 1.23
Deficiency of professional development opportunities
7.5 45.0 20.0 25.0 2.5 1.7 1.02
TBA is often incompatible with my subject area
22.5 50.0 10.0 15.0 - 1.18 .97
Often insufficient time to prepare TBAs
5.0 30.0 30.0 25.0 10.0 2.05 1.09
Students have inadequate access to computers (at home, in school) for completion of assignments
5.0 40.0 27.5 27.5 - 1.78 .92
Teachers often have low self-efficacy regarding their competence in TBA
15.0 20.0 50.0 12.5 2.62 .91
TBA jeopardizes the security and integrity of student data on the system
12.5 52.5 20.0 10.0 5.0 1.43 1.01
Faculty lacks knowledge in combining technology with learning theory and assessment principles
25.0 25.0 37.5 12.5 2.38 1.01
Research question # 3. Is there a relationship between participants’ demographic
and personal characteristics and their usage of TBA?
Independent samples t-tests were used to determine whether the means of different
pairs of independent groups varied significantly on usage of TBA: gender, level of training,
school access to computer, and amount of daily usage of computer. The assumption of the t-
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 21
test was met, with equal variance assumed in all cases, as determined by the Levene statistic,
p >.05, in other words, the variability of each group is approximately equal.
There was no significant difference in the mean score of usage of TBA between
faculty who had little to no training and those who had moderate to high amounts of
training, t(38) = -.187, p=.071. Additionally, faculty who had access to their own computer
at school scored higher (M=25.25, SD=5.74) than those who did not have access ((M=20.00,
SD=2.45), but this was not a statistically reliable difference, t(38) = 1.795, p=.081. Results
indicated however, that there was a statistically significant difference in mean usage of TBA
between faculty who used the computer for more than 6 hours daily and those who used
computers less than 6 hours daily, -4.98 (95%CI, -.8347 and -1.612), t (38) =-2.994, p=.005.
An ANOVA was conducted to assess the mean differences on the usage of TBA
across variables in the study with three or more groups: (1) age (N=4, 30-39, 40-49, 50-59,
>60) (2) level of education (N=4, Diploma, Associate/Bachelors, Graduate (Masters) and
Postgraduate (Ph.D), (3) years in education (N=5, 0-5 years, 6-10 years, 11-15 years. 16-20
years, >20 years); and (4) computer literacy level (N=4, low, moderate, high and very high).
With regard to age, there was a difference in means between all age groups with an
increase from 30-49 to 40-49 age group, followed by consecutive decreases in mean usage
from the 40-49, 50-59 and >60 age groups (Figure 2). However, these differences did not
reach statistical significance, (F(3, 39) = 1.328, p = .281). Similarly, the mean score usage of
TBA increased from the diploma level to graduate level to post graduate level of
qualification but this difference was not statistically significant (F(2, 37)=.659, p=. 523).
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 22
Figure 2. Mean of TBA usage with the increase of participant’s age
Results for years of experience and usage revealed no statistical difference, F(4,35) =
.368, p=. 830, but showed increased usage in respondents with less than 10 years of service
and those with more than 20 years of service (see Figure 2).
Figure 3. Mean of TBA usage with the increase in years of experience.
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 23
References
Albirini, A. (2006). Teachers’ attitudes toward information and communication technologies:
The case of Syrian EFL teachers. Computers & Education, 47(4), 373-398.
Almekhlafi, A. G., & Almeqdadi, F. A. (2010). Teachers' Perceptions of Technology
Integration in the United Arab Emirates School Classrooms. Journal of Educational
Technology & Society, 13(1).
An, Y. and Reigeluth, C. , 2011-11-08 "T153-TED Creating Technology-Enhanced,
Learner-Centered Classrooms: K-12 Teachers’ Beliefs, Perceptions, Barriers, and
Support Needs" Paper presented at the annual meeting of the AECT International
Convention, Hyatt Regency Jacksonville Riverfront, Jacksonville, FL. Retrieved on
June 2, 2013 from http://citation.allacademic.com/meta/p511789_index.html
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school:
Critical conceptual and methodological issues of the construct. Psychology in the
schools, 45(5), 369-386. doi:10.1002/pits.20303
Atkins, D. E., Bennett, J., Brown, J. S., Chopra, A., Dede, C., & Fishman, B. (2010).
Transforming American education: Learning powered by technology. Learning, 114.
Bebell, D., Russell, M., & O Dwyer, L. (2004). Measuring teachers' technology uses: Why
multiple-measures are more revealing. Journal of Research on Technology in
Education, 37(1), 45-64
Beeput, R. (2012). Information and communication technology (ICT) in the primary school:
Perceptions of ICT and teacher training in Trinidad and Tobago. Abstract retrieved
from http://www.uwispace.sta.uwi.edu/dspace/handle/2139/11943a
TECHNOLOGY-BASED ASSESSMENT IN HIGHER EDUCATION 24
Appendix A
IRB Approval