ICT research in Africa: Towards appropriate research

methodologies

Duan van der Westhuizen

Faculty of EducationUniversity of Johannesburg

“There is a growing concern internationally that the investment in ICT in schools is not impacting on literacy development. This concern arises from a belief held by many - including governments as well as schools - that ICT is beneficial to learning, and specifically literacy learning.”.

Torgerson C, Zhu D (2003)

"It is generally believed that ICTs can empower teachers and learners, promote change and foster the development of '21st century skills', but data to support these beliefs are still limited".

infoDev.org

The positive impact of ICT use in education has not been proven In general, and despite thousands of impact studies, the impact of ICT use on student achievement remains difficult to measure and open to much reasonable debate.

“There is neither a strong and well-developed theoretical case nor much empirical evidence supporting the expected benefits of ICT …”

Contrasting evidence: BECTA (2002) and Machin et al(2006) found positive effects, while Fuchs & Woessman (2004), Leuven et al (2004) and others found no real positive effect

Threat 1: Publish or perish

• The research endeavour is a primary imperative for universities

• Universities under state pressure to up research output• Universities rewarded for publication by funding model• Accredited journal articles preferred and rewarded• Cascades down

– University leadership– Faculty leadership– Departmental leadership– Individual faculty members

• What drives the push for increased output?

Solani Ngobeni: Fostering academic mediocrity

• About 96% of research output in South Africa is journal articles• Tied to the funding model for universities• Financially lucrative for universities to publish in accredited

journals• This turns universities into managerialised corporate entities• Lecturers become revenue-generating employees• Publishing becomes a Key Performance Area at all the levels• Therefore, publishing is not about scholarship anymore, but

about generating revenue• It is not about quality, but about quantity• The system of journal publication is not without problems• And nor is the system of ‘rated’ journals (ISI, IBBS, DoHE)

Threat 2: Quality of Educational Research

• Hargreaves (1996): Educational research is poor value for money, remote from practice, and of poor quality

• Considerable amount of second rate research that is irrelevant to practice and clutters up journals that nobody reads

• A study by OFSTED explored several (41) educational publications.– Partisanship in the conduct of research, the presentation of

research and the argument of research– There are methodological issues– Non-empirical research did not consult primary resources– Focus was often obfuscated

• Researchers “employ weak research methods, write turgid prose, and issue contradictory findings” (Reeves, 2002)

• Dillon and Gabbard (1998): “statistical analyses & research methods are frequently flawed, limiting our understanding of these important issues” and limiting generalisation of findings.

Pasteur’s quadrant (Stokes)

Research inspired by use

No Yes

Research inspired by quest for fundamental understanding (theory)

Yes Research by Bohr for an atomic model

Research by Pasteur for pasteurisation

No

Discoveries and research done by Edison

Basic Research and Applied Research

• In Science—The Endless Frontier (Bush, 1990) it is argued that basic research should be performed “without the thought of practical ends”

• There is a linear progression from basic research to application, to social innovation

• Stokes (1997) claims that this thinking is fundamentally flawed• The motive for fundamental understanding and the drive for

application are not separate or in opposition to each other• Research can be undertaken both as a quest for basic

understanding (rigor) and with considerations of use (relevance).

Researching Education:The hardest science of all?

• Educational Science is incredibly hard to do• Hard and soft science: a flawed dichotomy?• ‘Hard’ science is based on randomization, generalization, based

on ‘empiric’ data, often in laboratory conditions• ‘Soft’ science is squishy, unreliable and imprecise• Berliner (2002) claims ‘hard’ science is easy science, and

educational science is the hardest science of all• Deal with contexts that limit generalization and theory

building• Challenge to replicate findings• Challenge to find the numbers for statistical analysis• Does not seek to find the regularity that nature provides

The hardest Science (cont)

• Humans learners in schools are embedded in complex and changing networks of social interaction

• The context can never be controlled• The variety of ‘variables’ (e.g. ability, status, motivation,

teacher competence, conceptions of learning, beliefs about assessment) are continuously interacting, and the direction or reciprocity of interactions are difficult to determine

• Educational science also suffers from the ‘decade by findings interactions’ phenomenon– Change in social, cultural and intellectual environments

negate previous findings

Threat 3: Quality of Educational ICT Research

• ICT research lacks a sense of history• Longitudinal studies lag behind ICT developments• The focus on ICT (as an object) research, not on learning

impact • Over-reliance on Qualitative methods • Quantitative work is generally on ‘attitude’ and ‘usage’• Simple descriptive analyses• Few comparative analyses (e.g. that of Lim, 2003, and

Pelgrum, 2001) • Small-scale studies that fail to produce findings that are

reliable and generalisable• Much of the research is dispersed at sub-discipline level

Threat 4: Lack of large-scale meta-analyses

• Most prior meta-analyses were conducted on Distance Education

• Kulik & Kulik pre-1980• Most recently by US DoE: Evaluation of

Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies

• Project RED: THE TECHNOLOGY FACTOR

Threat 5: Insufficient local research

• Van der Westhuizen (2002), there is little research addressing and locating specifically local concerns

• Annotated Bibliography on e-Learning and Application of Educational Technology in African Countries, or in Contexts Relevant to Africa (Carnegie)

• Deficits in Academic Staff Capacity at African Universities (Partnership for Higher Education in Africa, 2010)

• Growth in student numbers not met by growth in staff• Lack of female staff• Low numbers of PhD and M students• Staff qualifications• SA: CHE Report on Higher Education Landscape

Case Study M & D ICT Research in SA

• Finding congruence between the purpose of the inquiry, the theoretical framework in which the study was conducted, the research design and the interpretation of findings.

• Not simply mapping the nature of theoretical frameworks in the inventory, but to interpret and critique the way in which the authors used theoretical framing as a scholarly device.

Method

• Seven leading universities ito publication output• Sample studies were identified through library

keyword searches • Samples were either taken off the shelves or

obtained digitally• Research instrument was applied to 103 studies• Instrument captured summaries of

– Research question– Supporting theories– Research paradigms– Findings (conclusions)

Discussion: towards theoretical emancipation (Best cases)

• a small insignificant portion of the selected studies had theoretical frameworks as epistemological guides

• these studies were conceptualised coherently and the various aspects of the inquiries were congruent

• studies that were theoretically developed yielded data that could be interpreted in more depth

Discussion: towards theoretical emancipation (2)

• A substantial majority of authors employed theoretical frameworks in a very limited way

• theoretical frameworks in literature reviews without using them as epistemological guides

• findings that were descriptive in nature, contrary to a claimed interpretivist stance.

So ……

• Educational ICT is a developing research field and the disciplinary training is not yet cohesive

• There is evidence that the researchers remained practitioners of Educational ICT and did not manage to become scholars

• It raises the concern that supervisors in M & D programmes are not equipping their students with the necessary skills to “find an epistemological home” for their studies and to analyse and interpret results.

Possible causes

• The demands of transformation and standardisation have been many

• Academic and social “fall out” of mergers • Certainly, the state of supervision in the country is

generally not sufficient, as is evident in the drive of an organisation such as SANTRUST (ex SANPAD)

• supervisors are increasingly expected to have a high and accelerated output and ‘through put’

• The demand to increase a university’s income is seductive for an academic and linked to promotion

How?

• Abandon case studies• Abandon “user experience” ‘research’• Course evaluation is not research• Research projects must be theoretically founded, and have

theoretical outcomes• Research-competency training• Select appropriate research designs • Encourage research teams

– Supervisor 1 (Senior) – PhD student 1 - PhD Student 2 - Supervisor 2 (Junior) Master Student 1, 2, 3

– Work on (dimensions of) focussed, authentic problems– International collaborative research

Appropriate Designs?

•Experimental/quasi-experimental research (considering the ‘random assignment’ requirement)•Large-scale studies•Longitudinal studies•Repeated-measures studies•Design-based Research (which addresses all of the above)

Thank you