Paper presented at ‘Outdoor education research and theory: critical reflections, new directions’, the Fourth International Outdoor Education Research Conference, La Trobe University, Beechworth, Victoria, Australia, 15-18 April 2009.

1

Learning from a review of research on outdoor learning Kathleen Pleasants La Trobe University

In a recent study (Pleasants, 2008) I examined the purposes and roles of reviews of research in the fieldsi of outdoor education by reflecting on one example of this genre, namely, A review of research on outdoor learning conducted by Rickinson, Dillon, Teamey, Morris, Choi, Sanders, and Benefield (2004). My study drew on a range of parallel and intersecting conversations about the functions of reviews of research in education to highlight a lack of debate about reviews of research, and more particularly, the role of research within outdoor education. I contend that this silence illustrates a continued and concerning divide between research and practice. In this paper I will discuss the application of both Foster and Hammersley’s (1998) framework for reading reviews of research and Cherryholmes’ (1993) proposition in favour of reading pragmatically to A review of research on outdoor learning (Rickinson et al., 2004). I will focus on the recommendations of the 2004 review, and briefly consider some future directions that outdoor education research might take by highlighting how understandings about the role of research within the field of education might be useful in addressing critical questions about the achievements of outdoor education research, its progress, and current and/or emerging challenges.

“Pragmatists believe that they must choose and act without knowing whether they or anyone else have got things right” (Cherryholmes, 1999, p. 75)

Introduction & background The impetus for the study on which this paper is based grew out a desire to develop my own understandings of the functions and potential for research in outdoor education. My explorations led me to the review of research conducted by Mark Rickinson and his team (Rickinson, et al., 2004). From here, I moved to examine other reviews that had gone before (see, for example, Barrett & Greenaway, 1995; Cason & Gillis, 1994; Crompton & Sellar, 1981; Ewert, 1987; Hattie, Marsh, Neill, & Richards, 1997). I came to recognise that one way of developing a deeper understanding of the functions of research in outdoor education might be to explore the purposes and roles of reviews of research within education more broadly and apply these to a specific example from within the field. An additional desire was to draw attention to the idea that all research findings need to be seen in relation to the assumptions being made about the field from whence they emanate. In the case of the Rickinson et al. (2004) review I might ask, ‘What types of outdoor education are being highlighted? What does this review of research say about the effects of outdoor education? What might the review have missed?’. The idea that it might be useful to raise awareness about the context of research findings was influenced by Rickinson’s (2003) own reflection on his previous review of research on learners and learning in environmental education (see Rickinson, 2001), in which he suggested that future reviews of research might benefit from “a deeper engagement with literature and debates on criteria of methodological quality before, as well as during, the review process” (p. 263). My intention, therefore, might be summarised as being to scrutinise the review in order to determine:

1. How the reviewers had interpreted outdoor education research; 2. What other interpretations might exist of what constitutes research in outdoor

education; and 3. Potential gaps or silences in the representation of outdoor education within the

review.

Learning from a review of research on outdoor learning

2

My eventual thesis consisted of two, parallel intentions. While the primary purpose of the study was to examine the purposes and roles of reviews of research within the field of outdoor education by reflecting on one example of this genre, a parallel intention was to explore the suitability of two complementary methods of interrogating texts: Cleo Cherryholmes’ (1993) argument to abandon “fundamentalist” approaches to reading research in favour of reading pragmatically, and the four-part model for reviewing reviews of research proposed by Peter Foster and Martin Hammersley (1998).

Reading pragmatically Cherryholmes (1993) draws attention to how readers of research can and ought choose among various rhetorical strategies, in order to highlight the variety of understandings that can arise. He highlights how the stories told by researchers can be historical and connected to political agendas: “research findings are often fragmentary, incomplete, dispersed [and] subject to multiple readings” (p. 2). The rhetorical strategy adopted by the reader might be that of a feminist, critical, deconstructive, historical, or rational analytical reader, for example, and can offer insights and make available an alternative lens. Tactics such as these allow the reader to seek additional stories in the texts of research (see, for example, Cherryholmes, 1992, 1993, 1999).

Cherryholmes (1999) suggests that a pragmatic exercise requires the reader to trace a text’s conceivable practical consequences: “Reading to clarify meanings by pursuing consequences and forsaking essentialist and foundationalist conceits necessarily looks beyond provincial and parochial categories, interests, boundaries, and narratives” (p. 76). On the other hand, according to Schole (quoted in Cherryholmes 1993), a fundamentalist reading, demands “that texts in time-bound languages convey that meaning so directly that it can be discerned without interpretation” (p. 2). This approach requires readers to ignore issues of context and embark on a search for some essential truth within the production of research. Again, rather than search for truth in meaning, pragmatism demands that readers clarify meaning by examining the consequences of research.

Reviews of research are produced from within social contexts and Prior (2003), among others, draws attention to the ways that readers might question the processes and circumstances surrounding their production. Popkewitz (1999) further highlights how the choice of papers in a review of research may confer recognition, and hence power, to different political agendas within a given field. A framework for interrogation Initiation by whom? For whom? For what purpose? Foster and Hammersley’s (1998) model of inquiry recommends four areas for investigation: “initiating reviews, defining the field, coverage and treatment of relevant research, and, drawing conclusions” (p. 610). They concentrate their attention on reviews of research rather than individual research documents, asserting that the nature of the phenomena studied by social and educational researchers can produce a lack of reliability in the results of single studies, suggesting instead, that the foremost avenue for communicating between researchers and practitioners should be reviews of entire research fields as opposed to single study reports. Foster & Hammersley maintain that, regardless of the acknowledged initiating body, researchers and readers of research reviews would do well to recognise “the complexity of the motives frequently involved in the production of reviews, the negotiative and sometimes even coercive relationships between sponsors and reviewers, and how these features can affect the eventual product” (p. 611). They further argue that those producing research

Kathleen Pleasants

3

reviews cannot afford to define their intended audience too narrowly because there is no such thing as a “well-defined and internally homogeneous professional audience. Instead the audience is likely to consist of multiple, and overlapping publics whose boundaries and characteristics are ill-defined” (p. 613), resulting in significant diversity. In the case of Rickinson et al. (2004), at least eight partner organizations were involved in the funding of the project, however, the environmental education charity, the Field Studies Council, is acknowledged as the primary initiator. According to Rickinson et al. (2004), A review of research on outdoor learning was initiated in response to apparent concern about declining opportunities for outdoor learning available to school students in England. It is not clear how widespread this concern is, and evidence for this claim is limited (an issue which is discussed in more detail later in this paper).

Foster and Hammersley (1998) differentiate between the character of internally and externallyii generated reviews, but warn against drawing simplistic contrasts based on distinguishing between reviews commissioned from within and without the world of research. Such a distinction, they argue, may be too one-dimensional, and risk ignoring the complexity of shaping factors. The significant difference they do highlight is the independent nature of the outside agency and reviewer versus the potential constraints operating on an internally generated review, such as the need to seek external sponsorship. They also identify a complication in the possibility that some educational researchers may also identify themselves as practitioners, hence influencing the nature of a review. This is an issue that Rickinson et al. (2004) discuss, but do not address satisfactorily for me.

Although an independent body commissioned Rickinson et al.’s (2004) review, there is every possibility that the mixture of charitable organizations, professional groups and a large and influential government department (the Department for Education and Skills) who partnered the initiative, resulted in various constraints on its eventual nature. It is only possible to speculate as to the scope of such constraints however, as Foster and Hammersley (1998) argue, regardless of the alleged initiating body, those who consume research reviews must acknowledge the various motivations that may lie behind reviews, and how the relationships between sponsors and reviewers may affect the shape of the outcome. Prior (2003) also recognises the active role of those who consume documents, including reviews of research, maintaining that both author and reader are active in the construction and production of knowledge itself. This claim is supported by a number of other authors (see, for example, Apple, 1999; Baker, 1999; Lather, 1999; MacLure, 2005; Sauve & Berryman, 2003).

In highlighting that there is rarely such a thing as an “internally homogenous professional audience” (p. 613), Foster and Hammersley (1998) argue that the producers of reviews of research should avoid narrow definitions of their intended audience, but recognize the potential for overlap and significant internal heterogeneity within audiences. Similarly, Cherryholmes (1993) demonstrates three alternative approaches to a pragmatic reading of a single research study, illustrating the varying and alternative conclusions that may be drawn from each. As researchers have little or no control over who may read their work, or how they might choose to read it, they may do well to anticipate an audience that differs from that stated and consequently shape their work to also address their “other” audiences. Rickinson et al. (2004) acknowledge this potential to some extent when they write “we are mindful of the fact that research findings can be understood and used by users in many different ways depending upon individual interests, roles and contexts” (p. 52). I suggest that this is extremely important in a field as diverse as outdoor education (see note i). It was explicitly practitioners within the field of outdoor education (in its various forms) who were identified as the target of this review of research and according to Rickinson et al.

Learning from a review of research on outdoor learning

4

(2004), the categories of learning activities used within A review of research on outdoor learning were intended to “be recognizable and understandable to practitioners within the outdoor sector” (p. 16). Publication of the report was largely limited to the United Kingdom, although the report summary is available online (Field Studies Council, 2007). The differentiation of practitioners as seemingly distinct from researchers is problematic for me and is nevertheless contradicted elsewhere in the review. I will briefly consider the ways in which this review of research has been taken up in the field elsewhere in this paper.

Despite explicitly identifying practitioners as a major target audience, it is clear that the authors of the review were at least mindful of producing a review that also addressed policy makers and researchers. In Chapter 8 of the review they identify key messages from the review for future practice, policy and research in outdoor learning:

we feel there is much in this review that is of relevance and use to practitioners and policy makers. We are aware . . . of the challenges and opportunities associated with using research to inform educational practice and policy. We are mindful of the fact that research findings can be understood and used by users in many different ways depending upon individual interests, roles and contexts. (Rickinson et al., 2004, p. 52).

According to Prior (2003) documents “both express and represent a set of discursive practices [and] as such they can be recruited as allies in various forms of social, political and cultural struggle” (p. 13). Foster and Hammersley (1998) describe how the purpose of a review may be multifaceted: to endorse policy, to promote a political issue, to demonstrate the practical value of research, or to emphasise the need for funding. Rickinson et al. (2004) identify a range of issues relating to outdoor learning by school students in England that led to this review and that fall into several of the categories identified above. The interests and agendas of the initiating bodies substantially influenced the project aims. For example, Rickinson et al. (2004) identify that “in response to the project funders’ core interests . . . the research team did not look at research on learning beyond the classroom in indoor settings, such as museums, art galleries and zoos” (p. 9). In terms of practical consequences (Cherryholmes, 1993) then, it is likely that “the stories told by [these] research findings are fragmentary, contradictory, incomplete and dispersed. . . . [and] subject to multiple readings” (p. 2). As I have noted previously, Rickinson et al. (2004), acknowledge this possibility, but do not address it. Despite potential limitations such as this, there is evidence that this research review seeks to document a degree of unity and development within the field of outdoor learning in the manner suggested by Eisenhart (1998) and Popkewitz (1999). Rickinson et al. (2004) conclude that their review demonstrates the extensive amount and diversity of research undertaken in the field of outdoor learning in the timeframe considered. They highlight several positive indicators from within the three areas of outdoor learning they considered, including: “a diversification of research . . . increased empirical enquiry . . . more sustained theoretical exploration of the history and philosophy of outdoor education, . . . [and] a growing number of meta-analyses and reviews of research” (p. 54).

The role of factual and value assumptions in providing a framework for research reviews I suggest that identifying a body of research to be reviewed necessarily introduces assumptions, particularly if the field is one that is not easily identified as in the case of outdoor education (see, for example, (Brookes, 2002, 2004; Easther, 1982; Lugg, 2004; McRae, 1990). This may necessarily result in artificial constructions, and lead to fuzzy, and even controversial boundaries according to Foster and Hammersley (1998). Ultimately the

Kathleen Pleasants

5

reviewer makes a decision about how to carve up the particular field however, and they should be explicit about such demarcations. In investigating how Rickinson et al. (2004) chose to focus their review, and hence define the field, I looked to the assumptions made within the research review. Following Foster and Hammersley (1998) some of the questions I asked included, for example:

What presuppositions are and are not justified in the structuring of research reviews? How should these be handled? . . . when do they need to be made explicit? . . . is it necessary for these decisions to be supported by argument when they are controversial, or can they simply be adopted as working assumptions? (pp. 617-618)

Eisenhart (1998) writes that “good reviews . . . establish dimensions, so that the width and breadth of the field are defined” (p. 394), but others warn that the way in which boundaries around the focus of a study are drawn, and the reasons why the field has been so defined, are significant issues that may need more attention and are some aspects of reviews about which readers may need to ask questions. In order to examine the ways in which Rickinson et al. (2004) chose to define the field, I will look to some of the assumptions made within the research review. A decline in opportunities for outdoor learning?

Rickinson’s et al. (2004) focus on “research on outdoor learning” (p. 9) is in response to “growing concern that opportunities for outdoor learning by school students in England have decreased substantially in recent years” (p. 9). Although two examples are cited to support this concern (Barker, Slingsby, & Tilling, 2002; Harris, 1999), little further support for this assertion is provided. Adopting the decline of opportunities for outdoor learning as a working assumption and the basis of this review becomes problematic for me when predicated on only two studies, one of which is based on fieldwork conducted in biology. However, for the purpose of this review it may be a suitable reference as science studies fieldtrips fall into the category of outdoor learning used by Rickinson et al. (2004). Of more immediate concern is the inclusion of a case study (Harris, 1999) conducted within a Local Education Authority in the south of England, containing a narrowly-focussed and somewhat limited discussion. This paper was published in Horizons, the non-refereed journal of the Institute for Outdoor Learning in the United Kingdom. Foster and Hammersley (1998) point to the distinctions between published and unpublished research, highlighting that although it may often be able to provide more up-to-date information, the inclusion of unpublished material may be questionable given that it “has not been through any public process of collective assessment” (p. 621). Following from this, I question the use of Harris’s (1999) study as one of only two ‘cornerstones’ on which to build a review. Outdoor learning defined

In the United Kingdom, as elsewhere (see, for example, Board of Studies, 2000; Brookes, 2002, 2004; Ford, 1981; McRae, 1990; Nichols, 1982; Priest, 1988), there are diverse views regarding the nature of outdoor education. Rickinson et al. (2004) briefly outline some of the key traditions influencing outdoor education within the UK including: the nature study movement; school journeys; field studies; rural studies; urban studies; and outdoor adventure activities. They conclude that “outdoor education in the UK has been wide-ranging, in both content and context, for more than a century [and] many of the current debates about outdoor education are not new (p. 12).

Learning from a review of research on outdoor learning

6

The age ranges included in the study embraced primary students through to undergraduates. Rickinson et al. (2004) define outdoor learning “as learning that accrues or is derived from activities undertaken in outdoor locations beyond the school classroom” (p. 9). Such a broad definition appears to be self-evident, however, this type of learning is further described as occurring in three key areas: fieldwork; outdoor adventure education; and, school ground or local community based projectsiii. Some parameters, reflecting the interests of the initiating bodies, were imposed on the way in which outdoor learning was defined. The researchers have acknowledged that:

in response to the project funders’ core interests and the practical need to define limits for the project, the research team did not look at research on learning beyond the classroom in indoor settings . . . For similar reasons, we also excluded research on general school sport and physical education except that involving outdoor adventure activities, and work looking at virtual field trips except where these had been investigated alongside actual field trips. (Rickinson et al., 2004, p. 9, [original emphasis])

The value assumptions inherent in these boundaries serve to legitimise particular types of learning experiences. For example, by including outdoor adventure education but not outdoor physical education the authors (and hence the initiators) may be seen to be privileging one and diminishing the other. Such distinctions are problematic in the light of Cherryholmes’ (1993, 1999) claim that reading to illuminate meanings, and hence consequences, requires that we abandon foundationalist and essentialist approaches in order to see beyond insular interpretations. Coverage and treatment of the relevant research

It is difficult and probably unusual for a research review to include all of the research available and relevant to its focus. Reviewers must make judgements and apply some sort of selectivity criteria to research findings. Foster and Hammersley (1998) argue that such criteria might include relevance to the field (based on the researcher’s or other definitions of the field), currency, methodological questions, and whether research has been published or not. Prior (2003) maintains that it is ideal that the reason for including and excluding cases is defined in advance of any study. Good reviews ensure that selection criteria (and their limitations) are clearly specified, along with issues of inclusion and exclusion (Foster & Hammersley, 1998; Popkewitz, 1999; Prior, 2003). According to Evans and Benefield (2001), such details go directly to the “credibility that can be placed in the review’s conclusions” (p. 534). Similarly, Popkewitz (1999) calls for researchers to be willing to address the issue of who benefits and who is excluded through their enactment of the research review. Popkewitz (1999) also draws attention to the ways in which a field may be so narrowly defined as to limit the material available to the researcher or, alternatively, so broadly defined that setting limitations becomes an issue. In the case of outdoor learning, I have previously highlighted the diversity of views about its nature, however Rickinson et al. (2004) are explicit about their views of outdoor learning, defining it as “learning that accrues or is derived from activities undertaken in outdoor locations beyond the school classroom” (p. 9) and consisting of the three areas described earlier.

Rickinson et al. (2004) provide a limited discussion of the inclusion and exclusion criteria, the search methods, and the processes they employed to analyse the selected studies. According to Rickinson et al. (2004) “the scope of this review was determined by a series of search parameters decided through discussions with FSC and its partners at the start of the project” (p. 18). This scope is limited to empirical research (published in English only) on

Kathleen Pleasants

7

outdoor learning, including outdoor adventure education, fieldwork/educational visits and school grounds/community projects, and published between 1993-2003. Research literature was identified by Rickinson et al. (2004) via a range of methods including: bibliographic database searches; hand searches of relevant reviews, bibliographies and key research journals; online searches of outdoor learning related websites; email requests to researchers working in the area via networks and organisations. According to Rickinson et al. (2004), the selection of studies for inclusion in the review from amongst the list generated in this way was then based on whether or not it included “a clear research/evaluation dimension . . . and whether the focus was in line with the parameters of the review” (p. 18). Subsequently, 150 publications were selected for reviewiv. The selected research was categorised according to its focus in relation to the three areas of outdoor learning outlined earlier. Rickinson et al. (2004) sought to achieve “commonality and comprehensiveness in the review process [by ensuring that] all publications were reviewed using a common framework” (p. 19)v, the objective of which was to generate information for use in cataloguing and reporting, evaluation, and evidence based analysis. Rickinson et al. (2004) contend that “an important part of the review process was critical analysis of the available evidence, both in terms of the validity or trustworthiness of individual studies’ findings, and the strengths and weaknesses of the evidence base as a whole” (p. 19). They identify two ways in which they sought to achieve this analysis and I will consider each in turn whilst also highlighting some limitations of their approach. Of research paradigms and methodological strengths and weaknesses

According to Rickinson et al. (2004), the first stage of the analysis lies in “recognising the importance of different research paradigms” (p. 19) and they maintain that this involves applying a research paradigm appropriate to the research in question:

For example, quantitative . . . programme evaluations were considered in terms of positivistic research traditions, while qualitative case studies were examined from the perspective of interpretivistic and/or socially critical inquiry. The concern was to examine how well the researchers had carried out what they had intended according to the paradigm in which they were operating. (p. 19)

Rickinson et al. (2004) attempt to make a distinction between more conclusive and less conclusive evidence by identifying methodological strengths and weaknesses in individual studies via a process of distinguishing between:

Findings based on empirical evidence and those based on anecdotal reflection or unjustified prior assumptions; claims based on empirical findings; statistically significant results and those based on description of trends; and survey findings based on very small samples and those based on larger representative samples. (Rickinson et al., 2004, p. 19)

Foster and Hammersley (1998) draw attention to the role of methodological criteria in the selection and exclusion of studies for a review, maintaining that “the comparative significance of methodological defects is not easy to determine [and] is made even more difficult by current disagreements about methodological matters, and specifically about criteria of assessment among social and educational researchers” (p. 619). Although I recognize a need to make distinctions between studies, I am mindful of Hammersley’s (2001) caution that the likely validity of a study’s findings rests on far more than its research design. He maintains that gauging the validity of studies’ findings is a matter of contextually-sensitive judgement and that the amount and nature of the evidence required

Learning from a review of research on outdoor learning

8

is variable and based on the character of the knowledge claim made. Neglecting the contextual nature of such judgements raises the possibility of overestimating the validity of some studies and disadvantaging others that could potentially offer an additional dimension to a review. For example, in evaluating systematic models for educational reviews of research Evans and Benefield (2001) suggest that:

the inclusion of studies which had an explicit research component . . . led to a bias towards studies carried out in the USA, where the tradition of experimental social research is more long-standing, and towards studies carried out within a psychological rather than a sociological perspective. (p. 538)

In the same way, I question whether Rickinson et al’s. (2004) explicit focus on empirical research is problematic in terms of the possibilities it raises for the exclusion of conceptual and descriptive research among other types of research and other voices. Hammersley (2001) notes the potential hierarchy of design that can result when studies are selected for inclusion in a review on the basis of “the likely validity of the results they produce” (p. 544) and criteria are so rigid that they lead to “minimal reliance on judgement or discretion by the reviewer” (p. 544). The inherent positivism of Rickinson et al.’s (2004) selection criteria is typical of systematic reviews that, according to Hammersley (2001), are:

frequently seen as concerned with providing research-based answers to specific questions about what works, or what works best, in relation to some practical problem. They are treated as a bridge between research, on the one hand, and policy-making or practice, on the other. [Whereas] narrative reviews . . . often address large and complex areas involving multiple issues – frequently being designed to provide a map of research in the relevant field. (p. 544)

I suggest that the criteria applied by Rickinson et al. (2004) promotes a fundamentalist approach to reading that is predicated on the premise “that texts always say just what they mean, . . . meaning [is] fixed eternally, outside of time, and that texts in time-bound languages convey that meaning so directly that it can be discerned without interpretation” (Scholes, quoted in Cherryholmes, 1993, p. 2). The tendency to privilege the positivist paradigm apparent in the review of research by Rickinson et al. (2004) is in contrast to a pragmatist approach that seeks to deny foundationalism and the determination of conclusive truth through research. As previously noted, pragmatists look to consequences to clarify meaning. Following Cherryholmes (1993), a pragmatic reading of research on outdoor learning would be one that asked “wide-ranging questions about what research findings mean for their preferred communities and ways of living” (p. 24). In Figure 1 I have diagrammed some of my own readings of Rickinson et al.’s (2004) review of research on outdoor learning. In asking, “how might this review of research inform my practice?” I am concerned with the various, sometimes overlapping, roles I may enact: those of teacher, researcher, instructor and policy-maker. Following Cherryholmes (1992, 1999), who is in turn following Peirce and Dewey, this question highlights a central concern for me as a practitioner that is developed in the questions that follow: what kinds of outdoor learning communities do I wish to promote and what kinds of outdoor learning would such communities value and require?

The actions I might take are not limited to, or by, those described in Figure 1. According to Cherryholmes (1992), if some of the strategies I have outlined work at cross-purposes to my desired community, I may eliminate them or “choose to ignore or actively oppose” those that “work against” my desired consequences (p. 14). Cherryholmes (1992) maintains that, by

Kathleen Pleasants

9

reading pragmatically, I am accepting that “Not everything that works is desirable, not every belief that is ‘true’ is to be acted upon” (p. 14).

Figure 1. One critical feminist reader tracing in her imagination the conceivable practical consequences of a review of research on outdoor learning (Adapted from Cherryholmes (1999).)

Learning from a review of research on outdoor learning

10

Resounding silences

The criteria applied by Rickinson et al. (2004) and, more recently in Australia by Dickson, Gray and Mann (2008), represents a tendency to favour positivist paradigms and may have led to a failure to notice other types of research when searching for literature for their reviews. The limitations of such rigid criteria are reinforced when I consider figures that suggest that the majority of published material in three of the journals Rickinson et al. (2004) searched (the Australian Journal of Outdoor Education, the Journal of Adventure Education and Outdoor Learning, and the Journal of Experiential Education) is based on alternative paradigms according to their current editors. For example, only 10% of the papers published in the Australian Journal of Outdoor Education between 1998 and 2006 are described as positivist research, while 46% are described as either literature reviews (2%), action research (7%), interpretivist (33%) and research methodology reports (4%) (Thomas, Allison, & Potter, 2009).

One resounding silence in the review conducted by Rickinson et al. (2004), also briefly noted by Neill (2006), is the omission of Barrett and Greenaway’s (1995) UK-based review of outdoor education entitled Why adventure? The role and value of outdoor adventure in young people's personal and social development. For Greenaway (Personal communication, December 3, 2007), this omission highlights the issue of how outdoor education researchers can tend not to engage with each other’s work in their research. This judgement may be based on the lack of discussion between researchers occurring in journals such as the Journal of Adventure Education and Outdoor Learning. I suggest that such silences reveal a weakness within the field that contrasts starkly with the views expressed repeatedly in the Review of Educational Research, that research and reviews of research should highlight connections and disconnections, offer alternative perspectives, build on knowledge, critique ideas and theories, and reveal conceivable, and perhaps inconceivable, consequences (see, for example, Cherryholmes, 1993, 1999; Eisenhart, 1998; Grant & Graue, 1999; Lather, 1992, 1999; Livingston, 1999; Popkewitz, 1999; Schwandt, 1998). In Figure 1 I have drawn attention to the way in which my critical feminist reading of this review illuminates possible silences around the issues of gender, learning context and safety, however, there are other areas that may warrant attention in a different study. These include, but are not limited to: What types of outdoor education are portrayed and, hence privileged? Whose versions of outdoor education is represented? What types of relationships with the land are portrayed? Drawing conclusions The degree to which a research review will draw explicit conclusions might vary according to its purpose and intended audience. Whether a review is strongly evaluative in its discussion and recommendations, or whether it is less so, may depend in part on the initial purpose of the research review. According to Foster and Hammersley (1998), a review directed at a practitioner audience may focus on practical conclusions and suggestions, while a review directed at other researchers may be strongly evaluative and include explicit recommendations for policy. Rickinson et al. (2004) explicitly identify practitioners as the target of this review of research, maintaining that the categories of learning activities used within A review of research on outdoor learning would “be recognizable and understandable to practitioners within the outdoor sector, who are the major audience of this review” (p. 16). As noted earlier, the authors were clearly alert to also addressing policy makers and researchers. Chapter 8 of the reviews contains key messages for future practice, policy and research in outdoor learning. I will briefly consider each of these in turn.

Key messages for practice

Kathleen Pleasants

11

According to Rickinson et al. (2004), the review reveals a number of demonstrable outcomes of outdoor learning and hence the findings “provide a source of support and justification for practitioners seeking an evidence base for the area of work in which they operate” (p. 52). The authors place particular emphasis on research reporting positive psychosocial aspects of student experience, such as affective and interpersonal/social outcomes, contending that overall “the review findings give a clear endorsement for certain kinds of outdoor learning provision” (p. 52). Unfortunately they do not explore research that demystifies such claims (see, for example, Brookes, 2003a, 2003b). Rickinson et al. (2004) specifically draw attention to the benefits of programs that clearly articulate links between aims and practices, demonstrate alignment of learning activities and assessments, feature preparation and follow up and provide extended experiences. They recommend that these characteristics be used by those responsible for the planning, development and delivery of outdoor learning, including informing the thinking of school staff seeking to place outdoor learning within the curriculum. Key messages for policy

Rickinson et al. (2004) highlight the benefits of properly planned and executed fieldwork, opportunities for long-lasting personal and social development, an enriched curriculum, and strong community-school links, concluding that policy makers involved with outdoor learning “should be in no doubt that there is a considerable body of empirical research evidence to support and inform their work (p. 54, [original emphasis]). They include the caveat that, despite both encouraging research-based evidence and a tradition of outdoor learning in the UK, policy makers must recognize that opportunities for outdoor learning are less and they charge policy makers with the task of addressing this decline.

Key messages for research Rickinson et al. (2004) acknowledge the quantity and diversity of research conducted within the field of outdoor learning during the time frame they considered, commending what they view as:

• a diversification of research into outdoor adventure education . . . with greater attention being given to questions of learning processes, learning styles and individual learners

• increased empirical enquiry into learning through fieldwork in higher education, often involving action research and theoretical development

• more sustained theoretical exploration of the history and philosophy of outdoor education, and more critical attention to issues of inclusion and access in outdoor learning

• a growing number of meta-analyses and reviews of research, with clear attempts to provide accessible summaries for practitioners involved in work in school grounds, field study centres and outdoor adventure activity centres. (p. 55)

The reviewers also single out areas of research for attention, drawing on Wagner’s (1993) metaphor of blank spots and blind spots to illustrate what they describe as “weaknesses or potential areas for improvement” (p. 56) within the evidence base. Wagner describes what researchers know enough about to question as their blank spots, compared with that which they are not even aware require interrogation. Therefore research that is relevant to the questions already posed can fill in blank spots, while that which results in new questions can illuminate blind spots (Wagner, 1993). By adopting this approach the reviewers are doing what Eisenhart (1998) expresses as “reveal[ing] previously hidden or unexpected

Learning from a review of research on outdoor learning

12

possibilities” (p. 394) and highlighting to researchers what they may have failed to notice about the field of outdoor learning. For me, it follows that blind spots can only then become apparent in hindsight.

Rickinson et al. (2004) identify three areas that they perceive may benefit from illumination through research. These include the “nature of the ‘learning’ in outdoor education . . . the relationship between indoor learning and outdoor learning . . . [and] the historical and political aspects of outdoor education policy and curricula” (p. 57). They conclude the review of research with a call for researchers to consider the extent to which researchers can help to “facilitate closer connections between research and policy, research and practice” (p. 57).

Cherryholmes (1993) has argued that the stories told by researchers can be historical and connected to political agendas: “research findings are often fragmentary, incomplete and dispersed. [They] are subject to multiple readings” (p. 2). Following Cherryholmes (1992), I suspect that a pragmatic reader will reject claims that this review of research is representative of the entire field of outdoor education and look instead to the consequences of the review in terms of the kind of research (or practitioner) community they want to see and the types of research such a community might value and require in a manner similar to that outlined in Figure 1.

Bridging the gap You can't stay in your corner of the Forest waiting for others to come to you. You have to go to them sometimes (Milne & Shepard, 1995, n.p.).

Editors of the Review of Educational Research (RER) have highlighted a concern that teacher practitioners were being excluded from the exchange of ideas within the journal (see Grant & Graue, 1999). This concern is reflected in other contributions to RER and often has as a central theme of the notion of voice or representation (for three examples, see Baker, 1999; Lather, 1999; Livingston, 1999). At the outset, Rickinson et al. (2004) maintain that their review is directed at outdoor practitioners (although they have also acknowledged the potential for various other users to engage with their work). The extent to which they drew on practitioner-led research or to which practitioners have taken up the recommendations suggested by Rickinson et al. (2004) is unclear.

One gauge of engagement may be attempted by considering the number and nature of citations a publication receives although this is clearly not an exhaustive measure or a definitive assessment. I located several citations of A review of research on outdoor learning (Rickinson, et al., 2004). Of these, one was a paper published in School Science Review by the research team and summarising their key findings (Dillon, et al., 2006). This paper itself has subsequently been cited four times, three by the same author writing about the effects of fieldwork on student engagement with science and biology (Prokop, Kubiatko, & Francovicova, 2007; Prokop, Tuncer, & Kvasnicak, 2007; Zoldosova & Prokop, 2006). In each case the paper was used to shore up claims of the effectiveness of fieldwork on students’ learning. The fourth citation was within a doctoral study completed in Sweden (Magntorn, 2007) in which the author again used the paper to support his thesis without any analysis or discussion.

The remaining citations of Rickinson et al. (2004) are noteworthy for the minimal reference that occurs in outdoor education literature. They include papers in science-oriented journals discussing the benefits of fieldwork to subjects such as biology and natural history (see, for example Amos & Reiss, 2006; Bamberger & Tal, 2006; Tal, Bamberger, & Morag, 2005) and publications relating to school grounds projects, the value of botanic gardens or landscaping (see, for example Dyment, 2005; Miller, 2007; Rickinson & Sanders, 2005; Stewart, 2006).

Kathleen Pleasants

13

There are minimal references in outdoor education literature (Fabrizio & Neill, 2005; Higgins & Lugg, 2006). Of the citations I examined, none engage in a discussion of the review’s findings and all use the review to support their claims (usually of the positive benefits of outdoor learning activities). It would seem that these authors perceive this review as merely another brick in the wall of literature (Murray & Raths, 1994) surrounding and supporting their focus, but are not concerned with its structure or suitability for the task. An additional reference to A review of research on outdoor learning (Rickinson, et al., 2004) is located at an outdoor education focused websitevi. Neill (2006) commends the review’s international emphasis and systematic focus, along with its “efforts to draw practical, synthesizing conclusions and recommendations which are related to current policy and social opportunities” (n.p.). However, Neill’s (2006) descriptions read as more of an advertorial than any serious attempt at engaging with the content of the review insofar as it serves its stated purpose or meets its goals.

By way of comparison, Mark Rickinson’s (2001) earlier academic review of learners and learning in environmental education has been cited at least 93 timesvii. It was the focus of some debate within Environmental Educational Research (see, for example Dillon, 2003; Hart, 2003; Rickinson, 2003; Sauve & Berryman, 2003), which suggests to me that the degree to which a review may be taken up by its intended audience is highly contingent on how and where it is disseminated. I suggest that if Rickinson et al. (2004) and, perhaps more particularly, the initiators of this review were sincere in their assertion that “practitioners within the outdoor sector . . . are the major audience of this review” (p. 16) they would be well-served by publishing their findings in a range of locations, especially some of those journals included in their search strategy. However, I am also mindful of Prior’s (2003) contention that those who utilise research reviews (even if it is just to bolster their claims of doing good) are not merely passive participants in the process of communication, but active in the construction and production of knowledge itself. Concluding comments What follows is not a conclusion in the sense of many of the more commonly understood definitions of the word. There is no final result, definitive outcome, or final word on the purpose and role of reviews of research, or on the particular review conducted by Rickinson et al. (2004). The non-conclusion is to remind myself, and others, that there is no final word – as Gough writes (in Semetsky, 2006), “all words are always already sous rature (under erasure)” (p. ix). What I intend to do is to provide a brief comment on some possibilities for future directions in this area. My study served to illuminate my own disposition for enacting a critical reading as one primary means of reading research. By deploying Cherryholmes’ (1993) strategy of reading pragmatically, however, I have become more aware of the many other voices that may be heard and not heard within research writing, and reviews of research. The application of alternative lenses to those that I have described might serve to highlight different readings. For example, what might a deconstructionist reading tell us about the meanings and conceivable consequences of research?

According to Mark Rickinson (2003), “reviews are not just about an understanding of research; they are also about an understanding of practice, and most crucially, about the relationship between research and practice” (p. 268). Perhaps the challenge for practitioners & researchers alike is to engage in what Schwandt (1998) describes as a somewhat anti-modernist conception of education and educational inquiry - “a form of praxis, a kind of morally informed and morally committed action that is guided by ethical criteria immanent within practice itself” (p. 411). In this way “knowledge of educational phenomena cannot be

Learning from a review of research on outdoor learning

14

reduced to a scientific model . . . where our situated selves have no bearing on what constitutes knowledge” (p. 411). References Amos, R., & Reiss, M. (2006). What contribution can residential field courses make to the

education of 11-14 year-olds? School Science Review, 87(321), 1-8. Apple, M. (1999). What counts as legitimate knowledge? The social production and use of

reviews. Review of Educational Research, 69(4), 343-346. Baker, B. (1999). What is Voice? Issues of identity and representation in the framing of

reviews. Review of Educational Research, 69(4), 365-383. Bamberger, Y., & Tal, T. (2006). Learning in a personal context: Levels of choice in a free

choice learning environment in science and natural history museums. Science Education, 91(1), 75-95.

Barker, S., Slingsby, D., & Tilling, S. (2002). Teaching biology outside the classroom: Is it heading for extinction? A report on biology fieldwork in the 14-19 curriculum, Field Studies Council Occasional Publication 72. Shrewsbury: Field Studies Council.

Barrett, J., & Greenaway, R. (1995). Why adventure? The role and value of outdoor adventure in young people's personal and social development. Coventry, UK: Foundation for Outdoor Adventure.

Board of Studies (2000). VCE Study Design: Outdoor and Environmental Studies. Melbourne: Board of Studies.

Brookes, A. (2002). Lost in the Australian bush: Outdoor education as curriculum. Journal of Curriculum Studies, 34(4), 405-425.

Brookes, A. (2003a). A critique of neo-Hahnian outdoor education theory. Part one: Challenges to the concept of 'character building'. Journal of Adventure Education and Outdoor Learning, 3(1), 49-62.

Brookes, A. (2003b). A critique of neo-Hahnian outdoor education theory. Part two: 'the fundamental attribution error' in contemporary outdoor education discourse. Journal of Adventure Education and Outdoor Learning, 3(2), 119-132.

Brookes, A. (2004). Astride a long dead horse: Mainstream outdoor education theory and the central curriculum problem. Australian Journal of Outdoor Education, 8(2), 22-33.

Cason, D., & Gillis, H. L. (1994). A meta-analysis of outdoor adventure programming with adolescents. Journal of Experiential Education, 17(1), 40-47.

Cherryholmes, C. H. (1992). Notes on pragmatism and scientific realism. Educational Researcher, 21(6), 13-17.

Cherryholmes, C. H. (1993). Reading research. Journal of Curriculum Studies, 25(1), 1-32. Cherryholmes, C. H. (1999). Reading pragmatism. New York: Teachers College Press. Crompton, J., & Sellar, C. (1981). Do outdoor education experiences contribute to positive

development in the affective domain? Journal of Environmental Education, 12(4), 21-29.

Dickson, T., Gray, T., & Mann, K. (2008). Australian Outdoor Adventure Activity Benefits Catalogue. Canberra: Centre for Tourism Research, University of Canberra.

Dillon, J. (2003). On learners and learning in environmental education: Missing theories, ignored communities. Environmental Education Research, 9(2), 215-226.

Dillon, J., Rickinson, M., Teamey, K., Morris, M., Choi, M. Y., Sanders, D., et al. (2006). The value of outdoor learning: Evidence from research in the UK and elsewhere. School Science Review, 87(320), 107-111.

Dyment, J. (2005). Gaining ground: The power and potential of school ground greening in the Torinto District School Board. Toronto: Evergreen.

Kathleen Pleasants

15

Easther, B. (1982). The nature and scope of outdoor education. Paper presented at the Victorian Outdoor Education Conference.

Eisenhart, M. (1998). On the subject of interpretive reviews. Review of Educational Research, 68(4), 391-399.

Evans, J., & Benefield, P. (2001). Systematic reviews of educational research: Does the medical model fit? British Educational Research Journal, 27(5), 527-541.

Ewert, A. (1987). Research in outdoor adventure: Overview and analysis. Bradford Papers Annual, 2, 15-28.

Fabrizio, S., & Neill, J. (2005). Cultural adaptation in outdoor programming. Australian Journal of Outdoor Education, 9(2), 44-56.

Field Studies Council (2007). A Review of Research on Outdoor Learning Retrieved 13th November, 2007, from http://www.field-studies-council.org/reports/nfer/index.aspx

Ford, P. M. (1981). Principles and practices of outdoor/environmental education. New York: John Wiley and Sons.

Foster, P., & Hammersley, M. (1998). A review of reviews: Structure and function in reviews of educational research. British Educational Research Journal, 24(5), 609-628.

Grant, C., & Graue, E. (1999). (Re)viewing a review: A case history of the Review of Educational Research. Review of Educational Research, 69(4), 384-396.

Harris, I. (1999). Outdoor Education in secondary schools: What future? Horizons(4), 5-8. Hart, P. (2003). Reflections on reviewing educational research: (Re)searching for value in

environmental education. Environmental Education Research, 9(2), 241-256. Hattie, J., Marsh, H. W., Neill, J. T., & Richards, G. E. (1997). Adventure education and

Outward Bound: Out-of-class experiences that make a lasting difference. Review of Educational Research, 67(1), 43-87.

Higgins, P., & Lugg, A. (2006). The pedagogy of people, place and activity: Outdoor education at Moray House School of Education, the University of Edinburgh. In B. Humberstone & H. Brown (Eds.), Shaping the outdoor profession through higher edcuation: Creative diversity in outdoor studies courses in higher education in the UK (pp. 157-165). Penrith: Institute for Outdoor Learning.

Lather, P. (1992). Critical frames in educational research: Feminist and post-structural perspectives. Theory into Practice: Qualitative Issues in Educational Research, 31(2), 87-99.

Lather, P. (1999). To be of use: The work of reviewing. Review of Educational Research, 69(1), 2-7.

Livingston, G. (1999). Beyond watching over established ways: A review as recasting the literature, recasting the lived. Review of Educational Research, 69(1), 9-19.

Lugg, A. (2004). Outdoor adventure in Australia: Is it a case of roast for Christmas dinner? Australian Journal of Outdoor Education, 8(1), 4-11.

MacLure, M. (2005). 'Clarity bordering on stupidity': Where's the quality in systematic review? Journal of Education Policy, 20(4), 393-416.

Magntorn, O. (2007). Reading nature. Unpublished Doctorate of Philosophy, Linkoping University, Linkoping.

McRae, K. (Ed.). (1990). Outdoor Environmental Education - Diverse purposes and practices. Sydney: MacMillan Publishing Co.

Miller, D. (2007). The seeds of learning: Young children develop important skills through their gardening activities at a midwestern early education program. Applied Environmental Education & Communication, 6(1), 49-66.

Milne, A. A., & Shepard, E. (1995). Pooh's little instruction book. New York: Dutton Children's Books.

Learning from a review of research on outdoor learning

16

Murray, F., & Raths, J. (1994). Call for manuscripts. Review of Educational Research, 64(2), 197-200.

Neill, J. (2006, May 1 2006). A review of research on outdoor learning Retrieved January 12, 2008, from http://wilderdom.com/research/ReviewResearchOutdoorLearningRickinson2004.html

Nichols, D. (1982). Outdoor educators: The need to become credible. Journal of Environmental Education, 14(1), 1-3.

Pleasants, K. (2008). Tracing consequences: reading a review of research on outdoor learning. La Trobe University, Bendigo.

Popkewitz, T. (1999). Reviewing reviews: RER, research, and the politics of educational knowledge. Review of Educational Research, 69(4), 397-404.

Priest, S. (1988). Outdoor leadership around the world: A matter of semantics. Journal of Adventure Education, 5(2), 23-25.

Prior, L. (2003). Documents in social research. London: Sage. Prokop, P., Kubiatko, M., & Francovicova, J. (2007). Why Do Cocks Crow? Children’s

Concepts About Birds. Research in Science Education, 37(4), 393-405. Prokop, P., Tuncer, G., & Kvasnicak, R. (2007). Short-Term Effects of Field Programme on

Students’ Knowledge and Attitude Toward Biology: a Slovak Experience. Journal of Science Education and Technology, 16(3), 247-255.

Rickinson, M. (2001). Learners and learning in environmental education: a critical review of the evidence. Environmental Education Research, 7(3), 207-320.

Rickinson, M. (2003). Reviewing research evidence in Environmental Education: Some methodological reflections and challenges. Environmental Education Research, 9(2), 257-271.

Rickinson, M., Dillon, J., Teamey, K., Morris, M., Choi, M. Y., Sanders, D., et al. (2004). A review of research on outdoor learning. London, UK: National Foundation for Educational Research and King's College.

Rickinson, M., & Sanders, D. (2005). Secondary school students' participation in school grounds improvement: Emerging findings from a study in England. Canadian Journal of Environmental Education, 10, 256-272.

Sauve, L., & Berryman, T. (2003). Researchers and research in environmental education: A critical review essay on Mark Rickinson's report on learners and learning. Environmental Education Research, 9(2), 167-180.

Schwandt, T. (1998). The interpretive review of educational matters: Is there any other kind? Review of Educational Research, 68(4), 409-412.

Semetsky, I. (2006). Deleuze, education and becoming. Rotterdam: Sense Publishers. Stewart, K. (2006). Using technology to support sustained student inquiry in learning

environments beyond the classroom. Paper presented at the The nature of success: Success for nature. The 6th International Congress on Education in Botanic Gardens, University of Oxford Botanic Garden.

Tal, R., Bamberger, Y., & Morag, O. (2005). Guided school visits to natural history museums in Israel: Teachers' roles. Science Education, 89(6), 920-935.

Thomas, G., Allison, P., & Potter, T. (2009). A tale of three journals: A study of papers published in AJOE, JAEOL and JEE between 1998 and 2007. Australian Journal of Outdoor Education, 13(1).

Wagner, J. (1993). Ignorance in educational research: Or how can you not know that? Educational Researcher, 22(5), 15-23.

Zoldosova, K., & Prokop, P. (2006). Education in the Field Influences Children’s Ideas and Interest toward Science. Journal of Science Education and Technology, 15(3-4), 304-313.

Kathleen Pleasants

17

Notes i My use of the plural here is deliberate. There has been some degree of ongoing debate over what constitutes the ‘field’ of outdoor education, and indeed in many cultures it is a completely artificial construct. ii Rickinson et al.’s (2004) review is externally generated insofar as the Field Studies Council (FSC) is an independent charity (see http://www.field-studies-council.org). Three other partner organizations involved in soliciting the review, Groundwork UK, the Royal Society for the Protection of Birds (RSPB) and the Wildfowl and Wetlands Trust (WWT), are also registered charities. The English Outdoor Council (EOC) acts as an umbrella and advocacy body for members of the outdoor education, training and recreation field, while the British Ecological Society (BES) is a professional organization. Other than the Department for Education and Skills (DfES), SkillsActive was the only statutory authority involved in funding this review. The new Department for Children, Schools and Families (DCSF) integrates all children’s services, and consequently has subsumed the DfES (Department for Children Schools and Families, 2007). iii Fieldwork and outdoor visits focussed on tasks linked with particular subjects and undertaken in outdoor settings; Outdoor adventure activities occurring in settings removed from students’ everyday environments, and often with aims associated with personal development; and, School grounds and local community based projects located in or near to the school and spanning a range of curricular and/or extra-curricular objectives. iv Although they included a range of databases in their search, Rickinson et al. (2004) did not discuss the suitability, or otherwise, of those they chose. Evans and Benefield (2001) see the choice of databases as a potentially problematic issue given the lack of development within educational databases when compared with those found in other fields, such as medical research. For instance, one of the most comprehensive educational databases is ERIC, the American Educational Resources Information Centre (there is also an international version of ERIC which includes the UK in its coverage), however, as Evans and Benefield (2001) caution, ERIC does not include reporting of Australian and Canadian education. Rickinson et al. (2004) did, however, also search the Australian Education Index (AEI), the British Education Index (BEI) and the Canadian Business and Current Affairs (CBCA) databases. v For this framework see Rickinson et al., 2004, p. 64 vi http://wilderdom.com vii See http://scholar.google.com.au/scholar?q+learners+and+learning+in+environmental+ education&hl=en&lr=&btnG=Search