Evaluation of Science in Your World

download Evaluation of Science in Your World

of 79

Transcript of Evaluation of Science in Your World

  • 8/4/2019 Evaluation of Science in Your World

    1/79

    1

    Evaluation of Science in

    Your World

    A report by Consulting Inplace

    February 2011

  • 8/4/2019 Evaluation of Science in Your World

    2/79

    Evaluation of Science in Your World

    2

    Contents

    1. Executive summary 22. Introduction 83. Overview of the programme 104. Evaluation of the successes/progress of the networks 185. The impact and outcomes of the programme 376. The extent to which the programme met its objectives 487. Conclusions and recommendations 54

    Appendix I: Logic Model

    Appendix II: Online survey analysis

    Appendix III: Activity observations

    Appendix IV: Research tools and questionnaires

    Appendix IV Full list of documents reviewed

    1. Executive summary

    1.1 Introduction

    In 2010, Consulting Inplace was commissioned by the Museums Libraries and

    Archives Council (MLA) to evaluate the MLA managed strand of the Science in

    Your World programme, part of the Department for Culture, Media and Sport

    (DCMS) and Department for Education (DfE) funded Strategic Commissioning

    Programme. Science in Your World forms part of a national partnership between

    MLA, the Natural History Museum (NHM) and the Science Museum. The MLA

    managed programme supports museums to deliver science learning opportunities

    through a system of regional networks. This report summarises the evaluations

    findings, conclusions and recommendations.

    1.2 About Science in Your World

  • 8/4/2019 Evaluation of Science in Your World

    3/79

    Evaluation of Science in Your World

    3

    The two year MLA Science in Your World Programme started in 2009. It had the

    following aims:

    To deliver new science learning opportunities in English museums

    To build skills and capacity in museum educators and curators to develop and

    deliver science learning opportunities

    To share knowledge about science education and success between museums

    To build demand in schools for science learning opportunities in museums

    To demonstrate that museums and out of classroom learning have a major role

    in delivering the Science Technology Engineering Maths (STEM) agenda

    To maximise the advocacy benefits for the partners and the museums sector

    from delivering science learning opportunities

    These aims are delivered through a system of regional networks that cover the

    South East (SLIME), West Midlands (SMELT), North West (SMILE), North East

    (STEAM) and Yorkshire and Humber (YES-NET). These networks of museums

    share knowledge, ideas and experiences through online forums, meetings and

    events. They also deliver capacity building activities such as continuous

    professional development training, and work together to create and share

    resources such as workshop planning. Alongside the network development,

    the programme provided small grants and regional grants that encouraged

    museums to develop their own science learning opportunities, and make

    connections with local science professionals, teachers and schools.

    1.3 About the study

    Consulting Inplace was commissioned by MLA to conduct this evaluation, which

    included developing a logic model, stakeholder interviews, an online survey with

    museums and activity observations which included teachers, young people and

    science professionals.

    The aim of this research was to complete a robust and independent evaluation of

    the Science in Your World programme that measured the impact of MLA support

    of the regional networks, and the impact of the regional networks on the

    programmes outcomes.

    1.4 Conclusions

  • 8/4/2019 Evaluation of Science in Your World

    4/79

    Evaluation of Science in Your World

    4

    This evaluation has identified strong outcomes for museums, educators and young

    people as a result of Science in Your World, and provides an indication of the

    future impact of the programme. The conclusions based on these findings include:

    Demand from museums has been demonstrated: The offer by regional

    networks of opportunities to develop STEM (Science, Technology,

    Engineering, Mathematics) learning has been welcomed by local and regional

    museums. Furthermore, there is a need expressed by museums for ongoing

    support of this kind.

    The regional networks offer varying levels of support: The size and

    success of support appears to grow with the length of time the network

    operates, as the membership lists of the older SLIME and STEAM networks

    demonstrate, although SMILE and YES-NET in particular are growing quickly.

    The support that has been particularly valued includes: small grants, preferablylinked to partnership with other local museums and schools; facilitation of face

    to face peer to peer support, delivered through events; and facilitation of

    relationships with other relevant professionals, including teachers and

    scientists.

    Networks have built capacity in museum staff: Science in Your World

    funded regional networks have built capacity in museum staff to deliver science

    learning opportunities. This capacity was developed through increasing

    confidence and the skills of museum staff, and providing resources and

    networking opportunities with peers and other professionals, such as teachers.

    Teacher-museum partnerships have been developed: Many one-on-one

    teacher-museum partnerships have been developed. These relationships have

    changed teachers opinions on using museums for learning, and museum

    staffs ability to offer appropriate learning opportunities.

    Partnerships with STEM professionals have been developed: Other STEM

    professionals, such as scientists, engineers, university students and staff and

    bodies such as universities, aquariums, STEMNET (STEM network) have been

    involved in the project. Museums, other professionals, and schools work

    extremely well in partnership, as they bring artefacts, real world STEM

    knowledge and teaching and curriculum expertise together to produce higher

    quality opportunities for pupils.

    More and better STEM opportunities for young people have been

    developed: Over 3,800 young people and 500 family members have

    participated in programmes developed through Science in Your World funded

    opportunities, over 450 teaching staff, and over 350 museum and archive staff

    have participated in CPD programmes, learning links placements or other

    Science in Your World funded opportunities

  • 8/4/2019 Evaluation of Science in Your World

    5/79

    Evaluation of Science in Your World

    5

    1.5 Recommendations

    The recommendations emerging from this evaluation are:

    Regional networks should be supported to continue where possible, and

    where there is demand: There is demand for the regional networks, and they

    are an appropriate mechanism for delivering support to the sector. The

    outcomes and impacts outlined in this report also support the governments

    STEM skills commitments, and support schools to deliver on the Department

    for Educations plans for science and maths as core to the curriculum,

    delivered through more autonomous schools. All co-ordinators and national

    partners are exploring alternative methods of funding for the region, including

    sharing administrative tasks between larger regional and national museums,

    working with local authority Museum Development Officers, and merging withother networks.

    In some regions, sub-regional events and knowledge sharing may be

    more appropriate: For example, the North West. It may be more appropriate

    to have smaller, sub-regional events and sub-networks. This will enable

    museums to easily travel to events and training. There are resource

    implications with this approach, along with less ability to deliver shared

    outcomes. One solution may be to target events at sub-regions, but keep a

    wider regional network as an umbrella. However, there is value in wider

    knowledge sharing regionally, and nationally.

    A national online resource bank would be useful: Each regional network

    has produced or shared resources through the internet. A national online

    resource bank would maximise their accessibility, and ensure that they are

    available beyond the funding period.

    A national means of sharing information and learning should be

    developed: A national means to share information and learning amongst

    peers in the museum sector would give the existing regional and local work a

    focus. This could take the form of an annual conference, newsletter or forum.

    Given the DfEs focus on innovative delivery in more autonomous schools and

    the governments commitment to STEM skills, national information sources,

    such as this recommendation and the national online resource bank, may be inline with departmental priorities.

    Cross-sector links with schools, universities or science professionals

    should be encouraged: One of the clear outcomes of the current funding is

    the bringing together of professionals from related fields. This has considerable

    benefits in terms of improving practice and outcomes for young people. There

    is a danger that without this small seed money, these links will not continue to

  • 8/4/2019 Evaluation of Science in Your World

    6/79

    Evaluation of Science in Your World

    6

    grow. Future funding is recommended to continue this work. This also supports

    the Governments commitment to STEM skills policy through supporting all

    professionals with an interest in this area to work together for more and better

    outcomes. Similarly, there is scope for further collaboration with national

    organisations who also aim to increase science engagement, including but not

    limited to the British Science Association, National STEM centre, National

    Science Learning Centre, Association of Science and Discovery Centres

    Further research is needed to track young peoples outcomes for

    programmes such as this: Further research is needed to gain an

    understanding of the long term impact of programmes such as this on young

    people. University partnerships could be used in future to support further

    research into the impacts of the programme.

    Increased advocacy for science education in museums to build upon thegood individual relationships: There is a need to build upon the good work

    done by the programme in developing one on one relationships with the

    education sector through a widespread awareness raising campaign. This

    would enable the benefits of this approach to learning to go beyond Museum

    Champions into the mainstream.

    Relationships between national and regional museums should be

    continued and strengthened: There is a widespread recognition among

    partners and stakeholders that national museums have an important role to

    play in science education in museums. This should continue, and be

    strengthened where possible.

    Partnership structure for further collaborative programmes, for

    developing external funding bids: The network has laid the groundwork for

    museums to collaborate on further projects or funding bids. For example, the

    regional project led by Museum of Science and Industry (MOSI) is a good

    example of a project that has a broad enough impact that it could attract the

    support that one museum could not alone.

    Resources and learning to be distributed beyond museums: Schools are

    currently reacting to recent changes in the Department for Education, and are

    integrating new guidelines for learning as part of the National Curriculum

    Review. This programme can and should support schools by highlighting

    innovative teaching and learning outside the classroom experiences. It can

    also be used to influence how schools teach science by highlighting the

    successes of the programme. The resources created by the programme should

    be accessible to a wider audience, including teachers, perhaps through an

    existing mechanism such as the Times Educational Supplement website that

    can already be reached.

  • 8/4/2019 Evaluation of Science in Your World

    7/79

    Evaluation of Science in Your World

    7

    Regional and local partnerships should be used to support schools

    develop new curricula and programming: Government policy for more

    control to be devolved to schools and academies suggests that there is an

    opportunity for museums to deliver high quality targeted learning opportunities

    to schools. This programme has left a body of experience, case studies and

    resources that should be used to support schools to develop new curricula to

    support high quality out of school learning.

  • 8/4/2019 Evaluation of Science in Your World

    8/79

    Evaluation of Science in Your World

    8

    2. Introduction

    2.1 Overview

    This report summarises the findings of the evaluation of the MLA managed

    Science in Your World Programme, which was conducted by Consulting Inplace.

    The Science in Your World programme was part of the Department for Culture,

    Media and Sport (DCMS) and Department for Education (DfE) funded Strategic

    Commissioning Programme. It funded regional networks that support museums to

    offer science educational opportunities.

    This report will summarise the findings, conclusions and recommendations of the

    evaluation. The chapters focus on an overview of the programme, an overview of

    the regional networks, a review of the performance of the programme against its

    own objectives, a summary of its impact and outcomes, and conclusions andrecommendations arising from the research.

    2.2 Aims and objectives of the evaluation

    The aim of this research was to complete a robust and independent evaluation of

    the Science in Your World programme that measures the impact of MLA support

    of the regional networks, and the impact of the regional networks on the

    programmes outcomes. We have found out to what extent the MLA-supported

    networks are achieving their aims, and how this was achieved in each region.

    The evaluation has the following specific objectives:

    Develop an evaluation framework providing a logical structure for determining

    the success of the Science in Your World programme

    Evaluate the successes/ progress of the networks in each region

    Demonstrate the impact and outcomes of the programme

    Assess the extent to which the programme has met its objectives

    2.3 Evaluation methodology

    Two of the four phases of this evaluation were concerned with data collection. The

    methodology of these data collection phases were designed to meet the objectivesabove. We have summarised the methodology in this section.

    2.3.1 Data review and Logic Model development

    Data review: We undertook a desk-based review of documents relevant to the

    Science in Your World programme. This included a review of relevant policies and

    strategies, including MLA policy briefings; programme documentation such as the

  • 8/4/2019 Evaluation of Science in Your World

    9/79

    Evaluation of Science in Your World

    9

    application forms for grant funding, MLA end of project reports; and quantitative

    data such as data capture reports . We also spoke to some stakeholders to build

    our understanding of the programme. These included regional network co-

    ordinators and national partners. This informed our evaluation framework, through

    scoping the size and type of network activity.

    Logic Model development: We developed a logic model and corresponding draft

    research tools for the fieldwork phase of this assignment, eg interview topic

    guides, questionnaires, satisfaction forms etc. These tools are included in

    Appendix V.

    2.3.2 Evaluation fieldwork

    Management review: We conducted a management review of the programme.

    This involved speaking to relevant individuals and reading through all thenecessary files near the start and at the end of the evaluation. The monitoring

    information for the programmes was reviewed, and collated into the inputs,

    outputs, and any outcomes that were noted at the time. This process has fed into

    the evaluations of the regional networks, and informed the outcomes and impacts

    measured by the evaluation.

    Network observation: We observed the activities of the networks supported by the

    programme; in total we observed five networks during the evaluation period. This

    involved signing up to the internet groups of the networks and regularly monitoring

    the outputs of the programme. We aimed to understand how the regional co-

    ordinator used the online networks, how the members interacted, and the type andfrequency of communication, to inform our evaluation of the different regional

    networks.

    Activity review: We examined five activities run by the networks in more detail. We

    attended events and gathered information from the project leaders, and also from

    participants, including school children when possible. This enabled us to collect

    more in depth qualitative evidence that could demonstrate some of the softer

    outcomes of the programme. The findings are summarised in Appendix III.

    Online survey: We created an online survey using SurveyMonkey software and

    distributed it through the networks via e-mail. This survey aimed to capture theviews of museum staff that worked with the Regional Networks on how they use

    their networks, alongside any effects it may have on them. We used this tool to

    understand how the regional museums used the networks, and what impacts the

    network activity was having on them. The findings are summarised in Appendix II.

  • 8/4/2019 Evaluation of Science in Your World

    10/79

    Evaluation of Science in Your World

    10

    3. Overview of the programme

    3.1 Logic Model

    The first task for the evaluation was to create a logic model for the programme.

    The logic model is a systematic and visual way to present and share the

    understanding of a programmes outcomes, activities/processes and the

    theoretical assumptions/principles that underpin it.

    We created this model drawing on evidence from programme documentation and

    stakeholder consultations. The Science in Your World logic model can be found at

    Appendix I.

    3.2 Rationale

    A Programmes rationale defines the fundamental reasons behind the creation ofthe programme. It includes evidence of demand or need, and the identification of

    the problem the programme was created to solve. The Science in Your World

    Logic Model, which was agreed with the MLA and formulated by Consulting

    Inplace, states the rationale of the programme to be as follows:

    Museums were not fulfilling their potential to make a contribution to science

    education.

    There was a need:

    a. For museums to put school science in a real world context, enriching thecurriculum in and out of school hours. This followed the Out of School Learning

    and Extended Schools agenda.b. To extend offers by outstanding science education museums such as National

    History Museum (NHM) and National Science Museum across the regions by

    developing regional partnerships. There was also a lack of existing engagement

    between museums to share learning etc.

    Research showed:

  • 8/4/2019 Evaluation of Science in Your World

    11/79

    Evaluation of Science in Your World

    11

    Museums lacked confidence in the education arena, training and investment

    Lack of Continuing Professional Development (CPD) for teachers and museum

    educators in Science Learning.1

    This rationale was developed in discussion with stakeholders and based in part on

    research commissioned by the MLA and the NHM that was completed in March

    20082. This research scoped the potential for a national network of museums to

    deliver science learning, building upon a previous programme, Real World

    Science.

    The reports recommendations and outcomes include the following:

    There was a clear demand for a series of regional networks and co-ordinators,

    linked by a national advisory group. This model for delivery would allow the

    programme to respond to regional and local need, while still retaining some

    national coordination. The regional networks would be free to develop in an

    organic way; however, they would work together to achieve national outcomes

    and targets.

    The focus should be on secondary age students, with curriculum-linked activity

    such as practical science workshops, meet the scientist sessions, trails etc.

    This on-site activity would be supplemented by online and outreach activities

    that link museums, schools and students to produce and deliver learning

    experiences. 3

    There was a gap of CPD for museum staff or teachers to engage in natural

    science-based projects. This would require much closer partnership working

    between providers that work in sciences and also museums.

    1Science in Your World Logic Model from the Science in Your World Evaluation Framework,

    Consulting Inplace, November 20102

    Alison Porter, Tim Caulton, Ian Blackwell, Natural Science Network: Draft Report on Consultation

    and Scoping;; 31st March 20083

    It should be noted that the recommendations were combined with stakeholders views, and it was

    decided that in order to allow widest access to the network and to encourage museums who may

    have no experience creating STEM programmes to not put a limit on the audience age or area of

    stem provision.

  • 8/4/2019 Evaluation of Science in Your World

    12/79

    Evaluation of Science in Your World

    12

    The research found a clear demand for high quality science activities among

    teachers and education partners.

    Evaluation is important to ensure consistency and learning into thisprogramme.

    The national co-ordinator has a role working with the Science, Technology,

    Engineering and Mathematics Network (STEMNET), examination bodies,

    producers of textbooks and others nationally, and Science Learning Centres

    regionally to ensure that the profile of museums as science learning

    destinations is recognised and supported,

    This report provided clear evidence of need and demand for Science in Your

    World.

    3.3 Policy summarySince the start of the programme there have been shifts in policy with the change

    in government. The need and demand for the programme is still there, but there

    are new challenges to delivery including the MLA being closed as part of the

    reorganisation of arms length bodies,4 and the transfer of some of its functions to

    the Arts Council5. In addition, changes to and a reduction in funding for the

    museum sector6 and potential changes to the education system, with a focus on

    the core subjects may impact the Learning Outside the Classroom agenda7.

    There are opportunities in this new environment, however. STEM subjects are

    recognised as being vital for the economic recovery and the economic health of

    4Review of arms length bodies by Department of Culture, Media and Sport, MINISTERIAL

    WRITTEN STATEMENT Department for Culture, Media and Sports arms length bodies, The

    Secretary of State for Culture, Olympics, Media and Sport (Rt Hon Jeremy Hunt), Monday 26th July

    2010, (http://www.culture.gov.uk/news/news_stories/7280.aspx, accessed on 10th

    February 2011)5

    Included in an announcement by Ed Vaizey, Minister for Culture on Tuesday 23 November 2010

    (http://www.mla.gov.uk/news_and_views/press_releases/2010/mlafuture_ace, accessed on 10th

    February 2011)6

    For example, the Comprehensive Spending Review outlines cuts to the public sector, sets out

    some of the effects in the Museums Journey, Patrick Steel, 20thOctober 2010,

    (http://www.museumsassociation.org/museums-journal/news/20102010-comprehensive-spending-

    review-museums , access on 10th February 2011)7

    White Paper: The Importance of Teaching, Department for Education, November 2010

  • 8/4/2019 Evaluation of Science in Your World

    13/79

    Evaluation of Science in Your World

    13

    the UK in the future.8 In addition, science and maths remain in the core subjects in

    the National Curriculum and are part of the focus of the current National

    Curriculum Review.9 This emphasis means schools need to find new ways to both

    support science learning and inspire young people to go on to science based

    careers. Finally science learning is important to many large companies, as they

    depend on young people learning the skills to become scientists and engineers 10.

    The regional networks and museums could develop partnerships to leverage

    funding for their activities with these companies.

    3.3.1 Schools and education policy

    There has been a recent shift in schools and education policy, following the

    evolution of the DCSF into the DfE. The priorities for the Department as set out in

    their Business Plan,11 and expanded upon in the White Paper The Importance of

    Teaching include:

    Greater autonomy to schools, through the expansion of the academies

    programme. This may impact on how schools choose to deliver the curriculum,

    removing barriers to science teaching in museums.

    The introduction of the pupil premium for disadvantaged pupils, which will be

    administered by the schools. These are to be used by the schools to improve

    attainment and aspiration, which museums are well placed to support through

    delivery of opportunities such as those promoted by Science in Your World.

    Improvement of the system with support for teachers, which was further

    developed in the White paper The Importance of Teaching. Museums arerecognised in the White paper as having particular value to teachers when

    delivering education to students12.

    8http://www.bis.gov.uk/policies/science/science-and-society/stem-skills (accessed in February 2011)

    9Review of the National Curriculum in England: Remit,

    (http://www.education.gov.uk/schools/teachingandlearning/curriculum/b0073043/remit-for-review-of-

    the-national-curriculum-in-england/ accessed on 10thFebruary 2011)10

    http://www.bis.gov.uk/policies/science/science-and-society/stem-skills11

    Business Plan 2011-2015, Department for Education, November 201012

    White Paper: The Importance of Teaching, Department for Education, November 2010, p 46

  • 8/4/2019 Evaluation of Science in Your World

    14/79

    Evaluation of Science in Your World

    14

    Changes to the National Curriculum are included in the business plans. The

    National Curriculum Review was announced by Michael Gove, Secretary of

    State for Education, on 20 January 2011.13Science and Mathematics are seen

    as core subjects and the teaching of the new programmes of study for English,

    Mathematics, Science and Physical Education becomes statutory in

    September 2013.14This is a key time for the museum sector to influence how

    science and mathematics are taught, and so the evaluation of Science in Your

    World is timely.

    13http://www.education.gov.uk/schools/teachingandlearning/curriculum/a0073149/national-

    curriculum-review-launched (accessed 10thFebruary 2011)14

    Timetable for curriculum review is at

    http://www.education.gov.uk/schools/teachingandlearning/curriculum/a0073092/timetable-for-the-

    national-curriculum-review (accessed 10th February 2011)

  • 8/4/2019 Evaluation of Science in Your World

    15/79

    Evaluation of Science in Your World

    15

    3.3.2 Arts and Culture

    Although currently the cultural policy landscape is in a state of flux, there areindications from the DCMS, as well as current events that affect the museum

    sector:

    A greater focus on exploring new ways of finding funding and delivering

    experiences, learning from the private sector, supported by NESTA (National

    Endowment for Science, Technology and the Arts) and the Arts Council.15

    Science in Your World currently plays a part in disseminating information and

    advice about funding arrangements, and could support this agenda in the

    future

    The continued uncertainty of funding sources as a result of reductions in public

    sector funding16 have a clear impact on the continuation of activity of Science in

    Your World. This includes both the reduction in national strategic funding

    organisations that could support this programme of activity, and the reduction

    in funding for local authorities to support local or regional museums.

    3.3.3 STEM policies

    In 2004, the then DCSF commissioned a mapping review of STEM provision

    across the UK. This uncovered around 470 separate initiatives delivered by a

    myriad of government departments, Higher and Further Education Institutions,

    professional and sector bodies, non-departmental government bodies, localeducation authorities, education business partnerships, charities and foundations.

    These findings demonstrated the vast proliferation of provision at the national,

    regional and local levels, a lack of coordination between providers and as a result

    overlapping provision.

    15http://www.culture.gov.uk/news/news_stories/7745.aspx

    16The Comprehensive Spending Review outlines cuts to the public sector, sets out some of the

    effects in the Museums Journey, Patrick Steel, 20th

    October 2010,

    (http://www.museumsassociation.org/museums-journal/news/20102010-comprehensive-spending-

    review-museums , access on 10th February 2011)

  • 8/4/2019 Evaluation of Science in Your World

    16/79

    Evaluation of Science in Your World

    16

    These findings influenced the Science and Innovation Investment Framework

    2004-2014 and the STEM Programme Report (2006), which tried to rationalise the

    number of programmes and increase coordination between providers. It did so

    through creating a high level STEM Strategy Group to join up STEM across all

    phases of education and make recommendations to Ministers about national

    STEM priorities.

    Consequently, STEMNET (previously SETNET) played a greater role in

    coordinating provision for primary and secondary provision. In particular, activity is

    coordinated through sub-regional brokerage contracts called STEM Points. These

    are awarded to various different types of institutions and aim to improve access,

    provide advice and guidance and organise events. The Higher Education STEM

    programme provides a similar role, alongside funding opportunities, within the HE

    sector.

    More recent initiatives are likely to include a greater focus on outcomes and

    evaluations are currently taking place of these initiatives like STEMNET (which

    has a strong regional focus) and East Midlands Development Agencys STEM

    Programme. These are as yet unpublished or not yet complete, however.

    In terms of future STEM policy, the Science Cluster of Sector Skills Councils is

    undertaking a strategic planning and review of activity to provide a fit between

    STEM skills provision and skill needs. The National Curriculum Review recognises

    the core importance of Science and Mathematics. There is a renewed commitment

    to expanding apprenticeships, being recognised as a major contributor to thenation's STEM skills. In further education, the Learning and Skills Improvement

    Service is working to improve the content and delivery of STEM skills. The

    Government has outlined its support for initiatives such as STEMNET, the Big

    Bang Fair, and HEFCE's strategically important and vulnerable subjects, and

    National HE STEM programmes.17 In addition Lord Browne, in his independent

    review of Higher Education funding and student finance, signals that public

    investment should be targeted on the teaching of priority subjects including

    17Hansard HC vol 513, Part No. 31, c678W (13 July 2010) (Hansard records of House of Commons

    debate)

  • 8/4/2019 Evaluation of Science in Your World

    17/79

    Evaluation of Science in Your World

    17

    science.18Government rhetoric therefore suggests that there will be a renewed

    commitment to STEM skills policy. This emphasises the need for more outcome

    focused, robust evaluation.

    18Lord Browne, October 2010Securing a sustainable future for higher education An independent

    review of higher education funding and student finance

  • 8/4/2019 Evaluation of Science in Your World

    18/79

    Evaluation of Science in Your World

    18

    3.4 Overview of the programme

    Science in Your World was managed by MLA, as part of the DCMS/DCSF funded

    Strategic Commissioning Programme, and is a two year programme that started in

    2009. It has the following aims:

    To deliver new science learning opportunities in English museums

    To build skills and capacity in museum educators and curators to develop and

    deliver science learning opportunities

    To share knowledge about science education and success between museums

    To build demand in schools for science learning opportunities in museums

    To demonstrate that museums and out of classroom learning have a major role

    in delivering the STEM agenda

    To maximise the advocacy benefits for the partners and the museums sector

    from delivering science learning opportunities

    It does this through:

    Developing regional museum networks to share knowledge

    Building capacity through continuing professional development for museum

    staff and volunteers

    Resourcing the development and delivery of science learning opportunities in

    regional museumsThe development of the regional networks and associated activity was part of a

    National Partnership with the NHM and the Science Museum, who managed their

    own strategic commissioning programmes.

    3.4.1 The first year (2009-2010)

    The programme commissioned three regional networks in its first year to deliver

    the Science in Your World objectives: SLIME (Science Links in Museum

    Education) in the South East, STEAM (Science and Technology Around

    Museums) in the North East, and SMELT (Science, Museums & Archives

    Encouraging Learning Together) in the West Midlands. SLIME and STEAM both

    existed before the programme and had a number of members already. SMELT

    was a new network set up for the programme itself. The funding allocated to these

    networks was 15,000 each, in total. They were to recruit members from across

    the science learning providers; deliver networking events; link members through

    online activities; promote and support members to access MLA funded

    opportunities including Science in Your World Learning Links and small grants; to

  • 8/4/2019 Evaluation of Science in Your World

    19/79

    Evaluation of Science in Your World

    19

    support the dissemination of the learning and resources created through these

    strands; and to deliver training.

    The networks supported the delivery of 17 small grants administered by the MLA

    of up to 3,000 each. These were designed to allow museums and archives to

    take on small, discrete projects that would develop new science learning

    opportunities and build partnerships across science learning providers. The total

    value of small grants was 47,340.

    Networks were also offered opportunities brokered through the national

    partnership, including three Talk Science programmes delivered in each region

    from the Science Museum.

    3.4.2 The second year (2010-2011)

    This year refined the delivery model of the networks, as well as expanding across

    the North area. In the North West, Science in Your World has been developed

    using a partnership approach; SMILE (Science in Museums Inspiring Learning and

    Engagement) has been developed and led by the Learning Manager at Bolton

    Museums and Archives, and the regional project developed and led by MOSI in

    Manchester. Leeds Museums and Galleries took on the development of the

    network and regional project for Yorkshire, in partnership with Space Connections

    (YES-NET). This partnership model seeks to utilise the existing network and

    expertise developed by Space Connections across Yorkshire, which already has a

    small number of museum and archive members, as well as a number of additional

    partners across the Science Education field in Yorkshire, using establishedmechanisms and relationships.

    In addition to the previous activities building the capacity of the network and its

    members, all of the networks this year were given funding for a regional project.

    Each region had a different focus. This was designed as both an opportunity for

    the networks to deliver a programme engaging local museums in a regional

    project, focusing on regional need, supported and delivered through the network,

    but also for case studies and resources covering different audiences and areas of

    the STEM curriculum to be developed and shared nationally. This is important

    both as a learning tool and as an opportunity to demonstrate the breadth of

    engagement that museums and archives can contribute to the STEM agenda.

    The MLA project manager worked with the regions to plan programmes across the

    other Strategic Commissioning national workforce development programme

    funding streams, including Initial Teacher Training Grants, the Learning Links

    programme and in one case, the Talk for Writing Programme. This allowed the

    networks to be much more strategic in how they used available funds, as well as

  • 8/4/2019 Evaluation of Science in Your World

    20/79

    Evaluation of Science in Your World

    20

    planning a more coherent approach to the provision for their networks. In addition,

    the Talk Science programme, delivered by the Science Museums training team,

    delivered the Talk Science programme again in part during a SMILE CPD day in

    Bolton, as well as to interested non-network members in the East of England, one

    of the regions targeted for expansion should the programme have run beyond the

    current funding round.

    Total funding available directly to network leads from the Science in Your World

    programme is summarised in the table below.

    Regional

    Project

    Networklead costs

    Learning LinksManagement

    costs

    Learning LinksPlacements

    InitialTeacherTraining

    Total

    SLIME 10,000 2000 2000 6500 20,500

    STEAM 10,000 2000 2000 6500 4000 24,500

    SMELT 10,000 2000 2000 6500 20,500

    SMILE 10,000 2000 2000 6500 20,500

    YES-NET 10,000 2000 2000 6500 20,500

    The Learning Links programme is designed to offer opportunities for museum and

    archive educators, teachers, trainee teachers and other cultural sector and

    children's and young people's professionals to initiate, build and sustain learning

    partnerships through experiencing each other's working environment and practice.

    STEM themed Learning Links placements were offered to network partners

    through the regional leads. The Initial Teacher Training (ITT) grants programme

    builds learning partnerships between museums, archives and organisations

    offering initial teacher training.

    The National Literacy Trust worked in partnership with Renaissance teams, to run

    a series of training events offering the exciting 'Talk for Writing'19approach. This

    range of tools used by schools across England can be used

    effectively with museum collections and archives to support literacy development.

    In the North East, STEAM used the Talk for Writing approach as part of their

    Science in Your World programme.

    19http://www.literacytrust.org.uk/projects_networks/talk_for_writing (Accessed 14

    thFebruary 2011)

  • 8/4/2019 Evaluation of Science in Your World

    21/79

    Evaluation of Science in Your World

    21

  • 8/4/2019 Evaluation of Science in Your World

    22/79

    Evaluation of Science in Your World

    22

    4. Evaluation of the successes/progress of the networks

    4.1 Introduction

    The regional networks are the backbone of this project. Each network was

    encouraged to develop models developed on local need, capacity and experience

    in response to the aims of the Science in Your World programme. They have each

    developed in a different way and are at different stages. This section of the report

    summarises the key features of each network. Each network is listed in turn, in

    order of their establishment.

    4.2 SLIME, (Science Links in Museum Education), South East

    4.2.1 Overview of the network

    SLIME has been running since 2006, and was one of the models for the Science

    in Your World programme20. SLIME exists to support museums to share practice,work together and offer more resources and programmes linked to science.

    It is the largest network in Science in Your World, and it had 78 members in 2009

    when it received support through Science in Your World. As this network receives

    other funding, we are concentrating on the substantial Science in Your World

    funded activities in this evaluation.

    For the second year of the programme the regional project was GreenSLIME, a

    sustainability themed project. It worked with eight museums (selected from SLIME

    members), and three members took up Learning Links funding. Museums had the

    flexibility to decide the nature of their projects but each involved a school and linkto schools with an environmental purpose. Each project helped improve the

    museums own environmental sustainability. Each project produced a case study

    to be shared across the network.

    The network was originally established with three aims:

    To support members to develop science learning from their collections

    20Alison Porter, Tim Caulton, Ian Blackwell, Natural Science Network: Draft Report on Consultation

    and Scoping;; 31st March 2008

  • 8/4/2019 Evaluation of Science in Your World

    23/79

    Evaluation of Science in Your World

    23

    To raise members confidence in developing science learning programmes

    To advocate the use of museums to teachers to support science learning

    4.2.2 Outputs of the network

    2009-2010

    SLIME promoted and supported members access to the small grants and Science

    in Your World Learning Links, produced ten case studies, and collected data for

    the Initial Teacher Education (ITE) providers across the South East21, including

    contact details for those with a remit for science and information about how they

    plan to deliver to students. This was with funding from Science in Your World,

    although they also delivered other activity such as online forums, and CPD events.

    The small grants programme involved four museums, and reached 1,008 children

    and young people, 90 teachers and 140 adults22over the time period of the

    funding.

    2010-2011

    The Regional Network SLIME used Science in Your World funding to hold the

    following:

    In addition to the GreenSLIME project the network has held the Ruth Borthwick

    Award for Excellence in Science Education in Museums in September and also

    held a network meeting for those interested in Medical Matters in Museums.

    A meeting was held on 29 September 2010 to share ideas, discuss a toolkitand plan a final event five museums were able to attend.

    An article has been submitted to a forthcoming publication Sustainable

    Museums: Strategies for the 21st Century

    A final event ran at Brooklands Museum on 3 February 2011. 40 people

    attended.

    21SLIME Data Capture Report, March 2010

    22This included 120 participants of all ages in family groups.

  • 8/4/2019 Evaluation of Science in Your World

    24/79

    Evaluation of Science in Your World

    24

    A workshop session on sustainability and museums was given to the

    Association of Science Educators national event

    They delivered eight small grants23

    , and at the time of writing, these reached atotal of 263 children and young people, 46 museum/archives staff taking part in

    CPD activity as well as seven other environmental organisations.

    4.2.3 Successes of the network

    Small grants in the region were used effectively to work together to

    create new resources: These contributed to SLIMEs aims of giving members

    support to, and raising their confidence in, developing science learning

    programmes, for example:

    o Canterbury City Museum developed a resource that was included in

    the A History of the World Kent launch at the Museum of Canterbury

    on 20 February 2010 which attracted over 1,100 people

    o Six museum trails with science themes were created in Lewes House,

    and seven contacts made with local schools,24which will be used long

    beyond the funding period.

    Promotion and development of cross-sector relationships to share best

    practice: which contributes to all three aims, for example:

    o Powell Cotton Museum involved a creative practitioner at an early

    stage. The museum cited the enthusiasm and willingness to participatewith the students of this practitioner as enhancing student experience25,

    o Fort Nelson/ Royal Armouries used the Education Business

    Partnerships and Extended Schools to market their programme, and

    felt that this helped to endorse and promote their work to a wider

    audience. 26

    23Some are still ongoing at the time of writing.

    24Data capture report, end of project report, and SLIME case study write ups

    25Data capture report, end of project report

    26RA Fort Nelson Science in Your World Summative Evaluation, June 2010, Liz Denton

  • 8/4/2019 Evaluation of Science in Your World

    25/79

    Evaluation of Science in Your World

    25

    Links to national or existing events or schemes: The SLIME network has

    an outward looking approach, with their activities including marketing to non-

    museums, attending and delivering workshops at conferences aimed at

    science educational professionals and engaging with other potential

    stakeholders such as STEMNET. Museums such as Fort Nelson/ Royal

    Armouries cite links to national events such as National Science Week as an

    ideal way to market and attract schools for new science programmes27. This

    linking to existing events helped SLIME to contribute to all three aims, using

    the expertise of non-museum staff and networks to support its members, and

    also making links that could be used to advocate for museum involvement in

    science education.

    Interesting or unusual approaches trialed: SLIME and GreenSLIME have

    used the grants available to them to trial interesting approaches, such as

    student mentoring by secondary pupils in Rochester Guildhall. GreenSLIMEitself, a regional focus on sustainability, is a new approach for the national

    programme.

    Large themed projects increased cross museum learning and

    development: Stakeholder interviews, including national partners suggest that

    using a theme, such as sustainability, for projects held museums to share

    learning, resources and advice between them: A slightly higher scale project

    with less overall partners, can have a longer legacy in those museums,

    whereas small grants funding one off events can have a greater audience, but

    cannot be embedded into the museum, Stakeholder, National Partner. The

    programme has shown that small one off grants and projects can leave alegacy within the institution (such as partnerships and resources) but the

    evidence does suggest greater learning and impact from themed projects.

    4.2.4 Learning for the future

    Networks take time to build: The SLIME network has been established since

    2006, and has developed in that time a large and active membership.

    However, interviews with the regional co-ordinator suggest that this success

    27RA Fort Nelson Science in Your World Summative Evaluation, June 2010, Liz Denton

  • 8/4/2019 Evaluation of Science in Your World

    26/79

    Evaluation of Science in Your World

    26

    did not happen in twelve, or eighteen months, and only now are the longer

    term impacts beginning to become apparent.

    Changing opinions and raising awareness of science learning inmuseums takes time: Although relationships with individual schools and

    teachers have been developed, the regional co-ordinator does not feel that

    there has been a large scale change in the awareness of museums as science

    learning venues.

    Networks need active support: The experience of SLIME suggests that

    networks do not build themselves, but needs a co-ordinator or co-ordinators to

    ensure that they grow.

    4.3 STEAM, (Science and Technology Around Museums), North East

    4.3.1 Overview of the network

    The STEAM network is led by the North East Renaissance and was established

    before the Science in Your World Programme began. It aims to develop:

    confidence in delivering STEM based activities in museums

    increased subject knowledge in science and STEM related activities in the

    North East

    an increased awareness of the opportunities for STEM learning in museums

    across the North East

    The Science in Your World funding was used to develop further partnerships and

    to raise the profile of science in museums across the North East, through network

    events and promoting and supporting access to the Science in Your World small

    grants and Learning Links.

    In the second year of funding STEAM built upon its success, with CPD and

    networking meetings, a presence at the Regional Annual Science Conference and

    continued links with STEMNET, STEMPOINT, Durham Science Learning Centre,

    the BASE, and Northumbria University. It also engaged with teachers and trainee

    teachers to develop their skills in delivering science learning outside the

    classroom, in addition to promoting museums as places that can be utilised in thedelivery of the science curriculum.

    STEAMs regional project focused on developing a resource kit for museums that

    is aimed at secondary school pupils. This kit explores behind-the-scenes science.

    This was supported by a programme of Learning Links placements based on Talk

    for Writing aimed at improving scientific literacy.

  • 8/4/2019 Evaluation of Science in Your World

    27/79

    Evaluation of Science in Your World

    27

    4.3.2 Outputs of the network

    2009-2010

    The regional network delivered a science development day and regional science

    conference. It worked with 50 museums as members, and of those 18 were new

    contacts.

    Nine organisations received small grants funding, and reached children and 1033

    young people, 129 teachers and 223 adults.

    2010-2011STEAM has delivered:

    STEAM developed and finalised a resource kit with enormous potential for longterm work and attracted new network members, including teachers, venues

    and students. At the time of writing this has not gone live, but it will be cited atthe North East museum Hub website: http://www.twmuseums.org.uk/hub/

    A Regional Science Conference

    Five Learning Links using the Talk for Writing approach

    20 year three science Bachelor of Education students acted as science

    curriculum advisors to 20 host venues, and completed a programme of work

    aimed to develop the science offer in these museums

    At the time of writing, the regional project and learning links projects worked with

    584 children and young people and the CPD elements included 148 teachers and

    trainee teachers and 22 museum/archive staff.

    4.3.3 Successes of the network

    Large and varied membership: Inthe first year of fundingSTEAM has

    increased its membership to 50 individuals from a range of organisations

    delivering science learning, including museums, archives and heritage sites.

    The varied membership allows learning to pass between the different

    educators.28 The membership continued to grow, and added further venues,

    teachers and teaching students. 29The ability for museums to learn from

    28STEAM End of Year Report 2009-2010

    29STEAM End of Year Report 2010-2011

  • 8/4/2019 Evaluation of Science in Your World

    28/79

    Evaluation of Science in Your World

    28

    different organisations and individuals, along with the understanding that

    develops when knowledge and learning is shared, contributed to the networks

    aims.

    Inclusion of national partners: STEAM has been able to use national

    partners to enhance its offer, including the Talk for Writing approach in

    partnership with the National Literacy Trust. This has increased opportunities

    for knowledge sharing and developing confidence for the museum members.

    Use of conferences to disseminate resources: STEAM has been closely

    integrated with the NE Renaissance activity, and used this to disseminate more

    widely the resources and learning developed by the network. For example,

    Monkwearmouth Station Museum has developed a toolkit Feel the Force that

    was utilised by other museums and at least once by another region30. This was

    disseminated during the Hub Science Conference in February 2010, and theworkshop attracted 44 participants from science museums and schools.31 The

    Conference in 2011 was also well attended, and included the two universities

    to add further to the opportunities to learn from different organisations.

    4.3.4 Learning for the future

    Benefiting from national support: Links to national partners such as Natural

    History Museum and Science Museum have brought benefits in advocacy,

    networking, and best practice.32There were also benefits in linking to these

    national organisations for CPD events and the annual science conference.

    Integration into other activities in the region: STEAM has been successful

    at operating with other activities and Strategic Commissioning strands, partly

    through its base at the NE Museum Hub. For example, it held a combined

    workshop with the History Learning network to be of benefit to members of

    both, that was a Talk for Writing approach.

    Membership included strategic local organisations that can support

    advocacy: These strategic organisations include the National Draft Centre, the

    Blue Aquarium, STEMNET at Newcastle college, Science City in Newcastle (a

    30Evidence from the Online Survey

    31Feel the Force Case study produced by STEAM

    32Stakeholder interview with STEAM Network Co-ordinator

  • 8/4/2019 Evaluation of Science in Your World

    29/79

    Evaluation of Science in Your World

    29

    partnership between the local authority, the university and business), North

    East Field Officer for Association for Science Education (ASE), universities of

    Newcastle, Durham and Sunderland, and Robert Stevenson Trust. 33These

    links enable museums to advocate their value to science learning beyond the

    sector, particularly importantly to teacher trainees.

    4.4 SMELT, (Science, Museums & Archives Encouraging Learning Together),West Midlands

    4.4.1 Overview of the network

    SMELT was a new regional network for the West Midlands established during the

    first year of Science in Your World and led by the Thinktank museum in

    Birmingham. Its delivery model was based on that of SLIME, with outreach to local

    museums, an online Google group to share news and resources, and CPDmeetings and events. The aims in 2009-2010 were to establish a network that

    would make contact between local museums, develop its ways of working, support

    best practice, and work towards a greater understanding of the needs of the

    museum members.

    The aims of the 2010-2011 programme were to build on the learning from the first

    year, and to use the network and the regional project to meet the needs of network

    members that were identified in year one of the programme. SMELT built upon its

    initial gains through maintaining its online presence, and developing partners to

    support members in developing science learning in relation to their collections.

    33Stakeholder interview with STEAM regional co-ordinator

  • 8/4/2019 Evaluation of Science in Your World

    30/79

    Evaluation of Science in Your World

    30

    4.4.2 Outputs of the network

    2009-2010

    The SMELT network delivered one networking events and one CDP meeting, with

    an attendance of 55 adults.

    SMELT promoted and supported access to the Science in Your World grants and

    Learning Links programme. Four small grants were delivered that reached a total

    of 496 children and young people, 56 teachers and 28 adults.

    2010-2011

    The SMELT Network ran one CPD session and one showcasing event. It delivered

    one regional grant and worked with a range of partners to produce a virtual toolkit.

    It also delivered three learning links grants.

    At the time of writing, SMELT network activity included 31 children and young

    people, five teachers taking place in CPD and 27 museum/archive staff.

    4.4.3 Successes of the network

    Need and demand for network established: The first year established a

    need and demand within the region to support science learning within

    museums and archives. This was developed through meetings and

    consultation with museums that identified a lack of confidence in developing

    and delivering science learning. There was also consultation with teachers and

    schools that determined that they would take advantage of opportunities to

    learn outside the classroom.34 The second year built on this, and although take

    up of 2010-2011 grants has not been to the extent expected, the showcasing

    event did indicate that there was a demand for the network to continue.35

    Grants were successful: The grants contributed to the aims of supporting the

    overall Science in Your World aims:

    o The Herbert delivered a very popular workshop programme Fun with

    Forces, two other activity sessions, developed new relationships in the

    region, and pupils put up a display in the Natural History Museum

    34SMELT End of Year Report 2010-2011, Stakeholder interview

    35SMELT End of Year Report 2010-2011

  • 8/4/2019 Evaluation of Science in Your World

    31/79

    Evaluation of Science in Your World

    31

    o RAF Cosford delivered workshops to a large number of students and

    adults (over 380 pupils), that included outcomes of intergenerational

    contact and learning.

    o The Pen Room engaged with a local school to develop a museum

    detective tour, which museums volunteers identified as being important

    for their future sustainability as a museum.

    Online toolkit developed: SMELT engaged in a partnership approach to

    developing an online toolkit to support museums to deliver science learning.

    The virtual toolkit, https://sites.google.com/site/smeltnetwork is aimed at

    museum professionals from across the country and uses experiences gained

    from all the network grants and other professional organisations to give

    museum staff greater confidence in developing secondary science activities.

    This has contributed to the aim of building on the learning from the first year.

    4.4.4 Learning for the future

    Regional networks may need time to develop their own needs and

    identity: The network has grown through the funding period, however it is not

    yet sustainable. There has also been a lack of take up of the available grants.

    There is no obvious reason for this, however, as the network developed using

    the templates of SLIME and STEAM, it may be that the network needs more

    time to embed and develop its own regional identity.

    Museum Development Officers and sub-regionalnetworks: SMELT have

    developed good links with the Museum Development Officers working in the

    Local Authorities. This offers another perspective, and there is scope to further

    increase this partnership in the future.

    Specialist science learning museums have an important role to play: The

    leadership of the Thinktank museum enabled the membership to access a high

    level of science knowledge and support around delivering science learning in

    museums.

    Google groups is not accessible for all: Local authority based museums

    and members are not able to access Google during work time. As this was the

    basis of the online network, this had ramifications for its suitability. It is noted

    that this is contrary to the experience of STEAM in the North East.

    4.5 SMILE, (Science in Museums Inspiring Learning and Engagement), NorthWest

    4.5.1 Overview of the network

    Started in 2010, SMILE is a North West based network that is led by Bolton

    Museum and Archive Service. It has established an online network group with

  • 8/4/2019 Evaluation of Science in Your World

    32/79

    Evaluation of Science in Your World

    32

    Google Groups; delivered an awareness raising campaign by using existing on-

    line education networks like GEM; planned, advertised and co-ordinated three

    CPD events for museum staff with interests in science learning; worked with the

    MOSI Regional Science Project; promoted & co-ordinated the Science Learning

    Links project and ensured that appropriate case studies are identified and

    collected from the SMILE network.

    The objectives of the network are:

    To increase opportunities to access training to deliver Science Learning in

    museums and archives through CPD events and access to Learning Links

    placements.

    To build partnerships across the Science Learning sector in the North West.

    To share best practice across the region, and to raise the profile of museum,

    library and archive learning.

    The regional project for the network was developed by a partnership led by

    Museum of Science and Industry, in Manchester. The project:

    Engaged students directly through a laboratory science programme that uses

    museum objects, narratives and activities that are learner centred. This

    learning experience fuses the themes of natural history, chemistry and social

    history by engaging Key Stage 3 students with a real life case study which

    looks at science in the 19th century. Students examined the case study of

    William Perkin, the scientist who discovered the first synthetic dye.

    Trains partners how to deliver a practical science workshop that features a

    social history narrative to illuminate collections and highlight the industrial

    history of the Northwest.

    CPD enabled smaller venues to successfully deliver science learning

    opportunities to hard to reach audiences at Key Stage 3.

    It aims are to:

    Enhance scientific skills, literacy and understanding for chemistry units at Key

    Stage 3.

    Interpret natural history collections and textile collections in a manner that is

    relevant to art/design and science today.

    Give students insight into original practice in textile manufacture which was

    imperative to the development of the North West.

  • 8/4/2019 Evaluation of Science in Your World

    33/79

    Evaluation of Science in Your World

    33

    4.5.2 Outputs of the network

    SMILE has delivered three CPD days and one Learning Links placement. They

    used the rest of the available learning links grants to recruit 10 museums for mini

    Learning Links that involved a shorter one day meeting with teachers to develop

    the science learning. Nine of these occurred within the timescale. They have also

    set up a SMILE science online network. Plans have been put in place to make the

    SMILE network sustainable and beneficial to the region beyond March 2011.

    MOSI designed and tested the workshops, including science lab workshop.

    Resource material completed, including detailed social history. It worked with three

    partners, and presented at a SMILE CPD day The workshop resources will be

    made available to all Science in Your World partners.

    The SMILE Network delivered CPD to 50 museum/archive professionals, and the

    MOSI regional project included 113 KS3 students and 17 museum/archive staff

    taking part in CPD activity.

    4.5.3 Successes of the network

    Membership has grown quickly: Despite the late start of the network, it has

    20 members so far all connected to SMILEs Google groups. The first CPD

    event was successful with 18 people attending. It was used to define the

    regional need for the programme and the parameters of the network.36 The

    following events reached similar levels of people, including new members as

    well as repeat attendees.

    CPD days were identified as the most successful area of the project: Inthe End of Year Report the three CPD days were identified as being

    particularly successful, with sharing of ideas, methods of teaching and

    engagement. These events were held in Liverpool event on 28 September

    2010, Bolton on 7 December 2010; and Blackburn on 14 January 2011.

    MOSI overcame setbacks to deliver in a short timescale: There was

    cancellation of a local programme that would have been used to recruit schools

    36SMILE Interim Report 2010-2011; Stakeholder Interview with SMIKE Network Co-ordinator

  • 8/4/2019 Evaluation of Science in Your World

    34/79

  • 8/4/2019 Evaluation of Science in Your World

    35/79

    Evaluation of Science in Your World

    35

    4.6.2 Outputs of the network

    Yes-Net delivered one CPD event, a conference that attracted teachers and

    museums. They produced quarterly e-newsletters. Their regional project is still

    being delivered, and the majority of the activity will occur during half term. This

    worked with four partners to deliver science based activities. There are four

    learning links planned, although not all have been completed at the time of writing.

    At the time of writing, 320 children and young people and 335 family visitors have

    been reached through the regional project, and 21 Teachers and 43 museum/

    archive organisations took part in CPD activity

    4.6.3 Successes of the network

    Hit the ground running: This programme was delivered using an existing

    network of 27 members, so they were able to hit the ground running with the

    outreach and promotion, helping them to meet their targets. Although they did

    not deliver all five learning links planned, they did deliver four, and similarly

    worked with four regional partners and delivered a large and successful CPD

    event in an extremely tight timescale. This network started later than the

    others, and so has had less time to achieve its goals of embedding overall

    Science in Your World aims in the Yorkshire region. This has been

    compounded by the end of the programme being moved forward, making the

    delivery of the programme challenging. However, its approach has meant that

    it has grown rapidly.

    Successful conference with high teacher attendance: The conference held

    in Leeds Museum on 7 February 2011 was extremely successful in attracting awide audience, with over 60 attendees, including teachers and museums. It

    aimed at ways of sharing innovative science learning opportunities in

    museums. A full description can be found in Appendix III.

    4.6.4 Learning for the future

    Piggybacking on existing networks is a good way to build momentum

    for a project quickly: The fact that YES-NET already existed and was

    successful meant that the programme was able to market its activities to a

    wide audience from the start. It could use established channels of

    communication to museums, schools and stakeholders, and could takeadvantage of the high regard of these audiences for the network.

    Practical sessions for innovate teaching: The event was a success,

    especially in its ability to attract teachers. The teachers at the event highlighted

    the desire to learn practical ideas and skills for teaching science, as well as

    keeping abreast with innovative teaching practice.

  • 8/4/2019 Evaluation of Science in Your World

    36/79

    Evaluation of Science in Your World

    36

  • 8/4/2019 Evaluation of Science in Your World

    37/79

    Evaluation of Science in Your World

    37

    5. The impact and outcomes of the programme

    5.1 Outcomes from the programme

    5.1.1 For museum staff

    The programme has delivered impacts for museum staff throughout the country.

    These are summarised here:

    Developed confidence and skills to develop and deliver science

    education in museums: The regional networks have developed confidence in

    their members. This is shown by the Science in Your World evaluation online

    survey of network members, which cites gains in confidence as a clear impact

    on them when attending events: I also gained confidence in using the

    equipment needed to teach secondary science safely. This meant that I now

    feel more confident in teaching secondary school children within the museum

    setting. The activity observations for the SMELT conference, SMILE CPD day

    and YES-NET one day conference also observed gained confidence in

    delivering science learning from all non-science specialist museums we spoke

    to, including history based museums. For example please see the SMILE CPD

    day activity observation in Appendix III. This confidence was developed

    through:

    o Increasing knowledge of science based workshops or programmes for

    example39, the MOSI project created a workshop that was used by

    other museums as part of the project40

    , the conference with YES-NETincluded sharing of workshops and activities that were found to be

    useful by other museums for example, at one demonstration by the

    Science Museum there were museum staff of another museum who

    stated I will steal that idea about demonstrating explosions.

    o links to other museums delivering similar work that could be contacted

    for support, as cited by participants in CPD day for SMILE, for example

  • 8/4/2019 Evaluation of Science in Your World

    38/79

    Evaluation of Science in Your World

    38

    all 18 participants agreed strongly with the statement I have made new

    contacts today and I will contact people I met today following this

    meeting.

    o Links to national and regional science based museums highlighted by

    stakeholder interviews41 and as demonstrated by projects such as

    MOSI42. The participants cited these in the online survey (for example

    This gave me confidence in teaching the lesson dying to change the

    world )and in the activity observation as being confidence raising (At

    SMILE CPD day there was a comment about a presentation by

    Science Museum I feel I can have a go [at delivering science based

    learning], now).

    o Links to teachers, students, university staff, and other professional

    scientists to work with the museum in developing these programmes

    i

    .This was highlighted as being particularly important by participants in

    the SMELT celebration conference, where particularly close

    relationships had been built in projects such as a new workshop

    developed for the Pen Museum by teachers, university students and

    the museum volunteers. In addition small grants that involved

    professionals also developed links such as Canterbury City Museum

    used a creative practitioner and students from Canterbury Christ

    Church University to develop a programme.

    Personal networks between museum educators: These were developed

    throughout the programme, particularly through the events and the regional

    projects, although it is notable that respondents also used the SLIME online

    network for peer to peer activity, and valued it for helping them gain advice

    from other members. This included observation of details being exchanged

    during the activity observation at the CPD day at SMILE, and during the YES-

    NET conference. Stakeholders also suggested: The network was the strongest

    elementwe were able to share different ideas from museums around the

    42Interim Grant Report Form, SLIME, October 2010.

    42Activity Observations with organ

  • 8/4/2019 Evaluation of Science in Your World

    39/79

    Evaluation of Science in Your World

    39

    regionit was particularly useful for those museums without a science offer. It

    improved their confidence and able to work more with science in their venues.

    We know through informal discussion at network meetings, and hard copy

    evaluations.

    Personal networks developed with individual teachers: These were

    developed, particularly through the small grants programme, and the Science

    in Your World Learning Links programme. As one museum stated I know now

    that I can pick up the phone if I have a question about the national curriculum,

    and she can at least point me in the right direction43Evaluations of the small

    grants also support this, with statements about individual teachers being

    supportive, or suggestions of future activities to be done with teachers: Our

    meetings enabled an opportunity to brainstorm ideas and forge new links

    between schools and museums to enhance future learning opportunities44.

    The stakeholder interviews also cited the importance of changing minds one ata time, and suggested that museum champions in schools are the key to

    engagement with the wider institution.

    SMILE CPD day, Bolton

    Building networks between museums and museum educators

    A CPD day was held at Bolton Museum and attracted 18 attendees from

    museums throughout the North West. The day held participative workshops

    showcasing science learning approaches from regional and national

    museums, including a hands-on Talk Science Session led by the Science

    Museum, looking at using objects to engage in contemporary science

    discussion. All participants agreed strongly that: they had made newcontacts and that they would contact people from the CPD day in the future,

    they felt more confident delivering science workshops, and that they had

    learnt something new.

    5.1.2 For museums

    Increased funding opportunities were supported through the programme.

    The largest funding leveraged using the Science in Your World grant for

    43Attendee at SMELT conference 2010

    44Mini Learning Links Report, Bolton Museum

  • 8/4/2019 Evaluation of Science in Your World

    40/79

    Evaluation of Science in Your World

    40

    537,000 for George Marshal Medical Museum. During the second phase of

    the grant making process, the museum lead on this bid used the innovative

    project as evidence of their ability to both attract outside funds, and deliver

    interesting science programming. While it was not the only factor in the

    decision of the Heritage Lottery Fund, the museum lead felt that it did

    significantly strengthen their overall bid45. Other funding sources have been

    disseminated over the internet by all the regional networks, and the online

    survey identified a successful bid for a Public Engagement Grant from the

    Institute of Physics to run an Astronomy Evening for Brownies in conjunction

    with the Mary Rose Museum.

    Networking with other museums was done through staff time, as

    summarised above. We have not collected evidence that this resulted in any

    formal partnerships between museums, beyond those attached to funded

    projects such as the MOSI project, or the YES-NET regional project.

    Links with schools were made by museums, although it was felt by

    stakeholders that these were often between individuals, and concern were

    raised that they were vulnerable to changes of staff at either the museum or

    the school. A stakeholder stated There were many one to one relationships

    developed, although I worry that these may change is the teacher changes

    jobs regarding the SLIME network.

    Development of new ways of working: In some cases there was evidence of

    a more strategic impact on the programming of individual museums that were

    influenced by the programme. This included instances of focusing more

    attention to new audiences, (for example, Activity Observation with MOSI

    suggested that the museum could attract secondary pupils, a new market for

    them), rethinking existing provision (Booth Museum of Natural History

    response to our Online Survey) and working more collaboratively.

    SMELT Showcase event

    Building Links with schools

    The event included presentations from recipients of Science in Your World

    45SMELT End of Project Report 2010-2011

  • 8/4/2019 Evaluation of Science in Your World

    41/79

    Evaluation of Science in Your World

    41

    funding, feedback on the SMELT regional online resource kit and discussion

    about the future of the network. Museum staff discussed the value of the

    funding and the additional opportunities available through the network. One

    museum was struggling to attract schools, but the resource and

    relationships developed will help us with this. This is vital for our

    sustainability and another needed the expertise of [the Special Educational

    Needs (SEN) teacher] to develop appropriate workshops. We are a small

    museum and the smaller class sizes make SEN schools a good market for

    us. Teachers in attendance added valuable insight around the future of the

    network and how it could continue to build partnerships with and demand

    from schools.

    5.1.3 For teachers

    New teaching ideas and skills have been disseminated: Teachers from the

    YES-NET conference came for the practical demonstrations of science

    learning along with a desire to understand what museums could offer their

    students. There was a particular desire for education activities that could be

    used by teachers to teach concepts or aspects of the curriculum that were not

    being understood in the classroom. We spoke to over ten teachers about the

    day during lunch and after the demonstrations, and they were positive about

    the practical ideas they had learned, and would use them in the classroom.

    One Primary School teacher was teaching about the human body and was

    pleased to learn a way of modelling it for young children in a fun way. This was

    corroborated by evidence from case studies and grants paperwork, forexample seeing pupils allowed to investigate and experiment gave me

    confidence to be more creative in my approach.46

    Individual awareness of museums as good venues for science learning

    was raised: Learning Links, small grants, and the events such as YES-NET

    conference in February 2011 were very successful at making the case for

    science learning in museums. This was clear in the end of project reports (for

    example, Powell Cotton Museums small grant included the note that the

    principal of the school that they worked with has become an advocate of the

    46Case Study for Feel the Force in Monkwearmouth produced for SLIME in 2009-2010

  • 8/4/2019 Evaluation of Science in Your World

    42/79

    Evaluation of Science in Your World

    42

    project and is looking to find further links to the museum in other subject areas

    including other science subjects and maths), interviews with teachers during

    activity observations such as the SMELT case study showcase, (I hadnt

    considered using museums to teach science to be honest) and the YES-NET

    conference confirmed that the awareness for science based learning in

    museum was changed as a result of the programme (I thought museums were

    for history) Stakeholder interviews with both regional co-ordinators and

    national partners confirmed the view that individual teachers were becoming

    more aware through the programme: The idea of using teachers to help

    museums to develop science opportunities is a good one, and the individual

    teachers will learn more about using museums in this way.

    Wider awareness of museums as venues for science learning

    opportunities has been less successful: The programme has had a stall at

    the Association of Science Educations Annual Conferences, and SLIME wasmentioned in the Telegraph this year for promoting science learning in

    museums at this event47. Despite these activities, stakeholders all feel that

    there is not a sufficient general level of awareness amongst teachers of the

    value of science learning in museums: One to one relationships have been

    built, but it is usually those teachers that have been involved that value us it

    does not go beyond these one to one relationships . This has been recognised

    as a slow process which requires changing the overall culture in schools, and

    this is the next step for many of the networks: We have not yet scratched the

    surface of the sector.

    YES-NET Conference, Leeds Museum

    New teaching ideas and skills have been disseminated

    This event attracted over 60 attendees, including a high number of teachers

    (mainly primary).. It comprised of talks by museums and educators

    presenting ways of delivering science learning in museums as well as hands

    on workshops demonstrating a range of techniques used to teach science in

    museums. This aspect of the day appealed to teachers, who commented

    that [they] wanted practical ideas to teach science and that they attended

    47http://www.telegraph.co.uk/science/steve-jones/8252195/Carry-on-camping-to-improve-science-education.html

  • 8/4/2019 Evaluation of Science in Your World

    43/79

    Evaluation of Science in Your World

    43

    because [they] are redesigning our curriculum at the moment, and are

    focusing on projects or themes that we can explore. They need to be

    cross-curricular, but we are struggling to include science. Teachers were

    inspired by the sessions and came away with a book of practical ideas.

    During a practical session on using skulls to teach evolution, one teacher

    commented I wish they would come out to our school.

    5.1.4 For schools

    In general outcomes for schools were harder to measure. Teachers, heads of

    department and head teachers developed relationships with museums. We

    collected little evidence that this went beyond individual networking to a more

    systemic and strategic impact. Individual interviews seem to suggest that it is

    necessary for a Museum Champion to be present in a school who will supportteachers to both take children on trips in spite of the expense of transport and loss

    of school based teaching time, and who will also get involved in working with

    museums to design new workshops and programming.

    5.1.5 For young people

    It was difficult for this programme to prove outcomes for children and young

    people in the timescale available. This is partially due to the long term nature of

    some of the outcomes desired (such as changing career paths, or attainment in

    science based projects) and also due to much of the delivery to young people

    falling outside the timescale of the funding. There is also the problem of attributing

    changes in attainment to one project, as there are many different influences onchildrens attainment. However, there have been some identifiable outcomes:

    A high number of young people accessing resources and programming:

    Data capture and initial data from the regional projects shows a high number of

    young people who engaged in the resources and programmes created in the

    period of time during and immediately after the period of the small grants

    programme, and during the period of the regional projects. Over 3800 young

    people have been reached directly at the time of writing, although as some

    projects are ongoing the final output will be greater.

  • 8/4/2019 Evaluation of Science in Your World

    44/79

    Evaluation of Science in Your World

    44

    Some projects used young people to support the design of resources:

    For example, Worcestershire History Centre developed a relationship with an

    engineering club in a local secondary school, which not only fed into this

    project, but are also likely to use the museum for future initiatives48.

    Activity observation showed young people engage well with museum

    based learning; We observed activity at both SEARCH in Gosport and in

    MOSI, and these events showed that the young people were particularly

    engaged with the projects, and comments from teachers at these events

    supported the use of museums to enable science to come alive for the young

    people.

    Other outcomes of the activity observations on young people: The sample

    sizes were small, however, indicatively there were the following outcomes

    gained:o At MOSI: Fun, engagement with science, learning about science and

    history

    o At SEARCH: Fun, practical learning, working as a team.

    These outcomes have been true for many activities: Forexample:

    o Some students were motivated to learn more: Some pupils who

    struggle in the formal classroom setting were motivated in the museum

    setting, Head Teacher - , Primary School49

    o The museum let students see science in a practical, real world

    setting: The students spent lots of time investigating and thinking

    about other variables that they could test and many students could see

    applications to everyday circumstances and It has opened up further

    discussion and students are seeing where they can apply this

    elsewhere and where they can go next with the materials50

    48End of Project Report 2009- 2010, Worcestershire History Centre

    49Segundum End of Project Report 2009-2010

    50Tyne and Wear Museum, End of Project Report 2009-2010

  • 8/4/2019 Evaluation of Science in Your World

    45/79

    Evaluation of Science in Your World

    45

    o Creativity was encouraged: lots of enjoyment and inspiration. Some

    of the pupi