Who the museum evaluations are for : From museum visitors to local societies

and collaborative project partners

Toru Sasaki Hokkaido University

1

2

The museum focuses on the Native Peoples of the North who live in both Eurasia and America.

Hokkaido Museum of Northern Peoples Abashiri City, Hokkaido Prefecture

Source: website of Hokkaido Museum of Northern Peoples

3

Tokyo

Kyoto

Hokkaido Museum of Northern Peoples Abashiri City

Sapporo

4

Permanent Exhibition

Presentation Outline

1. Background 2. Period of museum evaluation adoption 3. Period of reflection 4. Trial period ・Mie Prefectural Museum（MieMu） ・Special exhibition “Mushrooms! Mushrooms! Mushrooms!” Osaka Museum of Natural History ・Regional cooperative project “Singing Insects and the Township” Itami City in Hyogo Prefecture 5. Conclusion and prospects for the future

5

1. Background on the museum evaluation’s adoption in Japan

6

7

Trends in National and Local government

1999

2001 Central Government Reform

2002

1996

1997 Shizuoka Prefecture : the administrative evaluations using business inventory tables

National government

Local government

the Law Concerning Access to Information Held by Administrative Organs

the Government Policy Evaluations Act

M ie Prefecture : the office project evaluation system

8

Tokyo

Kyoto

Mie Pref. Shizuoka Pref.

Sapporo

9

1990 2000 2010 2020

introducing

evaluations

of their

operations

spreading ／

being set in p lace

Adoption of the evaluation of adm inistrative bodies in Japan

| | | |

• The Comprehensive Evaluation Method → an evaluation of programs • The Performance Evaluation Method → an evaluation by measuring result • The Project Evaluation Method → an evaluation with a cost-benefit analysis.

10

Issues on evaluation methods ：Tatsuya Ono (2009)

• Almost all of the local governments use the Performance Measurement(PM).

• The lack of process to assess the cause and effect relationship or logic in PM.

11

Issues on evaluation methods ：Tatsuya Ono (2009)

The method most widely adopted by museums is the Performance Measurement(PM).

12

Museums in Japan and establishers

5690 museums

4489 museums → 79%

Presentation Outline

1. Background 2. Period of museum evaluation adoption 3. Period of reflection 4. Trial period ・Mie Prefectural Museum（MieMu） ・Special exhibition “Mushrooms! Mushrooms! Mushrooms!” Osaka Museum of Natural History ・Regional cooperative project “Singing Insects and the Township” Itami City in Hyogo Prefecture 5. Conclusion and prospects for the future

13

2. Period of museum evaluation adoption

14

15

Office project evaluation system by Tokyo Metropolitan government

This was the first time that a public museum was considered a management organization and was evaluated as a whole.

Achievement rate Efficiency

Performance of a museum Necessity Impartia l i ty

Com prehensive eva luation result（ 5 grades）

16

Tokyo

Kyoto

Shizuoka Prefectural Museum of Art

Sapporo

17 Source: website of Shizuoka Prefectural Museum of Art

18

Exhibition & Workshop

• Began to create an evaluation system in 2001.

• A committee composed of third-party stakeholders and museum’s staff members in 2003

• “Museum Navi”(2005) created over roughly 2 years.

19

Shizuoka Prefectural Museum of Art

32 quantitative indicators 8 qualitative indicators

32 quan titative ind icators

40 ind icators

indicators that use

responses from visitors,

18, 56% indicator that uses

responses from residents, 1, 3%

indicators from data

derived from documents,

13, 41%

20

Evaluation indicators of “Museum Navi”, Shizuoka Prefectural Museum of Art

21

One of 8 qualitative indicators

“Did the museum change the lives of exhibition visitors or event participants.”

22

Tokyo

Kyoto

Sapporo

Shizuoka Prefectural Museum of Ar

Presentation Outline

1. Background 2. Period of museum evaluation adoption 3. Period of reflection 4. Trial period ・Mie Prefectural Museum（MieMu） ・Special exhibition “Mushrooms! Mushrooms! Mushrooms!” Osaka Museum of Natural History ・Regional cooperative project “Singing Insects and the Township” Itami City in Hyogo Prefecture 5. Conclusion and prospects for the future

23

3. Period of reflection on the evaluation of museums

24

25

The state of museum evaluation adoptions

Source: Created by the presenter using data from the Japanese Association of Museums（2009：4-5)

N=1,044

The requests from museums that have not adopted evaluations yet.

• “the introduction of specific examples of evaluations”

• “the provision of training, manuals, or guidelines to help with adopting evaluations.

→Lack of the know-how on adopting evaluations. →Lack of appropriate place to learn about it.

26

The opinions and requests from museums that appear to have already adopted evaluations. • “Most of the indicators are evaluations of measures to achieve

numerical goals but it is difficult to evaluate unseen effects.” • “If we can find a way to measure the economic effect of

museum activities or their cost-effectiveness, we can make evaluation activities even better.”

• “We should shift to a system that evaluates the contents of activities.”

• “The evaluation cannot be said to be analyzed or used enough, and we want the technical aspect to be covered better.”

→Skeptical opinions about the Performance Measurement (PM) →Opinions about improving the way the data was analyzed.

27

28

“The Progress of Activities Related to Evaluation and Current Issues.” : Takai (2012)

?

Reason that evaluations did not spread among museums. Takai(2012) 1) There are few museums that formulate midterm or

long-term project plans.

2) There are no tools available to plan midterm or long- term project plans and it is difficult to arrive at concrete indicators or goals from project plans.

→ Lack of midterm or long-term project plans that are connected to a museum’s mission and its self- evaluation. 29

Presentation Outline

1. Background 2. Period of museum evaluation adoption 3. Period of reflection 4. Trial period ・Mie Prefectural Museum（MieMu） ・Special exhibition “Mushrooms! Mushrooms! Mushrooms!” Osaka Museum of Natural History ・Regional cooperative project “Singing Insects and the Township” Itami City in Hyogo Prefecture 5. Conclusion and prospects for the future

30

4. Trial period for creating new evaluations and evaluation methods

31

Presentation Outline

1. Background 2. Period of museum evaluation adoption 3. Period of reflection 4. Trial period ・Mie Prefectural Museum（MieMu） ・Special exhibition “Mushrooms! Mushrooms! Mushrooms!” Osaka Museum of Natural History ・Regional cooperative project “Singing Insects and the Township” Itami City in Hyogo Prefecture 5. Conclusion and prospects for the future

32

33

（１）Designing an evaluation system that is linked to the deliberation of the project p Mie Prefectural Museum (MieMu) Opened in April 2014

34

Mie Osaka

Itami Kyoto

100km

35

Basic Exhibition

MieMu’s symbol Children’s Hands-on exhibition room

Mie’s real-life encyclopedia Learning Space

38

History of the Prefectural Museum in Mie 1953 Prefectural museum of M ie opend（ ~ 2007）

1986 Concept for a new museum began being deliberated

2008 Basic Plan for a New Prefectural M useum, M useum M ission

April 2011 Reconsideration of all sorts of projects

June 2011 "3 directions” and “7 items"

2012 W orking group to devise an evaluation system

April 2014 New M useum（ M ieM u） opend

MieMu’s mission in 2008

1. Preser