WHICH KPIs ARE NEEDED FOR BENCHMARKING AND MANAGEMENT …

1© Future Museum, 2021 1

WHICH KPIs ARE NEEDED FOR BENCHMARKING AND MANAGEMENT PURPOSES?

secondary research questionby MUSEUM BOOSTER

This report is intended for internal use of the Future Museum participants and contains confidential and other restricted material. Please do not pass on to third parties and do not publish its content.

2© Future Museum, 2021

TABLE OF CONTENTS

+ Why

+ What

+ How

+ Examples

+ Trends

+ Monitoring Tools

+ Best Practices

+ What and Why

+ Tools and Challenges

+ Management

+ Best Practices in Museums

+ Best Practices in Other Sectors

KPIs Future of KPIs Benchmarking + Management

01 02 03INTRO

Pages 74 - 104Pages 36 - 73Pages 5 - 35Page 3


INTRODUCTION

Modern tools allow museums to collect a lot of data and metrics. However, museum managers often struggle to identify the most relevant key performance indicators, which can be used for benchmarking, strategical development of the institution and planning.

Key Performance Indicators (KPIs) are crucial tools contributing not only to measure the performance and processes within organisations, but also helping them to focus and put their efforts on what is critical for their current and future success.

However, the relevance of KPIs lies not in the measurement itself, but in what is done with these figures (e.g. benchmarking analysis, organisational changes, etc.). In this report, we have brought together key insights into the relevance, monitoring, creation, reporting and the use of KPIs. Also, we attempt to cover the most relevant aspects that organisations need to be aware of in order to implement benchmarking and improve management processes.

In preparation for the report, we have asked Future Museum participants to share their challenges in this regard. The group was particularly interested in aspects such as:

KPIs for measuring and improving visitor experience KPIs for sales and visitor acquisitionKPIs monitoring tools

In order to address these aspects, we looked into the topics of KPIs relevance, tools for KPIs creation and monitoring, essential KPIs, and the use of KPIs benchmarking and relevant management processes. We highlighted relevant insights and cases from the museum sector and other sectors as well.


– Jonathan GoodacreSenior Consultant The Audience Agency

When thinking about research and using or collecting data, it is vital this is based on what is actually useful to us. Too often we see people chasing the latest trendy measure or on the other hand resorting to basic demographics. What is useful may mean researching the market, understanding the public or conducting evaluation into our existing programmes. Each has its own needs and only when we have the answers to these questions can we proceed. These are considerations in our own work, whether evaluating large scale projects such as Galway 2020 European Capital of Culture, developing and managing Audience Finder or helping advise a small-scale community arts project.

Nevertheless, there is much to be said for the consistency that good benchmarking can bring. In the UK, we have established the protocol of using demographic categories which match the Census. This ensures we are all using a standard benchmark as well as ensuring that it can be matched to comparable secondary data. It’s simple but important. Similarly, there is much to be said for the ongoing standardised research coming from the Target Group Index in the UK or the Eurobarometer at a European level. Understanding the different types of identity that people might have with a city, region, nation, or Europe (for example) is not easy and it helps that a question is consistently and regularly asked over several years and territories to provide useful starting points for our research.

“

”


KPIs

+ Why

+ What

+ How

+ Examples 01


WHAT TO EXPECT IN THIS CHAPTER

KPIs are common and relevant tools in order to monitor and improve processes and performance of organisations. However, while using KPIs, some concerns and questions arise: Are organisations, and museums in particular, using KPIs as expected? Are they benefiting from the true advantage they can provide? Do they know the benefits of linking strategy, goals, KPIs and benchmarking? In this context, answering the research question “Which KPIs are needed for benchmarking and management purposes?” with a list of metrics is not an actual answer. The answer varies depending on the type of the organisation, its needs and goals.

This chapter aims at providing an overview of KPIs, their relevance and how they can be implemented and add value to organisations in different sectors. Aspects such as the connection between strategy, goals and KPIs; essential factors to measure for museums; or the characteristics and steps for the development of KPIs are addressed to open the discussion for the following chapters.

Moreover, some essential metrics for key areas in the museum are included as merely suggestions when selecting relevant KPIs for an organisation’s goals. However, it is important to mention that these are only metrics for reference and it is highly recommended to avoid trying to measure all of them as KPIs.

Some examples in the museum sector and outside are presented at the end of the chapter with the aim of illustrating the best practice cases using KPIs, as well as the benefits they can bring to destinations and institutions when implemented in a systematic manner.


KPIS: WHY, WHAT AND HOW

Why?

What gets measured gets managed

KPIs are values (commonly expressed as #, % or rate) that are used to measure the performance of an organisation towards a desired outcome.

Their purpose is to reveal the truth about an organisation’s success in accomplishing long term goals; contribute to the decision-making processes for organisational improvement; and keep organisations on track for the right purposes.

When communicated across the different areas of an organisation, KPIs can strengthen the employee’s moral and responsibility and foster personal growth when hard work is recognised. A continuous monitoring performance allows people to achieve far more.

“If you’re not careful you might focus on things that are simple to measure, rather than things that matter” – Chris Unitt, Founder at One Further

What?Measure what is important

Nowadays, it is possible to find lists of metrics for measuring almost every aspect of an organisation. However, there is no homologous answer. Every organisation is different and has specific goals and critical aspects to address in order to succeed. Thus, every KPI makes only sense in a given context.

Even though the number depends on the organisation, the recommendation is to focus on just few core KPIs. Those most closely aligned to the organisation’s strategic goals and creating most value. Moreover, KPIs should be considered as part of a whole ecosystem that takes into consideration all areas: visitors (e.g., customer retention, customer satisfaction, reviews, interactions), staff (e.g., absenteeism, staff satisfaction), suppliers, finances, processes, etc.

Identifying what to measure helps organisations to understand why those figures are important and react accordingly.

How?Measure in comparative contexts

Organisations should just pick few KPIs, those that matter for the current end and strategic goals of the organisation. They should also be measured in a comparative context (e.g., over a period of time) in order to successfully detect paths on the organisation performance. This helps to predict downturn periods or run special offers based on previous numbers within the organisation and serves as a base for a continuous improvement of the organisation performance.

The data the organisation generates together with the data collected from visitor surveys (online and offline) form the basis of KPIs creation. However, it is important to have standardised processes supporting the KPIs tracking. This can be obtained through the use of tools such as scorecards, dashboards, or software that specialise in collecting and aggregating data, as well as in creating reports.

It is also possible to use KPIs for benchmarking against competitors aiming at finding best practices that lead to a superior performance.


+ Why

01


WHY CONTEXT - STRATEGIC PLAN, MISSION, VISION, VALUES AND GOALS

An organisation develops a strategic plan in order to document and establish the direction of the business by assessing where it is and where it wants to go (long term). In it, the mission (a goal at the highest level of attraction), vision (description of the future), and values of the organisation are reflected and embodied. They, in turn, help to focus on the efforts of the teams and the work itself as building blocks for the corporate culture.

The mission must be communicated and serves as an inspiration for the staff. It is the context where all the goals and the actions of the organisation should be framed. However, if the mission and valuesare not aligned and followed, the organisation culture can get harmed as this can cause a lack of trust among employees. On the opposite, it is said that companies whose employees see values being followed have a stronger performance and commitment towards the organisation.

Additionally, it is also important to identify the pillars or priorities of the strategy as they are the main factor of success.

The basis for KPIs

This is the role of the business goals and objectives as they are the specific aspects of the strategic plan of a company. Their purpose is to provide ideas about how the organisation can achieve its mission. Therefore, they need to be doable, understandable, manageable and beneficial for the organisation’s strategic plan.

Goals can be strategic or operational. Strategic are those that lead the company to where it wants to be. This means they are long-term goals, aligned to the mission and vision, measurable and achievable. In comparison, operational goals or so-called short-term goals are smaller and align with the strategic ones in order to make things work in the present.

Finally, in order to achieve those strategic goals, it is necessary to make them quantifiable and this can be achieved through the use of KPIs. They will be the measures used to determine the performance towards a specific goal. For instance, their role is to make goals more tangible and help to monitor if the strategic plan is being executed.

Source: BSC Designer

Level 1: Mission / VisionAllow achieving

mission and vision

Level 2: Strategic goalsDetails level: specific and measurable

Time frame: long-term (a 2-4-year period)

Doing the right thingsDoing things differently

Allow achieving strategic goals by giving workable execution plan

Give coherence to the operation goals

Level 3: Operational goalsDetails level: specific, measurable, and actionable

Time frame: short-term (weekly, monthly, quarterly goals)Doing things rightDoing things better

Does the metric align with the goals of your organisation?

Is the metric a Leading Indicator of success?Yes No

NoYes

Track it Don‘t track it


THE PROCESS APPROACH

“Process is a set of interrelated or interacting activities that use inputs to deliver an intended result” - ISO9001:2015

In other words, a process is how an organisation is operating a certain activity to convert something (an unfinished product) into an “output” (a finished product or service).

The Business Process Approach plays a significant role when defining, measuring and analysing KPIs. It focuses on the evaluation of the functioning of an organisation and to improve the operation of its key business processes (organisations are constituted of processes and not of products or services). By improving the way things are done, the ability to meet customer requirements is improved as well.

The approach considers an inter-functional management. This means dealing with those processes that depend on the action of several departments within an organisation and the improvement of flow processes across departmental boundaries. It also defines owners of specific processes with the aim to make persons accountable. Through the implementation of a process approach, it is possible to improve efficiency, reduce failures and achieve higher levels of performance.

In the context of the process approach, KPIs are the values that help to track, measure and make informed decisions based on the implementation of a process approach within organisations. The use of the process performance metrics bring the information needed to analyse the improvement of processes towards the achievement of the business objectives.


+ What

01


METRICS VS KPIS

Metrics are quantifiable measures that are tracked over time to assess the progress and performance of a specific activity within an organisation (e.g., monthly visitors). KPIs are also quantifiable measures. The difference lies in their relevance. Not all metrics are key aspects for an organisation. Critical goals of organisations evolve over time and only those metrics most closely aligned to the critic goals should be selected as KPIs. In other words, all KPIs are metrics, but metrics only become KPIs when aligned to the critical organisational goals. In this way, organisations can focus on what matters the most.

Metrics are relevant for KPIs as they can provide a bigger picture and detect why a specific KPI performs in a specific way. Simply put, KPIs that do not support organisations in their decision-making processes are just metrics.

But how to choose these most important KPIs from a metric list when creating organisational reports? Asking the following questions for each of the metrics can help with the selection process.

Why should this performance indicator be measured?What is the target?How often should this performance indicator be reported on?

What actions will be taken to influence the results of this performance indicator?Which are the tools needed to report it?Who in the museum is responsible for this performance indicator?

“KPIs simultaneously tell a story and predict future results, whereas metrics are just a snapshot in time” – Tony Leo, President and CEO, RLS Logistics


TYPES OF INDICATORS

Lagging and Leading Indicators

Lagging indicators are those presenting the success or failure related to results and help to validate the achievements, for example NPS (Net Promoter Score), in the context of improving the customer service.

From their side, leading indicators are those helping to predict the success of other metrics or the outcome of a process. For example, to predict what could be the NPS, organisations should measure product quality or customer service responsiveness as leading indicators.

Leading indicators are also used as early warning signals connected to the success factors. They help to understand the “why?”: Why did the sales or service quality drop? So, when put together with other data, they can be a key factor in forecasting changes in the final goal.

Thus, indicators can be both lagging or leading depending on the context. For that, firstly, it is necessary to understand how things are interconnected through a cause-and-effect analysis (stakeholders, success factors and desired outcomes).

For example, in the context of the goal “increase monthly sales”, “impact of sales training on business performance” would be the lagging indicator, while “training hours” would be the leading one. Training time is one of the success factors, but it is not possible to prove an increase in monthly sales just by measuring the “training hours” metric. Contrarily, “training hours“ would be the lagging indicator in the context of “training sales team”, while a leading indicator could be “number of training registrations received”. Ideally, it is good to have a combination of leading and lagging indicators when looking at the performance towards a specific goal.


WHAT NOT TO MEASURE

Focus on quality rather than on quantity

KPIs not aligned to your organisational strategic goals

KPIs that are vague or unclear

“Nice-to-know“ and not actionable KPIs

Refutable KPIs

Outdated KPIs (KPIs should be reviewed at least annually or

if the strategy or goals of an organisation change)

Having too many KPIs

SAY NOTO THIS


ESSENTIAL METRICS – FINANCES

Historically, museums have been competing for external funding. However, cultural institutions are becoming more and more market-oriented. They have started to pay greater attention to the wishes and needs of visitors and experiences they search for. Thus, cultural institutions need to evaluate how financially sustainable are these new market-oriented activities and use their resources efficiently and effectively. Typical financial metrics include the ability to meet fundraising targets, balance the operating budget, and meet revenue targets through diversified sources such as admission, shop and restaurant sales, and special events. As mentioned in our report on “Yield Management and New Revenue Sources”, financial metrics can be divided in revenue, costs and profitability figures. Here, some of the most used financial metrics are presented. The advantage of these metrics is their comparability for benchmarking purposes as they can be measured despite the organisation’s size and operational volume.

Revenue

− Revenue per visitor = total revenue / number of visitors(during the same time period)

− Earned income per visitor = earned revenue / number of visitors (during the same time period)

− Revenue per opening day = total revenue / number of opening days

− Earned revenue per opening day = earned revenue / number of opening days

− Revenue per opening hour = total revenue / number of opening hours

− Earned revenue per opening day = earned revenue / number of opening hours

− Revenue per exhibition− RevPASM = Revenue per available square meter (allows to

benchmark against market and competitors)

Cost

− Payroll to revenue ratio = staff cost / revenue (%)Payroll to earned revenue ratio = staff cost / earned revenue (%)

− Staff cost per visitor = staff cost / number of visitors− Cost per specific exhibition− Operating expenses per square meter = total cost of goods sold

+ total operating expenses / square foot of rentable space− Cost per visitor = total cost / total visits

Profitability

− Ticket coverage of expenses = revenue from ticket sales / total costs & expenses (%)

− Earned revenue coverage of expenses = earned revenue / total costs & expenses (%)

− Capacity of self-generated revenue = earned revenue / total revenue (%)

− Result per specific exhibition = revenue per specific exhibition –costs & expenses per specific exhibition

− Departmental results (e.g. exhibitions, collection, services, contributions, other, total) = departmental revenue –departmental costs & expenses

− Departmental results as % of revenue = departmental results / departmental revenue (%)

− GOP (gross operating profit) = departmental results –undistributed operating expenses

− GOP as % of revenue = GOP / total revenue (%)− EBITDA (earnings before interest, taxes, depreciation and

amortisation) = GOP – total non-operating income and expenses

− EBITDA as % of revenue: EBITDA / total revenue (%)− Net income = EBITDA – interest – taxes – depreciation &

amortisation− Net income as % of revenue = net income / total revenue (%)


ESSENTIAL METRICS - SALES

Even though sales in the form of tickets (online and offline) are the main generator of earned revenue for museums, additional revenue sources have become relevant as well for them (e.g., shops, restaurants, spaces rental for events, etc.). Thus, measuring the success of these activities at specific periods of time is fundamental for museums revenue monitoring.

If the purpose is to measure how much of the sales are coming from other sources rather than ticket sales (e.g., space rental, retail items or restaurants) then the following KPI is used:

− Ancillary sales per visitor (total ancillary sales / total visitors)

Additionally, some of the main KPIs focusing on sales at museums shops, restaurants and other ancillary services are listed here:

− Sales per visitor (SPV) (sales excluding VAT / total visitors)

− Conversion rate (retail/F&B transactions / total visitors)

− Average transaction value (ATV) (total revenue / overall number of transactions)

While tracking these KPIs, it is important to consider the visitor’s attendance at museums (among other factors). Particularly since higher SPV figures are usually harder to achieve for smaller museum shops and/or restaurants. Thus, internal benchmarking against historical data makes more sense.

Another highly relevant and traditional sales metric for measuring the operation of shops and restaurants in museums is:

− Net profit (gross profit - operating costs)

This metric is what ultimately matters when evaluating their financial performance of shops and restaurants and is what remains at the end of the year to pass it to the museum itself.

Measuring how sales are generated is also relevant for metrics related to marketing:

− Cost of acquisition (marketing expenses / total visitors) to know how much does it cost to bring in a new visitor.

− ROI of marketing tools (investment in marketing on a specific tool – total revenue / total investment in marketing in the same tool) to know how much sales volume each of the marketing tools is bringing in.

For relevant sales metrics, directly related to tickets, we can highlight the following ones:

− Ticket sales revenue it refers to the total sum of the revenue coming from ticketing

− Number of tickets sold per price segment (regular, discounted or complimentary)

− Average ticket price at a certain day

− Visitor numbers according to specific exhibition


ESSENTIAL METRICS – SOCIAL MEDIA

Social media metrics provide significant insights into the marketing performance made at different social media platforms. They tell how successful a campaign is or how a social media strategy is performing. There are both universal and platform specific metrics that are calculated differently depending on each platform and metrics tools (Facebook Insights tab, Twitter Analytics, etc.). For example, the engagement rates between social media networks vary (Instagram has higher engagement rates than Facebook or Twitter). The purpose of social media metrics is to address business goals such as: increase revenue, lower costs, or increase customer satisfaction. Social media metrics can be clustered in different classified groups such as:

EngagementThese metrics tell how much and how often the accounts of audiences are interacting with the organisation’s account. Some of the most common metrics include number of likes, comments, shares, clicks, engagement rate per post, account mentions, etc. However, more specific and thorough metrics include: average engagement rate (number of engagement actions a post receives to the total number of followers); amplification rate (ratio of shares per post to the number of overall followers); virality rate (number of people who shared your post to the number of unique views in a period of time).

AwarenessAs the name tells, these metrics focus on brand awareness and perception and include impressions (times a post shows up in someone’s timeline), reach (unique viewers a post could have), mentions, shares of post, or links to your social media. But it is also important to measure: audience growth rate (how quickly followers are gained compared to competitors); % post reach (how many people have seen a post since it went live); and social share of voice (how many people are mentioning your organisation on social media compared to competitors).

Share of voiceIndicates how much people are talking about the organisation in comparison to competitors. For instance, they are of great help when benchmarking social media. It includes metrics such as volume (number of mentions) and sentiment (positive or negative mentions). When analysing social media metrics both quantitative (degree of engagement) and qualitative (positive, negative or neutral) metrics are important.

Conversions or SalesSales is commonly a major business goal. In this context, social media is commonly linked to websites with e-commerce tools or booking platforms. Metrics for this purpose include conversion rate (number of visitors who take action on a page after clicking on a link in a post (e.g., subscribe to newsletter,

download content, buy tickets, etc.); click-through rate (how often people click on the call-to-action link

(to an additional content) in a post); bounce rate (% of page visitors who click on a link in a post and

leave the page on which they land without taking any action and benchmarking against other sources of traffic like Google Search); cost-per-click (amount paid per individual click on sponsored social media posts); cost per thousands impressions (amount paid every time a thousand people scroll and see a sponsored social media post); social media conversion rate (% of conversions that came from social media); referrals (to understand how a user ended up on your website).

Customer experienceEqually relevant at the digital level than at the physical one. Metrics commonly include: customer testimonials (reviews, assessments, comments, endorsement, or interview relating to a brand); customer satisfaction or Net Promote Scope. Response rate and response time tracking are also commonly measured to evaluate how many relevant messages is the organisation responding to in social media channels.

For more information about social media and website metrics, please refer to our “The Face of Museums” report.


ESSENTIAL METRICS – SOCIAL MEDIA

How to measure…

Source: Hootsuite

EngagementAwareness Conversion

Customer


ESSENTIAL METRICS - EXHIBITIONS

Measuring success

Museums tend to focus their energies and efforts on exhibitions because they are newsworthy, create awareness, give competitive advantage, educate, bring income, drive membership sales, etc. Thus, measuring an exhibition success is of great relevance for museums. However, exhibitions have different impacts and purposes according to their own goals and those of the institution, so trying to measure everything is not realistic or efficient. Moreover, in order to say an exhibition was successful, it needs to be compared to certain benchmarks or previous exhibitions (of the same or other institutions) serving as an evaluation reference.

One of the main aspects to evaluate in an exhibition is attendance. For that, metrics such as the number of visitors who paid full or discounted admission to special exhibitions, or total number of visitors to special exhibitions are basic metrics to consider.

In the digital era, social media metrics are essential to evaluate the success of an exhibition, specially in terms of awareness and audiences’ experience. Metrics such as the number of mentions, social share of voice (positives and negatives), customer testimonials (reviews, assessments, comments, etc.), customer satisfaction or NPS are metrics that can be taken into consideration to evaluate exhibition success. For more information about social media metrics, please refer to the social media metrics section.

An exhibition success also involves the museum management. Therefore , the balance of the expenses and profits resulted from an exhibition should also be evaluated. Some financial benefits metrics such as the costs of researching, insuring, packing, shipping, installing, promoting, costs of openings are also part of this balance. Organisation, curatorship, loans and transportation, insurance, setting-up, personnel, rights and royalties, publishing and merchandising, communication and promotion are only some of the items of expenditure normally supported for making a temporary event or exhibition. Also, the visitor side needs to be considered with metrics such as average visitor spend per day.

Social and learning impact is also another very relevant aspect that can be evaluated from an exhibition. Metrics regarding these aspects, would depend on the resources and events related to the exhibition but they could include number of attendees to lectures related to an exhibition, amount of income earned from tuition‐based exhibitions, number of schoolchildren visiting the exhibition, etc. For some (Maxwell Anderson), as only few of them provide a financial surplus, exhibitions should be judged by the degree to which they contributed to something. This could be measured in a survey to evaluate if the exhibition made visitors understand a specific topic better.

More elaborated indicators such as the social return on investment (SROI) have also been developed to assign monetary value to the social (educational) and environmental benefits of museums activities. Thus, the goal to achieve certain environmental standards could also be included as part of their strategy. In this case, the exhibition success would depend on its contribution to mitigate its environmental impact. Metrics such as carbon footprint or more elaborated KPIs such as the environmental impact of loaning an exhibition can be monitored for this purpose.

“Exhibition, as audience-oriented as it may appear, in fact serves to a complexity of intertwined aspects of a museum’s mission, not necessarily

related only to visitor experience” – Max Hollein, Director of The MET

https://static1.squarespace.com/static/57544cf9ab48de37be9c508c/t/5ed507512a6c4c5afdbf9983/1591019346621/Metrics_of_Success.pdf

https://www.fitzcarraldo.it/ricerca/pdf/measuring_museum_impacts.pdf

https://www.researchgate.net/publication/232859572_The_carbon_footprint_of_museum_loans_a_pilot_study_at_Amgueddfa_Cymru_-_National_Museum_Wales


ESSENTIAL METRICS - EXHIBITIONS

Some other aspects to be considered while selecting qualitative and quantitative metrics to evaluate the success of exhibitions are:

Quantitative

1. Extent and quality of the media coverage of a museum exhibition and the audiences reached through these media.

2. Level of support the museum receives (e.g., demonstrated by both increases in annual activity grants and comments from peer assessment juries).

3. Impact of the museum’s exhibitions for participating artists, designers, and other contributors through

− subsequent exhibition opportunities,

− grants, awards, public commissions, sales of objects, increased commercial representation,

− artist’s residencies and teaching positions,

− publications and other forms of critical recognition, and the numbers of letters of reference for artists in respect to the above,

− ongoing requests to represent people whose work is displayed (e.g., in respect to areas of advocacy, standards of professional practice, and adjudication of grants and public projects).

Qualitative

1. What do visitors look at and how do they interact with exhibit components.

2. Extent to which a visitor’s experience is meaningful.

3. What do visitors learn about the objects the museum displays, the creators or owners of the objects, as well as what are their different interpretations of the objects.

4. What do visitors learn about themselves and/or others during their visit.

5. What do visitors decide to do as a result of their museum experience (e.g., buy a related book or object in the museum’s gift store, share their experience with friends and family, return to the museum, visit the museum’s website, donate an object to the museum, visit a related museum).


+ How

01


KPIS CREATION

12 Steps Guide

Even though standard KPIs can be of great help for inspiration, organisations should ideally develop their own KPIs instead of picking those “ready-to-use” ones. The first four steps to create KPIs can be considered as basic or essential, while the rest can be considered expert levels of creation.

1. First, it is important to define the name of the KPI. What is the aspect to be measured? Organisations should break down the idea of the aspect to measure (e.g. online collection) in more specific parts to get ideas of specific aspects to measure about online collection. For example, % of collection accessible on the internet.

2. The next step is to know the strategic goal behind the KPI as it needs to be measured with a specific purpose. Thus, it is important to ask why is it relevant to measure, for example, the % of online revenue.

3. Define the current value and target or range of tolerance. This provides the reference for comparing performance with itself and to other KPIs. For example, % of visits by students. (see more about benchmarking in the benchmarking chapter of this report).

4. Define data collection and the standard procedure (ideally an automatic one) for the continuous update of the indicator.

5. Find both lagging and leading indicators trying not to focus on only one type of them. This commonly happens for those focusing on outcomes (lagging KPIs) as they are good at telling the story of the organisations’ performance on a specific matter.

6. Define the weight of the indicator taking into consideration its relevance and the rest of the indicators aligned to the same goal.

7. Define update frequency (e.g. monthly or yearly) taking into consideration the cost of measurement.

8. Set expiration date according to the evolution of the business goals or factors like the increase in measurement costs.

9. Calculate cost of tracking KPIs in order to measure only those indicators whose value of measurement is more than its cost.

10. Define rights and roles, depending on the KPI, different stakeholders might be involved in the KPI life-cycle. Hence, it is important to define persons in charge for data collection and monitoring, as well as those with permissions to see the indicator and/or change its historical data.

11. Declare how the KPI will be used, will the indicator be included in the annual performance reporting, or is it just for the self-control of a specific department?

12. Analyse expected behaviour vs. reality to understand how the measurement of the KPI changed processes within the organisation.

BSC Designer


KPIS CHARACTERISTICS

SMART KPIS

As mentioned before, goals and KPIs should walk hand in hand. This means it is not about creating SMART KPIs but setting SMART Goals. When SMART goals are created, KPIs become obvious and SMART as well. But what does SMART mean? It stands for…

Specifica clear statement of what exactly is expected to be achieved with this goal. A clear path from A to B. If goals are vague, KPIs will be hard to set. In turn, unclear KPIs do not let the organisation see what is going right or wrong.

Measurableto help to know when the goal has (not) been achieved. KPIs help to measure, evaluate and communicate the progress and performance towards the goal.

Attainable/achievableGoals should be realistic but at the same time challenging in order to motivate the team instead of discouraging their efforts (e.g. reach better numbers as the previous period).

RelevantGoals should be aligning towards where the organisation wants to be. Every smaller or short-term goal should be relevant to the greater end-goal.

Time-boundingit is finite and includes key milestones on the way to achieving the goal. This forces organisations to map out and prioritise the necessary actions to reach the goals on time.

KPIs can become SMARTER if they are evaluated and revised. These steps ensure the continuous assessment of the KPIs. It is not only about reaching a goal but to know why and how it was reached.


KPIS CHARACTERISTICS

Besides being aligned and relevant to the strategic goals of the organisation, KPIs should also be:

Source: Mohammed B. A survey on exploring key performance indicators

Sparsethe fewer KPIs the better and more effective outcome they

provide

Simpleclearly indicate the required action by the staff

OwnedKPIs must have an owner

CorrelatedKPIs must drive specific desired outcomes and encourage the appropriated

action Balancedconsist not only of financial metrics and take into consideration all relevant areas

AlignedKPIs should not undermine each other

TrackedKPIs should be continuously tracked and measured

Validatedthe staff should be involved in their creation and address the required tasks for it

Distributedthe responsibility should be tied down to a team

Characteristics

Referencestheir origin and context are visible and accessible to everyone


CALCULATE KPIS

Performance vs Progress

The first step to normalise KPIs is to make them comparable. This means to homologate the scale and units of measurement. Making them comparable gives context to their performance or progress over time and helps to make sense out of the scorecards. They can be presented in absolute measures; or compared in the way of percentages or ratios.

The first aspect to define before calculating a KPI is to know exactly what should be defined, what is going to be measured. Is it the performance or the process of a specific metric which is being measured? Progress is the process of improving or developing something over time, while performance is the execution of an action or the accomplishment of a given task measured against previously known standards (e.g., when compared to some historical data).

For example, when calculating performance, it is necessary to include the minimum and the maximum values of the scale of the KPI in order to reveal the overall success.

Performance (%) = ((Value – Min) / (Max – Min)) * 100

On the other side, while measuring the progress it is necessary to use the baseline and the target to, in this case, know to what extent the target has been achieved.

Progress (%) = ((Value – Baseline) / (Target – Baseline)) * 100

Within the calculation, it is also relevant to consider that each of the KPIs has a different weight depending on their relevance on the achievement of a specific goal and this needs to be considered while calculating the total performance. Dashboards can help this purpose and are later on addressed in this report.



STANDARDISATION

Data and KPIs

In practice, KPIs standardisation within an industry is not an easy task. This is because different aspects are measured among organisations based on their strategic objectives and development strategy. However, the standardisation of KPIs is highly relevant for benchmarking.

When talking about standardisation of KPIs, we talk about the standardisation of organisational data. Organisational data determines how useful and usable the data assets of an organisation are.

In the specific case of KPIs, non-standardised data can be very problematic. Mainly when KPIs are comprised of more than one data source that cannot be merged. Thus, setting a standardised way of measuring KPIs should be the first thing organisations would need to focus on. This will reduce time and effort costs while evaluating performance on a regular basis.

On the other hand, a standard reporting of KPIs provide a benchmarking framework not only for the direct customers but also for all the different stakeholders (customers and partners) involved in the operation of the organisation. When stakeholders, and the industry in general, are tied to these industry accepted standards, everyone can focus and work on the specific actions that have bigger impact on the business.

KPIs standardisation also refers to the terminology and metric units. It is common to see that organisations have their own language according to their organisational culture. Two different organisations can measure the same indicator but with a different name, metric unit and format. To avoid this, it is suggested to use short and concise names for the industry uses and add the symbol according to what the KPI measures.

Although the standardisation of data for benchmarking purposes depends on the contribution of each organisation and the lobbying efforts made within associations, organisations can tackle it following specific steps. For example,

• the implementation of data standards and their communication among the organisation,

• finding and cleaning the current existing data anomalies within the organisation,

• modification of existing forms and data collection tools to gather data.

“A uniform approach gives KPIs, and the overall industry, additional credibility” – Marty Steinmetz, Head of the KPI

Working Taskforce in the Global Cold Chain Alliance

.


+ Examples

01


Strategy, Goals and Metrics for Managing a City

Belfast CityIreland

CASE STUDY


BELFAST CITY

Strategy, Goals & KPIs for Managing a City

How can an entire city measure its development and management and see if it has achieved its goals? As for museums, the answer is with key performance indicators and metrics. Belfast with its 330,000 inhabitants shows us how to do it.

The City Council partnered with Bernard Marr & Co., a business adviser and consultant, to develop a strategic performance management system and ambitious plan – The Belfast Agenda – to improve the quality of life in the city by improving both service delivery and the Council’s civic leadership role. In order to achieve this goal, mapping the city’s strategy, developing meaningful KPIs and putting in place performance management processes were indispensable.

In a first step, all involved parties sat together in order to identify the key strategic goals of the city and summarise them in nothing more than a one-page strategy map. The map is not static and can be adapted over the years to ensure that it remains relevant to the City Council as well as its inhabitants. After that, it was easy to determine which existing corporate initiatives, projects and programmes were of benefit to delivering the strategy and which ones had to be bolstered or established.

Key performance indicators helped to monitor and manage the delivery of the strategic goals. Firstly, the Council came up with relevant questions (so-called key performance questions) in order to design these KPIs. In return, the indicators helped to answer the most important questions and understand how well the Council was performing against its strategic goals.

Source: Bernardmarr https://www.bernardmarr.com/default.asp?contentID=1075

https://www.bernardmarr.com/default.asp?contentID=1075


With the KPIs in place, the Council could extract performance reports and dashboards to provide automated real-time insights into the performance of the organisation. Later, they were distributed among the whole staff of the Belfast City Council.

The strategic map saw such an enormous success that the city of Belfast decided to develop a strategic agenda including key themes to focus on for the entire city to help shape its vision and aspirations. This endeavour helped to focus everyone on the things that really matter and allowed the Council to manage and measure what is most important. Even if it sometimes seems difficult to generate a strategic performance management system, if it is possible for an entire city with its political diversity and different stakeholder interests, then this is something every organisation can do too.

Source: Bernardmarr


Aligning KPIs

The METNew York, USA

CASE STUDY


THE MET

Missions, goals, new challenges and KPIs alignment

The MET is a non-profit institution with the mission of offering public access (with a pay-what-you-wish policy for local visitors). At the same time, the museum has been affected, (like almost every organisation in the world) by a health emergency due to COVID-19 outbreak. In this context, even though the KPIs are not solely planned to focus on increasing financial figures but to measure its connection with its public as well, The MET does have a responsibility to raise revenue and lower costs. Thus, the museum has aligned its KPIs with both the current and long-term objectives of the organisation. Among its most relevant financial KPIs we can find: attendance per month (and its change rate over time), admission revenue per month (and its change rate over time), revenue per admissions, retail and F&B.

Demographic KPIs are also part of current museums strategic goals. They are closely monitored in order to observe the variation of the market source (local and international) and admission categories of visitors (consequence of current health situation as well). Furthermore, the demographic KPIs are relevant for the museum to ensure the mix of audiences and thus fulfil the museum’s mission. This also informs about the ways to re-approach and re-tailor programs (ticket packages tools and offerings) according to changing visitors.

Visitor experience KPIs are monitored in the form of members satisfaction. Its evolution is continuously monitored but the current situation has granted this KPI with a high relevant role due to the emotional connection they have with the museum. Therefore, one of the main MET goals is to make sure the audience, and specially members, are taking the museum’s “ownership”.

Monitoring and improving visitor (and member) experience allows to establish a better communication with them, understand their current behaviour (e.g. increase in time spend due to COVID-19), make intelligence effort for memberships renewal rates, etc.

Digital KPIs: MET‘s digital offer has not only increased in the last years following the digital era (it currently reaches millions of users through several channels), but it has also accelerated due to the current health emergency. Aspects such as website users, online collection users, all platforms users, countries reached, or online revenue are relevant digital KPIs for the museum. KPIs related to social media are also highly relevant for the MET. Particularly if we talk about how much conversation is triggered (mentions), the velocity and response rates, and the relevance of the mentions in digital channels (sentiments, interest, trends).The relevance and continuous monitoring of MET’s digital KPIs is supported by the goal of making its digital offer (e.g. virtual lectures) sustainable in the long run.

https://smbp.uwaterloo.ca/2017/06/metrics-for-the-met/


Science Museum GroupKPIs for the Family of Museums

UK

CASE STUDY


SCIENCE MUSEUM GROUP

Standardised KPIs for all

The Science Museum Group is a consortium consisting of five UK science museums which share the same ambition. Engage people in a dialogue about the history, present and future of human ingenuity in the fields of science, technology, medicine, transport and media; also, to help building a STEM-literate society that celebrates science, technology and engineering, as well as the impact these have on our lives, now and in the future. The consortium came up with a strategic paper writing down their objectives and goals for the period 2015-2030. Although each museum has its own slightly different core mission and vision, the museums agreed on applying standardised key performance indicators categorised by objectives in order to measure their goals. Each museum also predefined their aspired outcomes. The Science Museum Group serves as an example of how museums can work together to create common KPIs.

Also of note is the striving of The Happy Museum Project to define common KPIs for sustainability in the museum field. As Max Hollein mentioned in the Future Museum workshop on KPIs, these will become more and more important in the future.

https://www.sciencemuseumgroup.org.uk/

http://www.sciencemuseumgroup.org.uk/wp-content/uploads/2017/06/Science-Museum-Group-Plan-2013-14.pdf

https://happymuseumproject.org/measuring-matters-metrics-sustainability/


SCIENCE MUSEUM GROUPC O M M O N O B J E C T I V E S

Aspire to the highest international museum standards in the care and

preservation of collections, scholarship, programming, learning and advocacy for our subject areas

Implement clear audience strategies that focus on providing

life-enhancing experiences, strengthen the core narratives and deliver dynamic gallery displays and public spaces

Extend the reach nationally and internationally

Make optimum use of the estateBe an organisation that is extrovert,

entrepreneurial, efficient and dedicated to the development of

great people

K E Y P E R F O R M A N C E I N D I C A T O R S & M E T R I C S

Collections care: − Proportion of areas meeting

storage grade goals

Research: − Total external research

income/grant commitments received

− Number of academic publications− Number of academic conference

papers/talks

Satisfaction: − % Very Satisfied with their visit− % Definitely likely to recommend a

visit to friends and family

Physical visit number: − Total number of visits into the

museums

Virtual visit numbers: − Total number of online visits to

museum/SMG websites

Learning numbers:− Number of visits in booked

educational groups− Number of instances of

participation in on-site activities

Delivering life-enhancing experiences: − Annual assessment of performance

in delivering life-enhancing experiences (quantitative and qualitative data)

Core and growth audience segments:− Assessment of visitor profile –

proportions and numbers

Learning Outreach:− Number of instances of

participation in off-site organised activities

Loans:− Number of objects accessible on

loan− Number of loan venues− Number of venues hosting our

touring exhibitions and numbers visiting

Carbon reduction:− Qualitative and quantitative

assessment of performance in reducing CO2 emissions in light of programme and activity

Staff satisfaction:− % agreeing with employee

engagement questions within the Employee Opinion Survey

− Qualitative assessment of Employee Opinion Survey results

Income:− Capital donor commitments− Restricted revenue donor

commitments for temporary exhibitions

Restricted revenue donor commitments for on-going programmes:− Unrestricted revenue donor

commitments− SMG enterprises profit− Other income (including interest)

Operating cost:− Operating cost per visit


Future of KPIs

+ Trends

+ Monitoring Tools

+ Best Practices 02



The business environment evolves continuously and with it the strategy, goals and KPIs of organisations. The organisational adaptation of strategies is necessary not only to provide an up-to-date product and service to the visitor, but also to maintain the operational systems and processes of an organisation. Moreover, the emergence of new trends and business environments brings with it new KPIs that gain relevance at the competitive level.

The aim of this chapter is to present some of the KPIs and metrics used to address the new trends in the cultural sector. At the same time, we present some tools that have been developed to help monitor and display not only KPIs but also the whole strategy of organisations. This makes it easier to keep track of relevant business aspects and involve all stakeholders.

Finally, some best case examples of the use of scorecards and dashboards for monitoring purposes are shared.


+ Trends

02


VISITOR EXPERIENCE

Visitor or customer experience has become one of the most relevant aspects an organisation needs to observe if it aims for success. Monitoring experience requires a holistic vision. It goes from the psychological aspect of the visitor to the involvement of innovative technologies. Also, the visitor experience does not only start at the entrance of the attraction, but at all touch points of the journey. This means that when measuring visitor experience, aspects such as employee courtesy, entertainment value or cleanliness, among others, need to be included. The most used KPIs commonly used for products and services for customer service are:

• (NPS) Net promoter score (% detractors - % promoters) to tell how likely customers are to recommend the product or service, normally measured in surveys through rating scale questions.

• (CSAT) Customer satisfaction (all positive responses/total number of responses*100) also called Happy Customer Index helps to measure the overall satisfaction of products or services.

• (CES) Customer effort score (% customers who replied easy - % customers who replied difficult) telling the ease of using products or services. Normally measured in surveys through rating scales questions.

• (ART) Average resolution time (% of time needed to solve customer or visitor issues, on average) it focuses on customer service.

• (ServQual) Service quality (perceived service score - expected service score). It is integrated via 10 dimensions (tangibles, reliability, responsiveness, communication, credibility, security, competence, courtesy, understanding customer and access).

Other metrics can be as simple as: % of visitors who rate the overall visitor experience as good or excellent, or % of visitors who consider the tourist information service to be good or excellent.

However, when talking about a service, it is highly relevant to measure the efficiency and knowledge of the staff. An employee who does not meet the expected standard is likely to result in a negative experience. This can be measured in surveys through the % of first response time or staff helpfulness.

As mentioned, the experience is holistic and include all touch points and museums are becoming, at the same time, more and more digital. Thus, the online experience (e.g., online ticketing tool or websites) can also affect the overall attraction experience. In this case, metrics such as % page load speed, bounce rate (total number of one-page visits/total number of website visitors) or simply % of users who consider the destination website overall to be good or excellent; can help to track visitor experience.


KPIS AND KEIS

"When you hire a hand, it comes with a head and heart attached“ -Peter Drucker, Austrian Management Consultant

Experience became a new economic value and, with it, emotions acquire an enormous relevance. Emotions play a relevant role for several aspects such as how people make decisions, how the consumer or visitors consume products, how the job is performed, or how productive or creative the staff can be. Positive emotions are associated with better performance, quality, and customer service. When consumers’ senses are dazzled, their level of engagement with a brand is stronger and this also needs to be measured.

Even though emotions will always be subjective, it is urgent for organisations to start considering emotions and include Key Emotional Indicators (KEIs) as part of their dashboard.

Employee emotions KEIs, just as KPIs, cannot exclusively point towards the evaluation of the visitor experience. Instead, they should also measure the emotional progress and engagement of the stakeholders within all the areas involved in the achievement of the organisation’s goals. Employees’ emotions are a relevant part of what happens within an organisation. For example, joy has been found to be one of the strongest drivers of employee satisfaction and commitment in a company. Therefore, emotional culture within organisations has become as relevant as the organisational one. It refers to the affective values and norms that govern the emotions people have and express (or suppress) at work. Thus, tracking elements of performance such as ”joy at work” can be as relevant as those for productivity or creativity.

KEIs related to employee emotions can be gathered through the regular use of employee engagement surveys related to job satisfaction and commitment. KEIs can be related to performance (how satisfied are the employees with their own work), management (how employees feel towards their manager) or collaboration (how employees feel the collaboration with their colleagues or degree of affection, caring, and compassion employees feel and express toward one another). Here it might be convenient to remember that the emotional culture in a company is mainly transmitted through nonverbal cues (e.g., body language or facial expressions).

A great example of the impact a healthy emotional culture has on employees and consumers is Vail Resorts. Its efforts for joy cultivation among employees has helped not only to make the experience more fun for customers but also to retain valuable employees.

On the other hand, organisations such as The Happiness Index use data analysis, AI and neuroscience to provide visibility into the key happiness drivers impacting cultural health, employee engagement, customer engagement and productivity.

Source: Harvard Business Review – Manage your emotional culture

Happy Employees = Happy Customers = Growth (Personal & Organisational)

http://www.vailresorts.com/Corp/info/values.aspx#:%7E:text=BE%20SAFE%3A%20Be%20committed%20to,and%20share%20the%20contagious%20spirit.

https://thehappinessindex.com/


KPIS AND KEIS

Consumer emotionsIt is not a secret that happy customers lead to more visits and sales. But how to trigger and measure those positive emotions leading to, ideally, more loyal visitors? It is said that the more emotional an ad is, the more memorable it is. Likewise, the more positive emotions a product or a service triggers in consumers, the more likely they will engage, trust and recommend a brand, and vice versa (85% of customers increase their spend after a positive experience 70% of them cut it when unsatisfied).

Measuring emotions of visitors help organisations to understand and identify buying behaviours that drive purchasing decisions and brand loyalty. As well as to discover what has exactly triggered these emotions.

The most common and used KPI for measuring emotions is Consumer Satisfaction. It is measured by dividing all positive responses by the total number of responses and multiplying the result by 100. However, it is important to notice that satisfaction itself might not be enough as it focuses only on one of a wide range of emotions customers feel. As well, it does not tell us, for example, which were the emotions that drove a specific behaviour.

The Net Promoter Score is another relevant KPI used to measure both consumer loyalty and consumer satisfaction/experience. It helps to create a meaningful score based on customer loyalty behaviours. A high NPS means likeliness to outperform competitors. The question to ask is “How likely are you to recommend us to others?”

The Consumer Effort Score is helpful in predicting customer loyalty. It can be measured by asking the consumer the total effort required to complete a transaction or perform an action. Every evaluation of effort shows an emotion (joy, irritation, etc.).

Source: Emolytics / Hosany, S. - Determinants of Tourist Emotional Responses


KPIS AND KEIS

Visitor emotionsIf organisations and destinations want to keep customers interested and loyal, they need to go beyond and surpass basic experiences. This has led not only towards a fierce competition among tourism stakeholders providing everyday more memorable experiences, but also to the need of tools to measure the role of visitor emotions. Thus, innovative concepts such as the WOW-effect (state of amazement after being surprised by something) have emerged as useful tools to enrich the comprehension of the visitor experience at destinations. However, due to their subjectiveness and qualitative nature, it is difficult to convert them into actual metrics or KPIs.

An interesting approach proved to be a useful tool for measuring the emotional experience and responses of travellers towards destinations is the Destination Emotion Scale (DMS). This scale is represented by three emotional dimensions: joy, love and positive surprise (vacations are characterised as a positive experiential process). Each of them triggered by different determinants, specifically:

Pleasantness: Perhaps the most powerful evaluation, as consumption emotions are differentiated by it. Whether the outcome is pleasant or unpleasant. It can be measured by asking the visitor: “How pleasant or unpleasant was the overall experience with the destination?”

Goal congruence: In holidays, tourists expect to have satisfying and memorable experiences. Thus, at the end, they evaluate whether they have achieved these goals. It can be measured by asking the visitor: “To what extent the overall experience with the destination was important or not important in achieving your goals/needs/desires?”

Source: Emolytics / Hosany, S. - Determinants of Tourist Emotional Responses

Internal self-compatibility: It is the assessment of the degree to which the experience is consistent with one’s self-concept. This is, the greater the match between a destination’s image and one’s self-concept, the greater the tendency for tourists to visit that place. It can be measured by asking the visitor: “How did the overall experience with the destination affect feelings about yourself, such as your self-esteem?”.

Currently, tools such as neuromarketing study the brain activity responses (specifically those related to emotions) to the exposure of marketing. Also, digital platforms, like Emolytics, have created their own KEI (the Emoscore ) to measure the emotional value or benefit of products and services. The score positively correlates with metrics such as conversion rates, word of mouth, loyalty, purchase intention, etc. Furthermore, the emotions gathered can be filtered by demographic characteristics such as gender, age group or geographical locations.

https://blog.emolytics.com/


DIGITAL CONTENT

With the emerge of digitalisation, the need to measure the success of “digital” within organisations has become more relevant – especially when talking about the offer of online exhibitions. Making online exhibitions attractive for wide audience forces museums to start a deeper analysis of their online offer. The migration of all the collection data into a content management system, creation of memorable online experiences (film, AR and VR) require a lot of time, funds and expertise. Thus, KPIs that monitor how online visitors interact and experience the online collection, become more and more relevant.

Some online content -related metrics that can be considered as KPIs (depending on the organisational goals and objectives) rotate around visitor interest and loyalty. Here are some suggestions:

− Average page views per visit (total number of page views / total number of visitors) as visitors become more engaged with the content provided, they continue to click and generate additional page views,

− % of high-medium-low frequency visitors (total number of low-medium-high frequency visitors / all visitors) helps to break down the frequency with which visitors come to the online site (collection). In this regard, it is important to consider timeframes as a visitor returning many times a day is likely more engaged than the one returning many times a month,

− Average visits per visitor (total visits / total visitors) to measure visitor loyalty,

− Ratio of new to returning visitors (total new visitors / total returning visitors) to measure the online visitor acquisition and retention activities with the aim to maintain near or below 1.00,

− % of high-medium and low time spent visits (total number of visits spending less-around-more than 30 secs on the site / all visits) to measure the visitor’s interest and engagement which when compared to average page views per visit tell us how visitors spend their time in the online content,

− Landing page “stickiness” (single access page views of an specific page / entry page views of the same page) helps to track if visitors leave without exploring the online content (mainly for home page and top five entry pages),

− Subscription conversion rate (conversions / total visitors* 100) to track if the call to action for email-based newsletter, RSS feeds or podcasts subscription is working,

− Average searches per visit (total number of searches (page views) / total visits) Contrary to other industries, for museums’ online collections the higher this metric, the better it is.

X


KPIS AND COVID-19

Pillars for recovery

Due to the imposition of travel restrictions and the risk of infection while traveling, tourism organisations have been forced to change priorities and, for instance, their goals. Thus, short term goals have acquired a new high relevance, and with them the use of new KPIs that make these goals quantifiable.

Think LocalIn the framework of COVID-19, there has been a shift to regional and day trips. Hence, the industry is now turning its head to restore the relationship and engagement with local markets and respond to residents’ concerns. At the end of the pandemic, this can result in strong and enriching local community-museum relationships and, for instance, more interesting organisations for international travellers. Thus, metrics such as % of local visitors vs international tourists, revenue per visitor by source market, or the number of new partnerships with local institutions can get new relevance.

ExperienceWith the implementation of social distancing new measurements such as wearing masks, limited number of visitors, access restriction to certain areas or the cancelation of certain events, measuring customer satisfaction became one of the main priorities of tourism destinations (see slide 39).

Costs reductionCosts need to be spent strategically. The low demand of staff can be tackled using freelancing and on-call contracts for seasonal employees (always within the framework of an open and clear communication). The cost of maintenance or special services can be addressed through leasing contracts. However, investing in marketing (e.g., social media), advertising or digitalisation is highly relevant. Total cost savings, or costs avoidance are key financial metrics to follow in these times.

Customer focusIt’s time for paying attention to what is relevant for the audience and build up customer relationships and trust in a more human level. Measuring customer participation and engagement, specially in social media and digital channels, is a way to do it (see slide 16). Also, intended time for next visit is one of the best metrics to predict the actual visitation behaviour and can provide a great insight for management matters.

DigitalisationThe fact that guests cannot visit the destination should not mean there is no business on-site. Therefore, staying connected with audiences digitally is a way to let them experience the attraction differently. Aspects such web traffic, number of virtual tours offered or % of assets digitalised should be tracked. Thus, attendance to those tours and workshops can still be measured but at the digital level. Same counts for % of returning virtual attendees and % of lead generation of online visitors signing-up to newsletters.

DistributionIt is perhaps one of the most relevant aspects to track in the COVID-19 context. Metrics such as the overall time spent in a specific area (shops, or and F&B), time spent in venue processes (e.g., queuing time) or number of visitors in specific areas.

Safetyis another highly relevant aspect to measure in order to comply with the regulations that allow keeping the destination open and provide visitors and staff with a safety environment and conditions. Metrics such as % of awareness training penetration and % of application of preventive measures can result in a KPI such as % of behaviour adjusted according to requirements (gap between expected and implemented).

Source: Blooloop

“What people get now because of the need to reduce capacity is a unique experience“

–Tim Reeve, Director of V&A


ENVIRONMENT

If museums want to be competitive, they need to address the environmental topic. Everyday more and more authorities are requiring organisations to comply with environmental regulations and visitors are more and more empathic with the cause. In this context, carbon footprint, green exhibits, energy efficiency are concepts that have arrived to stay. But overall, the concept of “Green museums” is already under debate and diffusion to understand how direct (electricity and water usage) and indirect (waste disposal, suppliers and visitor's emissions) activities museums perform contribute to climate change.

There are, at a global level, several environmental systems (EMAS, ISO 14001) that provide the tools to improve the environmental performance of organisations. Moreover, a lot of data which is required to report on environmental KPIs, is already collected either through standard business data (energy or water bills) or on internal reports (CSR reports).

Furthermore, each activity in the museum can contribute to climate change and be improved. For example, improving the infrastructure for a more sustainable use of the AC used for the conservation of collections. Also, loaning exhibitions is one of the activities contributing the most to climate change by museums. This requires not only the use of means of transports and carriers (specially those using aircrafts) highly contributing to greenhouse gases emission, but it also requires a great amount of material used for this purpose. In this case, instead of just offsetting carbon emissions, museums could focus on improving the efficiency of their activities through several actions. Among them setting upper limits to the number of objects in ground-shipped loans; try to use rail transportation; share couriers when possible; use renewable materials; reuse and recycle material used; use of nontoxic and low-emitting materials; use of certified wood; or use of local materials.

Once again, each institution is different and has different challenges, architecture, activities and, more important, strategic goals. Thus, KPIs that work for some might not be relevant for others. However, some of the most common metrics that can serve as a reference to improve environmental contribution are:

− electricity consumption (consumption of electricity/museum’s area)

− freshwater consumption (consumption of water/staff + voluntaries + visitors)

− consumption of consumables (expenses for consumable office materials (e.g., paper, ink, files, stationary)/staff + voluntaries)

− total of CO2 emissions per staff member

− percentage of compensated CO2 emissions

− percentage of renewable energies to total energy use

− percentage of recycled paper

− percentage of wastewater treatment to total wastewater

− percentage of solid waste treatment

https://www.researchgate.net/publication/232859572_The_carbon_footprint_of_museum_loans_a_pilot_study_at_Amgueddfa_Cymru_-_National_Museum_Wales


+ Monitoring Tools

02


SCORECARD

“BSC is used to know: How are we doing? Are we ready for the future?” – Scott Bangle, General Manager of the Riverside County Park and Open-Space District in California

What is it?A Scorecard is a tool used to present (internally and externally) a clear visual picture of the up-to-date organisation progress towards its specific business goals (global and individual). Thus, it focuses on the goals of the organisation; measures and monitor progress; prioritise projects, products and services; and align the day to day work towards strategic targets.

Scorecards show historical data and relationships (cause-and-effect connections) between the mission, goals and KPIs within the strategy. Therefore, it is possible to see the KPIs relationships towards the achievement of one or more goals and their progress over time (e.g., high revenues and high perceived value measured in the context of customer satisfaction). Moreover, each KPI is defined with an own weight making its contribution to the achievement of a goal clear.

Each department can have its own scorecard, which should align with the overall strategy of the organisation. According to the goals of the organisation, KPIs can be brought in, deleted or moved to the scorecards of the teams or individuals working on this specific aspect.

It is important to clarify that scorecards are not the same as KPI reports. The latter presents the summary of KPIs with visualisations, explanations, historical comparison but without showing how the mission, vision, goals and KPIs themselves are linked within the strategy of an organisation.

Four perspectives and BSCThe Balance Scorecard (BSC) is the strategy execution framework or tool most used among large organisations. It is called balanced not only because it is divided in the four “main” perspectives of an organisation, but because those perspectives also include the input from both internal and external stakeholders. This is of great help for organisations with a wide range of stakeholders (e.g., museums) and essential for not losing track of critical areas when solving an urgent problem. The four main perspectives within a BSC are:

Financial: one of the most relevant perspectives even for non-profit organisations as all of them have stakeholders with financial interests. For example, in the case of non-profit museums, local authorities are likely to care about their impact on the local economy, then the financial perspective becomes a success perspective.

Customer perspective: which includes both partners (internal and external) and end customers. Thus, it might involve departments with no direct contact with end customers. Possible goals are product quality or customer experience.

Internal perspective: it addresses the processes the organisation goes through in order to satisfy the customer needs and meet financial goals. This means KPIs within this area need to be strongly linked to those on the customer perspective. Goals of the internal perspective can be “developing new products or services” or “innovation of current ones”.

Learning and Growth Perspective: the customer, technology and environment are continuously changing. Thus, a continuous learning and skills updating are needed in order to keep track of the pulse of the industry.

https://bscdesigner.com/four-perspectives.htm#customer

https://bscdesigner.com/four-perspectives.htm#internal


SCORECARD

Balanced Scorecards



BOSTON CONSULTING GROUP – THE BALANCED SCORECARD FOR MUSEUMS

How can a museum performance be measured?

While measuring a business and its success is relatively unchallenging, evaluating the performance of a cultural institution is more complex. Focusing on financial strength is not enough, the value of artistic and cultural offerings has to be taken into account. In this context, the Balanced Scorecard emerges as the tool to help museums measure their success achieving target objectives and facilitating the allocation of scarce resources.

The Boston Consulting Group adapted the balanced scorecard to the specific needs of museums. This means that it not only includes financial metrics but compiles subjective measures, such as satisfaction and retention of both customers and employees, as well as perceived value by customers and the input of external stakeholders. This serves the museums which must consider the needs of a broad range of stakeholders: donors, employees, artists, local businesses, researchers, the public, and local authorities and their often-competing objectives. The scorecard creates four dimensions of value against which to measure performance:

1. Artistic Contribution: This dimension evaluates how museum assets such as collections, intellectual capital, and brand have been developed and shared. Measurements include percentage of works on display, the number of pieces on loan to other art museums, the number and quality of institutions to which the museum has lent art and artifacts, the number of pieces purchased in the last year, the % of permanent collections acquired and catalogued, as well as number of articles published by museum staff in scholarly journals.

2. Public Benefit: Its aim is to evaluate the contribution of museums to the community and the relationship and positive perception is built by offering programmes and visitor services, experiences provision, as well as collection preservation. Metrics include range and variety of offered programmes and visitor services, % of the museum’s budget devoted to marketing, % of return visitors, increase in first-time visitors and the number of schoolchildren visits per year. Also, some qualitative measures such as the increase in the community's cultural knowledge and awareness for a particular artist or artistic movement can be considered.

3. Learning & Growth: It measures if museums offer an environment for staff development and opportunities for professional and personal growth. Is the museum a place where employees can flourish? Measured can be the % of the museum’s budget dedicated to training and career development, % of satisfied employees, feedback from all levels, evaluation of staff performance, and the degree to which employees are involved in the museum’s governance.

4. Finance & Governance: The finances should be transparent and the resources use efficiently. Thus, this dimension measures how well-run and financially sustainable is the museum. Measurements include the ability to meet fundraising targets, balance the operating budget, and meet revenue targets through diversified sources such as admission, shop and restaurant sales, and special events.

Source: Boston Consulting Group. Measuring Museums Performance https://www.bcg.com/ru-ru/publications/2012/public_sector_arts_culture_art_and_business_measuring_museums_performance

https://www.bcg.com/ru-ru/publications/2012/public_sector_arts_culture_art_and_business_measuring_museums_performance


DASHBOARDS

A reporting tool

Dashboards are tools helping organisations to visualise in real time the insights gathered from the collected data. Their goal is to present the information in the best possible and tailored way for the organisations implementing them. They also help to simplify complex data converting it into manageable and meaningful visualisations.

There is a great variety of dashboards options for specific areas (finance, project management, social media, etc.) and different types (analytical and operational aspects) each of them tracking specific metrics and telling a specific data story. These software tools commonly provide easy drag-and-drop dashboards creators; easy data pulling and integration from other data sources (Google Analytics, Facebook, Salesforce, MailChimp, etc.); easy to use graphs, widgets, geographical maps tables, etc.; export and sharing across platforms; online and offline access from several devices (computers, tablets, or mobile phones); intuitive formula engines with statistical functions; etc.

One common mistake of organisations when using KPIs is not distributing them widely to take advantage of the staff talent. Thus, KPIs dashboards are an easy tool for informing the staff of specific departments about specific metrics and ask for feedback. This can even be done through schedule reports arriving at specific set-up times. However, it is necessary to make a distinction between KPIs and metrics and use internal dashboards to communicate the strategy of the organisation and align the work of the different departments towards those specific goals. This enhances the collaboration and communication within the team and between business units. In this context, the staff’s feedback is an important element to shape the different department versions of the dashboard.

In this context, only a few KPIs should be included in a dashboard. Only those that survive the proper discussion process. The others could be moved to the scorecards of internal teams or departments working specifically on that matter. Moreover, the proper structuring of a dashboard framework enables to keep account of the happenings in organisations.

In summary, dashboards need to make metrics matter (relevant and accurate information); be clean, simple and visual, be interesting and interactive, as well as accessible.


BUSINESS ANALYTIC TOOLS FOR MUSEUMS - TRAVELSEE

TravelSee, a young start-up from California, has made it its business to assist museums in harnessing the power of data in order to increase ticket sales, fundraising, memberships, gift shop sales and visitor engagement. The company offers a digital platform, TravelSee Analytics, which delivers data analysis services adapted to the needs of museums. The platform allows museums to upload their audience data, compare metrics and gain insights in three areas: event ticket sales, attendance trends and marketing campaigns.

“TravelSee analysed and visually interpreted the raw data from our summer exhibition season – it helped us to identify (whether) specific marketing plans were working – making my job easier.” – Todd Kenyon, Director of Marketing, Fenimore Art Museum

Event Ticket Sales Analysis – Fenimore Art Museum:

The museum wanted to understand how different advertising channels impacted ticket sales during a major exhibition. TravelSee used visitor zip codes, split them into two geographical areas, ranked the top 30 zip codes from each area and compared them to the advertising expenses spent in each respective area. By doing so, the company found out that the top 10 visitor zip codes all featured radio advertisements, to the surprise of the marketing department, who believed that digital marketing would attract more visitors.

Digital Marketing Analysis – Munson Williams Proctor Arts Institute:

The museum’s wish to understand the best advertising channels for a large art exhibition built the backdrop of this analysis. The used datasets for this analysis consisted of visitor zip codes, admission records regarding visitor types, exhibit advertisement expenses, the exhibit digital marketing report containing clicks, impressions, and the answers of exhibit visitor surveys. Advertising channels were divided into traditional channels (radio, print, TV) and digital channels. Traditional advertising expenses were then compared to visitor zip codes as well as visitor survey responses. Digital advertising was processed in a different way with a focus on comparing online website engagement with on-site attendance. The outcome was that although a large part of the budget was spent on print advertisement, the majority of the visitors was attracted via radio to the exhibit. The report also showed an upward online engagement trend in clicks tied to specific web banners and helped the executive director to shift funds to a better digital advertising service.

http://www.travelsee.org/index.php

http://www.travelsee.org/projects/fenimore/

http://www.travelsee.org/projects/mwpai/


BUSINESS ANALYTICS TOOLS - DATAPINE

Datapine is a business analytics and data warehouse tool helping organisations to quickly integrate data from multiple databases (internal and external applications or spreadsheets) to analyse, visualise and validate it. This is possible as the software has included native connectors to the most used databases (e.g., Google Analytics, Google Spreadsheets, SAP, Salesforce, Dropbox, SQLs (Azure, Server), Oracle, among others.)

The tool offers customisable and interactive dashboards to visualise and understand historical data (trends and patterns) in the form of charts, geographic maps, or scattering plots. Moreover, as Datapine is a browser-based tool its dashboards can be accessed from anywhere at anytime.

Datapine also provides the opportunity to share in real-time the information with thedifferent areas of the organisations through its dashboards. This helps to improve thecommunication of information and the empowerment of the team as they are easilyoperated (e.g., for the creation of individualised reports).

Even though the system tool brings services for many industries but the museum sector,its solutions can be developed for several museum areas such as finances, HR,marketing, sales, procurement, service, etc.

Furthermore, Datapine also offers a detailed guide of KPIs for the previously mentionedbusiness areas.

https://www.datapine.com/


BUSINESS ANALYTICS TOOLS - PLECTO

Plecto is a dashboard data visualisation software developed by a company in Denmark, which is designed to help companies boost performance and employee engagement by visualising real-time data and KPIs. The tool also offers gamification features for employees – such as contests, achievements, and leader boards. Another feature is that the platform allows to create custom KPIs to monitor custom metrics. Plecto offers different solutions for various divisions such as sales, marketing, finance, development or customer service.

The dashboards are compatible with more than 50 business systems, including Salesforce, Google Analytics, Google Ads, and can be easily integrated to them. In this way, all data coming from different sources can be displayed on the same screen.

Feature Overview:

− Real-time Dashboards

− Instant Notifications

− Real-time Reports

− Contests

− Achievements

− Performance Agreements

https://www.plecto.com/


+ Best Practices

02


Balance Scorecard as a Tool to Tackle Museum Challenges

Benaki MuseumAthens, Greece

CASE STUDY


BENAKI MUSEUM – THE BALANCED SCORECARD

The Benaki Museum is Greece’s second most-visited museum and its oldest private museum covering all eras of human history. It showcases various collections and artifacts in six buildings. Due to the challenging economic situation in Greece in the past, the museum was confronted with diminishing private and public donations, stalled revenues and rising costs. These were the reasons why the museum had to take action and tackle these challenges.

First of all, the museum had to analyse and define all its challenges (listed below):

− Need of defining and promoting a more focused vision and mission

− Find a common ground throughout the collections and exhibitions in order to communicate a compelling value proposition to visitors and donors

− Creation of a focused programme of exhibitions, events, publications, and education

− Search for new sources of revenue

− Attract more diverse visitors

− Improve sales of publications, events, educational programmes, restaurant, and store

− Optimal development of fundraising efforts

− Better supervision of costs and increasing governance

− Empowerment of employees

In a second step, the museum applied the Balanced Scorecard addressing its challenges in the context of the four aforementioned dimensions. It included pre-defined metrics and targets in each dimension and reinforced them with a set of clear improvement initiatives.

In the area of artistic contribution, for instance, one metric measured the number of institutional borrowers, thus ensuring to lend Benaki’s collections to internationally famed institutions. In the dimension of public benefit, the museum tried to present first-class exhibitions and measure the number of positive reviews by critics per year and visitor satisfaction ratings. In terms of learning and growth, the museum’s goal was to increase employee satisfaction and retention, as measured by job satisfaction ratings and staff turnover rates. Communication and staff engagement were driven by sharing information on current and upcoming exhibitions and programmes, messages from the curator, employee suggestions, best practices, and fundraising progress. It was essential for the museum that the employees understood the mission of the institution. Regarding finance and governance, Benaki announced two new positions for fundraising and finance management. The aim was to launch a new fundraising programme and establish budgeting and control processes aimed at managing costs and increasing transparency.

With all these measures taken, the museum was able to overcome its challenges and achieve its goals and look into a brighter future.

Source: BCG https://www.bcg.com/publications/2012/public_sector_arts_culture_art_and_business_measuring_museums_performance

https://www.benaki.org/index.php?lang=en

https://www.bcg.com/publications/2012/public_sector_arts_culture_art_and_business_measuring_museums_performance


The Balanced Scorecard in action


Dashboard Prototype for Data Collection and Monitoring

Guggenheim Bilbao MuseumBilbao, Spain

CASE STUDY


The Business Intelligence Dashboard

The Guggenheim Bilbao Museum has developed a prototype of a comprehensive monitoring system that allows the museum to have all the information on the activities performed within four different pillars: visitor, exhibitions, galleries and the shop. This is planned to be realise through an activity business intelligence dashboard that compiles several metrics helping to evaluate the activities of these four pillars.

All the information feeding the dashboard is collected from different sources of information. The ticketing cloud platform and the POS sales provide information related to dates of visit, prices, entry times and languages. Through the installation of Wi-Fi sensors, anonymous information from smartphones can be collected, as well as the localisation of the visitors in the different areas of the museum (e.g., galleries, corridors, cafeteria, etc.). Other sources such as the weather forecast can also feed the monitoring system.

The objectives of the implementation of this system are related to aspects such as: the anticipation of the visitor’s behaviour and needs; improve resources planning (staff, cleaning, etc); improve visitor experience; increase income; achieve more fidelity among the community; and compare offline and online businesses.

The KPIs integrated and monitored in the business intelligence dashboard related to each of the four pillars are included in the following screenshots.


VISITORS SECTION

VISITORS This section consists of three pages, two including graphics and one page with tables. Use the highlighted buttons to navigate through the section.

FILTERS

The existing exhibitions in the selected time period will appear here.

CARDS. In this section the data referring to the selected period of time in the filter “Date” will appear. We can also see the data for the previous day and the current year.

1. Tickets: total tickets2. Tickets / day: average of

tickets per day3. % Recurrence: percentage

of recurrence of visitors4. Average stay: average stay

in the museum (per min)


VISITORS SECTION

This matrix shows the data used in the graphics and others that add more value.

If you click on “+” or “-“ you can open or close sub-sections.


Focus mode: all graphics, tables and metrics have this option that allows us to expand the visual object and see it in full screen.

1. Tickets: Total tickets

2. Tickets / day: Average tickets per day

3. % recurring visitors: percentage of recurrence of visitors

4. Average stay in museum: average stay in museum (per min)

5. Recurring average stay and occasional average stay: in these two columns we differentiate the average stay according to the type of visitor.

6. Stay in the museum: total stay of all visitors

7. Stay in the museum / day: in this column we collect the total stay in the museum but in the subtotals the daily average appears


EXHIBITIONS SECTION

In these matrices we find the data used to generate the graphics. In this case, the first matrix is broken down by date and the second is broken down by exhibitions.

1. Visits: total visits to exhibitions

2. Visits / day: Average visits per day

3. Occasional visits / day: Average visits per day by occasional visitors

4. Recurring visits / day: Average visits per day of recurring visitors

5. Average stay in exhibitions: average stay in exhibitions (per min)

6. Recurring average stay and occasional average stay: in these two columns we differentiate the average stay according to the type of visitor (min)

7. Exhibition stay: total stay of all visitors in the exhibitions (min)

8. Stay in exhibitions / day: in this column we collect the total stay in exhibitions but in the subtotals the daily average (min) appears

9. Stay in occasional rooms / day and stay in recurring rooms / day: these two columns differentiate the total stay in exhibitions by type of visitor


ROOMS SECTION

In these matrices we find the data used to generate the graphics. In this case, the first matrix is broken down by date and the second is broken down by rooms.

1. Visits: total visits to rooms

2. Visits / day: Average visits per day

3. Occasional visits / day: Average visits per day by occasional visitors

4. Recurring visits / day: Average visits per day of recurring visitors

5. Average stay in exhibitions: average stay in rooms (per min)

6. Recurring average stay and occasional average stay: in these two columns we differentiate the average stay according to the type of visitor (min)

7. Exhibition stay: total stay of all visitors in the rooms (min)

8. Stay in exhibitions / day: in this case we collect the total stay in the rooms but in the subtotals the daily average (min) appears

9. Stay in occasional rooms / day and stay in recurring rooms / day: these two columns differentiate the total stay in the rooms by type of visitor


STORE SECTON

This section consists of three pages, two including graphics and one page with tables. Use the highlighted buttons to navigate through the section.

TIENDA

FILTERS


CARDS. In this section the data referring to the selected period of time in the filter “Date” will appear. We can also see the data for the previous day and the current year.

1. Sales / day: daily average of sales amount

2. Tickets / day: daily average of tickets

3. Average Ticket: average ticket


SHOP SECTION

In this matrix we find the data used to generate the graphics. In this case, the matrix is broken down by date.

1. Sales: total amount of sales

2. Tickets: total tickets sold

3. Average ticket: average ticket

4. Number of products sold: total products sold

5. Sales / day: daily average of the sales amount

6. Tickets / day: daily average of tickets

7. Visits / day: daily average number of visits in the store

8. Average stay in store: average stay of visits to the store (min)


Dashboards for Digital Data and Public Interaction

The Metropolitan Museum of ArtNew York, USA

CASE STUDY


Moreover, the dashboard made it possible for the museum to detect fluctuations in online traffic data based on related factors such as school calendars, new exhibitions, links from an article, or artworks going viral on social media. This also meant the identification of data sources for automated reporting that could save significant staff time and the provision of a standard presentation across the department.

Regarding KPIs, a basic mock -up was sketched (using dummy data) that enabled the department to discuss the metrics that needed to be included. Only those metrics leading to high-level conversations about the strategy of the department were selected as KPIs. Other metrics were included in subsections or sub-reports to show the performance of specific digital initiatives.

The data in the dashboard can be categorised in different ways according to strategic objectives, audiences, or usage context. It also focuses on the digital experience of a visitor into the Museum which will include data about the Met app, audio guides, Wi-Fi, in-gallery interactives, and website usage in relation to the visit.

Dashboard for digitalisation

After the COVID-19 outbreak, The MET has started to provide its services to millions of users (local, national, and global) through multiple digital channels. Thus, digitalisation has received even more relevance and the museum has started to select the online content and social media initiatives the audiences will access to (e.g., exhibitions, collections, and educational resources such as interactive performances and conversations with curators, educators, and artists).

In this context, having a clear track of the impact of all the different projects is of great relevance to allocate resources and set priorities in each department and the museum in general. Thus, the museum has started to develop and use dashboards that do not only contribute to having a clear picture of the projects, but also help to create a data culture in the organisation.

This is the case of the dashboard for digital activities. This board was implemented to track different channels (videos, social media, website updates, e-mail campaigns, blog posts, the app, audio guide, etc.) the museum is using to reach the millions of online users.

Also, using real time information helped not only to map all the reports circulating in the digital department to create a full picture accessible for all the staff. It also contributed to assessing the gaps in the data and illustrate the performance of several digital activities.

Source: The MET

https://www.metmuseum.org/blogs/digital-underground/2015/data-stories-centralized


Tracking digital activities at The Met

Source: The MET

https://www.metmuseum.org/blogs/digital-underground/2015/data-stories-centralized%20%20%5d


Interactive online and public dashboard

The MET has also implemented an online and interactive dashboard open to the public. This dashboard is updated on a monthly basis and it shows the several facets of the digital efforts of the museum (digital library services and collections and their active use; social media initiatives; digitisation; conservation; and cataloguing). Among these categories, the most relevant digital metrics of the museum are shown.

The dashboard, powered by Microsoft's PowerBI, is interactive and the data are shared among the staff.

Besides of that, the MET encourages visitors to engage with its several digital offers:

− web-based highlights (360-degree views of iconic spaces, behind-the-scenes videos, and dynamic explorations of exhibitions)

− online forums in social media for curators

− unique stories from the museum’s followers in social media

− hubs (e.g., Art at Home) for visitors to discover new and existing digital content

− events on the homepage related to the 150th anniversary and The Met Gala

− space for social justice movements (e.g., Black Voices)

Source: The MET

https://www.metmuseum.org/art/libraries-and-research-centers/thomas-j-watson-library/dashboard


Online Dashboards for an Open-Access Policy

The Smithsonian Group USA

CASE STUDY


An online accessible and interactive dashboard

The Smithsonian is an other institution, that has declared the relevance of digitalisation as part of its organisational strategy and has implemented an "open access" policy. Having this in mind, the institution has created an online and real-time updated dashboard with key metrics to communicate and align the staff working towards defined strategic objectives. These metrics are linked to individual plans to help assess the performance results and accountability across the major programmes and functions of the institution, but also for scope and reach purposes. The major programmes are:

− The virtual Smithsonian: including metrics such as open access, website visitors, social media engagement,

− Public Engagement: with metrics like visits, visitor experience, exhibitions, educational programmes, Smithsonian affiliates, travelling exhibitions,

− National Collections: including collection holdings, collection digitalisation, collection care, collections loans,

− Research: showing publications, academic appointments, sponsored projects, honours and awards,

− People and Operations: to share the employees, volunteers, facilities, sustainability and finances.

Sources: Arts & Metrics

https://www.si.edu/dashboard/virtual-smithsonian

https://www.si.edu/dashboard/public-engagement

https://www.si.edu/dashboard/national-collections

https://www.si.edu/dashboard/research

https://www.si.edu/dashboard/people-operations


A Reporting System Accessible for Museum Staff

KHM-MuseumsverbandVienna, Austria

CASE STUDY


KPIs Reporting System KPIs - Out of the box

For the Kunsthistorisches Museum (KHM), KPIs are not only tools to focus on the organisation business goals – they also serve as a bridge taking the organisations towards new and innovative goals. In this context, the KHM has created a KPIs reporting system, which is fully accessible for the museum staff through several reports. The KHM KPI system monitors metrics in different dimensions which allows to identify almost every cross reference between them. The different dimensions are quality and quantity; time and season; venue; collection; guest origin, ticket-type; revenue-type; etc. In this way, different metrics can be measured, analysed and compared among the different dimensions. For example, entrance revenue can be measured not only in terms of time or revenue type but also in terms of guest origin or ticket-type

KHM is using a performance figure triangle based on “Traditional KPIs”, “Business related KPIs” and “Outside the box KPIs”.

The Traditional Related KPIs encompass guest-related KPIs (numbers of visitors, local-tourist ratio, average time spent in exhibition, highlights and points of interest in the customer journey) and object-related KPIs (space and collection growth, research development, digitalisation, objects on display, restoration, etc.).

The Business Related KPIs encompass revenue-related KPIs (tickets, shops, fundraising events, grants, subsidiaries) and profitability-related KPIs (helping to monitor EBIDTA, EBIT, EGT, development of equity, self-sustainability rate, productivity per capita, etc.). A special KPI monitored by the KHM, and museums in Austria in general, is the self-sustainability rate. This KPI indicates the ratio between earned income and state subsidiaries. This is a highly relevant metric for the self-resilience of museums.

Finally, the KHM has started to analyse the inclusion of “Outside the box KPIs” in its report system. This group encompasses innovation-related KPIs (resilience, image-transfer, media/social media, innovation rate, etc.) and emotion-related KPIs (hmm (enjoy), wow (to impress) and aha (surprise) indicators; MOHI - the Moment of Happiness Indicator; and GOKI - Gain of Knowledge Indicator).

Outside the box

Traditional KPIs

Business-related KPIs

Mmm,

AHA!

WOW


Benchmarking + Management

+ What and Why


+ Management


+ Best Practices in Other Sectors 03



Benchmarking is what gives KPIs sense. Standing alone, KPIs can provide organisations with a reference – but without perspective (compared to internal or external benchmarks), they simply lose their organisational value and contribution to improve the competitiveness of the organisation. Nowadays, there are several tools and organisations within every industry (even with specialisation on specific operational areas) are collecting data to provide the benchmarks of the industry.

Moreover, when thoroughly implemented, KPIs and benchmarking serve as managerial tools that aim to improve the organisational performance and, for instance the quality of processes, products and services.

Thus, in this chapter the meaning, relevance, types, as well as benchmarking methods and tools are provided. We also address the cultural organisation and organisational quality, the two managerial concepts, which are highly relevant for the KPIs and benchmarking implementation. The chapter ends with some relevant benchmarking examples within the museum sector and beyond.


+ What and Why

03


WHAT IS BENCHMARKING?

Who is the best? And what makes it so successful?

Benchmarking is a management tool used to identify best practices and compare the performance levels of different areas within an organisation against internal processes (historical data) or those in other organisations. This allows to identify areas of improvement for goal-setting and the evaluation of the organisational performance related to aspects such as cost and time efficiency; the improvement of customer satisfaction; or quality of products and services. Therefore, when talking about benchmarking, not only products and services are evaluated but also the processes and operations within organisations (the latter more rarely measured in the museum world).

Organisations usually choose benchmarks based on standards (averages) or top performances in the specific industry as they set the performance levels for the rest of the actors. For this reason, benchmarking is commonly thought as an extension of quality programmes when best practices of other organisations or industries are implemented in order to gain competitive advantage.

Benchmarking can be applied through the compilation of qualitative (a process -based benchmarking asking about behaviour and practices) and quantitative data (a process performance benchmarking identifying the extent of the gap in numerical terms).

In this context, KPIs are not going to be compared against the achievement of a specific strategic goal. In this case, the comparison is made against the benchmarks or points of reference established either by other organisations or areas within the same organisation with an outstanding performance. Thus, benchmarks can be seen either as a goal (based on the industry’s standard or on top performance in the industry) or as a baseline or performance thresholds (showing the minimum level of performance expected).

In benchmarking, KPIs carry more meaning as they compare information that allows to put performance results into perspective.

“People ask me: What is your biggest competition?, Is it MoMA? Guggenheim? Our competition is Netflix. Candy Crush. It’s life in 2016”

– Sree Sreenivasan, MET’s chief digital officer


BENCHMARKING TYPES

InternalEvent though having an industry reference is of great help, every

organisation is different, even those within the same industry. Thus,

comparing to others based only on numbers might lead to wrong

decisions. In this context, it is also possible to perform an internal

benchmarking that compares similar processes (normally against past

performances) or units (e.g., retail stores, events, exhibitions) within

an organisation. The aim is to identify those areas or processes

working better and share the knowledge throughout the company.

Competitive and ExternalThis approach benchmarks an organisation against the competitors, or

even non-competitors, of the same industry (in the case of

competitive) and those outside of it (external). Thus, competitive

benchmarking focuses on comparing the performance of products and

services and the external on the possible adoption of best practices.

This helps to improve internal operations and understand competitors.

In the literature, it is possible to find several types of benchmarking. However, each organisation needs to ask itself what it wants to compare and against whom. Thus, several types and approaches towards benchmarking shall be taken.

GenericThis approach does not benchmark the business practices of a

particular organisation. Instead, it benchmarks excellent work

processes or generally accepted best standards from any relevant

industry.

FunctionalThis approach helps to discover how specific functional areas

(marketing, finance, human resource or operations departments)

perform in other companies or industries to apply best practices. In

this way, whole functional areas can be improved instead of separate

processes.

PerformanceIt benchmarks the outcome (product or service) characteristics. Those

that are quantifiable in terms of price, speed, reliability, design,

customer satisfaction, etc. It determines how strong the products and

services are compared to the competition.

StrategicIt involves the assessment of strategic matters instead of operational

ones. The goals for organisations are to identify outstanding

strategies (within or outside the industry) and to apply them to their

own strategic processes.

ProcessThe focus is to identify the best practices of those organisations with

similar activities that can be applied to own processes to improve

them. Performance benchmarking helps to identify weak competing

points on own products or services and, as a result, process

benchmarking can be used to eliminate those weaknesses by looking

at similar processes in other companies, for example, identify and

improve processes resulting on a pricier provision of a product or

service than the one of the competitor.


RELEVANT KPIS WITHIN THE FUTURE MUSEUM PROJECT

KHM

Revenue-related KPIs

tickets/entries in all dimensions; Shops (venue, per -capita, per transaction); fundraising, events; grants; subsidies

Profitability-related KPIs

EBITDA / EBIT / EGT; Development of equity; self-sustainability-rate; productivity per-capita; net-profit-calculations

Guest-related KPIs

number of visitors; visitor development - visitor benchmark; local/tourist ratio; countries of origin (= target markets); visitor/ticket structure; average length of time spent in an exhibition; highlights (Objects, Infrastructure) - Thermographic

Object-related KPIs

space/gallery growth; inventarisation/digitalisation; research; numbers of objects being on display, growth of collection; acquisitions

Swiss National Museum


Turnover; net profit/loss; net profit on average; balance sheet total; organisation capital ratio (%); liquidity ratio (%); income from museum operations commercial; income from museum operations non-commercial; income from deliveries and services; grants received; federal contribution; contribution to SNM accommodation; free capital; technical coverage ratio pension fund (%); personnel expenses; visitors per full-time position.

Guest-related KPIs

museum visitors; proportion of foreign visitors; guided tours adults; total guided tours; display equivalent value; website hits; website users; blog users; app downloads; weighted rate of in-tariffication, customer loyalty; annual tickets; own events; visitors of own events; Facebook interaction rate; magazine subscribers; newsletter subscribers

Staff-related KPIs

SNM staffing levels (actual); staff (full-time equivalents); gender rate (general, %); gender rate (cadres, %); learners; trainees; proportion of German-speaking employees (%); proportion of Latin-speaking employees (%); national languages according to personnel reporting; staff turnover rate (%)

Sustainability-related KPIs

guided tours for pupils and students; school classes not guided; total school classes; students at the study centre, new entries/registrations; university internships; learners

Expert specific-related KPIs

total consultations; information on the collections; specialist publications; teaching modules (hours), domestic loans, international loans; museums with loan requests; items on loan; prestige rentals; objects online; queries; collection online visits; collection online visitors; training modules in the study centre


Phaeno


profits; sales; revenue

Guest-related KPIs

Visitor numbers, length of stay, repeat visits, visitor satisfaction

Object-related KPIs

events; media response; use of digital media

Zeppelin


costs of exhibitions, grants, subsidies, fixed costs

Guest-related KPIs

number of tickets sold by category (adult, family, child, senior citizens, families, visitor cards, circle of friends, guests, online tickets, memberships, cooperations, combined tickets, etc.); number of guided tours (annual, monthly, weekly); group bookings; group sizes; languages (via audio guide); visitor satisfaction (via surveys and evaluation portals)

Object-related KPIs

acquisition; conservation; documentation; research and storage; loans, successful collaborations, special “out of the box” projects

Online-related KPIs

Click rates/impressions on social media sites (FB, Instagram, Twitter, You Tube), click rates/impressions for online advertising, newsletter subscription

Alimentarium


revenue, turnover (income vs costs); % self-generated income; % of online ticketing

Guest-related KPIs

attendance; audience category; audience source market (local/tourist ratio)

Online-related KPIs

press releases, web statistics (average time spent; nr pages per session; conversion rates)


Migros

Guest-related KPIs

visitor numbers

Object-related KPIs

collection acquisitions; media coverage; commissioned productions; number of publications

Munch

Financial-related KPIs

income per visitor (ticket and shop); contributions from sponsors, foundations and other private income; profit and loss KPIs (shop, café, restaurant, etc);

Guest-related KPIs

Number of visitors (digital and physical); number of children (digital and physical); number of international visitors; visitors per programme and per art experiences

Online-related KPIs

number of members and followers in social media;

Staff-related KPIs

gender and cultural diversity of the team; sick leaves

Object-related KPIs

research; publishing; conservation; number of press attending and covering (international and national);


BENCHMARKING STEPS

Planclearly define what needs to be improved and compared, assign metrics to it

Findidentify benchmarking partners or sources of information from where to collect the information from

Collectchoose methods to collect information and gather the data for the previously defined metrics Analyse

compare the metrics and identify gap in performance against competitors, the industry or internal records

Actprovide the results and recommendations on how to improve the performance

Improve develop and implement improvement plans and performance targets for products, services, processes or strategy

Reviewmonitor performance against the targets

Repeatdo the whole process again to make it a habit

Stepsto successful

benchmarking


BENCHMARKING PROVIDERS

Benchmarking helps to assess against competitors (or against other industries when appropriate metrics are selected) and to monitor performance. However, benchmarking is not as easy as it sounds, because in most of the cases the information is not publicly available or the required effort to obtain the data is very high. Therefore, it is quite common to observe that one of the major weaknesses in benchmarking studies is the lack of a proper research of relevant benchmarking organisations.

In this context, organisations are advised to use consultants, functional experts, as well as industry associations and public and private databases as sources of information in order to select organisations to compare with. Also, there are specialised performance measurement systems and platforms offering benchmarking data for almost every industry and collecting information periodically (mainly annually).

Benchmarking providers can specialise either in the global picture of the organisation in a specific sector or in specific organisational areas (facility management, social media, visitor management, etc.). They use specialised software tools and data collection methods to build the comprehensive benchmarks and market intelligence databases. These databases aggregate the information from several competitors in the sector.

The individual information the organisation provides is kept anonymous and is used only to build the overall industry or sector benchmarks. Thus, benchmarking providers also help to standardise data and make benchmarks and KPIs comparable between organisations. KPIs standardisation is of great relevance as having more comparable KPIs allows organisations (even from other industries) to learn from one another as well as to compare experiences and strategies.

The main advantage of benchmarking providers is that they can provide organisations across industries with an in-depth knowledge in the form of dashboards and/or reports of best practices. This can help organisations to achieve a superior performance, if continuous systematic processes are created to identify and rectify performance gaps between them and top performances.

Benchmarking platforms, like Baromitr, give the opportunity to create own benchmarking databases and groups with peer organisations to identify critical metrics and receive secure, anonymous, recurring benchmark insights via the platform. Moreover, the platform cares about data storing in terms of standardisation as not every organisation saves data in the same way or names. Therefore, Baromitr uses tools such as simple surveys, detailed manual entry or flat file upload to transform and standardise the data. Other benchmarking tools are illustrated later in the report.

https://www.baromitr.com/



03


SIX SIGMA METHOD

Six sigma is a series of methods and data driven approach to measure quality, improve profitability, increase customer satisfaction, eliminate waste, reduce costs associated with poor quality, and improve effectiveness as well as efficiency of operations. The methodology was developed in the manufacturing sector but nowadays it is used to improve all kind of process-oriented organisations in all sectors. The name of Six Sigma comes from the fact that quality can only be assured if a distance of six standard deviations between the mean and the nearest (upper or lower) specification limit is kept.

Six Sigma is based on a five-step process or DMAIC, the acronym for define, measure, analyse, improve, and control. To go through the Six Sigma method, it is necessary to form a representative team of employees of those areas involved in the process to improve. The five-step process can be summarised in:

1. DefineThe team identifies the issue to address through the systematic process. The purpose is to define the desired process output characteristics, the potential business impact and the goal or focus of the project.

2. MeasureThe team should develop a flow chart to describe the current process and collect the data to establish the baseline of the current performance and capability. The purpose is to validate the assumptions made about the process and the problem with data.

3. AnalyseThe team develops hypotheses about causal relationships to identify potential root causes of problems in the process under study. With the use of statistical analysis, those root causes of problems are verified. The hypothesis testing to validate assumption and possible solutions begins in this analysing phase and continues in the following one.

4. Improve The purpose of this stage is to try to eliminate those already verified root causes of problems. The team selects the solutions and begins to implement them. It also starts measuring results trying to demonstrate through further analysis that this actions have resolved the problem or are already measurable improvements in the process.

5. Control stageIn the final stage, the team should develop and implement a plan to ensure the improvements will be sustained to a daily work environment through, for example, trainings, standard operating procedures, and statistical process.

“Six Sigma is a quality program that, when all is said and done, improves your customer’s experience, lowers your costs, and builds better leaders.” — Jack Welch, General Electric’s CEO

implementing Six Sigma in the organisation

http://en.wikipedia.org/wiki/Jack_Welch


SIX SIGMA AND THE SERVICE SECTOR

The Atlanta Opera Company

The service sector is a complex range of organisations characterised by the inseparability of production, consumption and intangibility. In any sector there are highly customised and standard processes. These processes vary causing failures in services that avoid meeting customers’ needs and expectations. In average, service processes perform at an average defect rate of 3.5, or, in other words, 23,000 faults per million actions; something unacceptable in most manufacturing environments.

In this context, the arts and culture sector provides significant opportunities for applying these methods in order to boost efficiency and quality through process improvement. On the other side, even though the Six Sigma method has been based on successful implementations by large firms, it has shown to be equally beneficial for smaller organisations. However, its implementation requires a different deployment model that considers the limited resources available for these organisations.

That is the case of the Atlanta Opera Company, which partnered with MBA students from the Georgia Institute of Technology to examine a subscription ticket sales decline. The Atlanta Opera Company recently moved from the historic Fox Theatre where it had performed since 1995, to the modern Atlanta Civic Centre. By applying the Six Sigma approach, and through a survey addressed to the opera patrons, it was possible to identify that the relocation to the newer facility was playing a significant role in the decision made by the opera patrons to not renew their subscription. As a consequence, the recommendation was to improve subscribers’ social experiences, to promote subscribers’ benefits, and to publicise the advantages of the newer facility. As a result, the Opera improved its patrons’ social experience through catered dinners, initiating a subscriber reward programme providing discounts and special offers, and improving concession facilities.


BENCHMARKING PITFALLS

Lack of context

The benchmark data are commonly compared without taking into consideration their short-term and long-term context. They tell what a competitor or internal team achieved, without information on how they did it. This makes it difficult to identify areas for improvement.

Measurement issues

Like-for-like comparison is difficult when not everyone measures things in the same way. For example, NPS needs a scaling system to measure customer satisfaction, which is usually a 0-10 one. However, some surveys can use a different scale (e.g., the scale of 1 to 7 in Germany), which can distort the interpretation of results. Moreover, certain metrics can have comparing issues due to accounting expectations(e.g., profits).

Finding comparisons

Although it is getting easier to find data, it is not always the case in many industries. Many competitors do not want to share their data and this can force organisations to compare themselves against those outside the industry or to focus on easy-to-find data. Organisations need to be sure the chosen KPIs are reported by most of the benchmarking partners and data have been available for more than just one year.

Finding reliable data

Platforms aggregating data (e.g., TripAdvisor or Booking.com for ratings) provide an easy way to access competitive benchmark data. However, they are not always reliable and are available across every industry.

Benchmarks look backwardsBenchmarking only tells what happened but not what is occurring in real time nor predicts future performance. Thus, past performance cannot assure future results.


FINANCE COUNT

Benchmarking museums’ financial performance

The Association of Independent Museums allows its members to benchmark their financial performance for free through its tool “Finance Count”. The tool is a tailor-made benchmarking service developed in partnership with Agenda Consulting and Crowe which includes the development of questionnaires and scorecards. In the tool, organisations can select which museums they would like to benchmark against.

As a result, organisations receive a unique scorecard, as well as a report with a deep dive-into the sector-wide findings for a clear picture on exactly how the museum compares to others. Besides that, participating members also receive a sector-wide and detailed statistics report; access to the database for 18 months to review results; personalised consulting with the Agenda Consulting company to help them interpret their scorecard and use the findings; training and workshops with ways to improve finance processes and performance, as well as key trends.

Sources: Association of Independent Museums / Charity Finance Group / Crowe

https://www.aim-museums.co.uk/benchmark-museums-financial-performance-free/

https://www.cfg.org.uk/financecount19

https://www.crowe.com/uk/croweuk/news/finance-count


BENCHMARKING AUDIENCES

Audience Finder Community - Benchmarking audiences

Audience Finder is a free audience data and development programme enabling the access, comparison and analysis of cultural audience insights. The audience data is powered by several cultural organisations and gathered through ticketing systems and surveys. It provides personalised dashboards that can be compared to national benchmarks.

The benchmarking services provided by the Audience Agency’s dashboards allow to compare productions, seasons or specific audiences. It also helps to understand the different motivations for new and returning visitors, gain insight into the behaviour of different visitor types and compare visitor profiles for different productions and exhibitions.

The Audience Finder allows the creation of tailor-made benchmarking groups and works in collaboration with other organisations to collect data. It also permits the inclusion of additional questions to the standardised basic survey for better comparability and benchmarking.

Data should be collected for a minimum of one year to ensure a representative sample of the audience and allow yearly comparison. With the data obtained from ticketing systems, it is possible to gain metrics for events such as: tickets issued; average yield per transaction; percentage of first-time bookers to the venue; audience profile; booking lead time; group size, etc.

Source: The Audience Agency https://audiencefinder.org/

https://audiencefinder.org/


+ Management

03


ORGANISATIONAL AND MEASUREMENT CULTURE

Organisational culture It encompasses the norms, values, behavioural patterns, beliefs and goals broadly shared in an organisation. For instance, it is a highly influential factor in the organisational performance. If an organisational culture is established, led by example and commitment from each management level, organisations perform better financially and have higher rates of employee retention and satisfaction. This encourages employees to seek for guidance, having an open communication within organisations when facing ethical dilemmas and take decisions consistent with the organisational goals.

In order to achieve an organisational culture, it is important to:

− Establish a code of conduct in accordance with the organisational culture (place the responsibility of its effectiveness in the hands of employees),

− Reward conduct in accordance with the organisational culture (what gets rewarded gets exceeded (no monetary)),

− Ensure consistency between policy and organisational practice (organisational practices should be consistent with its policies),

− Provide ethics training and advice,

− Protect whistle-blowers (motivate people to take action to expose and stop conducts contrary to the organisational culture).

Measurement cultureIn the establishment of an organisational measurement culture, KPIs emerge as highly helpful, since they reinforce the employee morale. This is because KPIs acknowledge employees hard work, guarantee their feeling of responsibility and provide a sense of ownership in their work when goals are achieved. This makes employees more likely to push themselves to go the extra mile in order to experiment a satisfaction reward. The questions to ask are: How to achieve KPI’s acceptance? How to make my employees accountable?

Controlling KPIs with the use of real-time dashboards and sharing them with the staff allows teams to see how exactly they perform at any given moment. This means a daily learning process. In a measurement culture organisation, the indicators are found, suggested and owned by the employees and not created from top to bottom. The focus is to make the staff understand how they are helping the organisation.

Asking the following questions can help to create a continuous dialog in the organisation between management and staff: What are you going to do to execute this plan and achieve this goal? How will it help our organisation? What do you need to succeed? How will you know that you achieved expected results? How can you quantify this? In order to establish a performance measurement culture, it important to follow certain steps. a) analyse patterns of current measurement culture to understand if something needs to be changed; b) tell success stories of how a measurement system helped organisations in the past. C) create documentation with key principles and its explanations; and d) be transparent on the KPIs being used and show their performances.


MEASURING ORGANISATIONAL CULTURE

Culture acceptance rate

= total number of employees/number of employees finding the organisational culture positive, an anonymous survey can provide the insights.

Employee turnover rate

= number of separations/average of employees, as the culture exists because of the employees who carry and practice it.

Employee NPS(Net Promoter Score)= % of detractors - % of promoters, it helps to

understand whether current staff would recommend their workplace to friends or family as a positive place to work.

Employee and manager satisfaction score = positive responses/total number of responses * 100, to show how satisfied workers are with the way they are being guided and managed.

In order to measure the culture of an organisation at a given time, it is possible to think about KPIs such as:


QUALITY CULTURE

Measuring the perception of quality is in itself a way to the performance improvement in an organisation. However, quality assurance is not only about having indicators and reporting them. Instead, the idea is to understand the processes in the organisation and to prevent possible quality issues. This means that quality culture is achieved if there is a recognition among all organisational levels that quality should not be met for a compliance, but for its own sake. Therefore, quality management is the area of organisations where benchmarking is commonly and particularly used for.

In this context, the implementation of a quality culture is the establishment of new behaviours and habits (e.g., making regular analysis of processes, quality challenges, reasons of challenges and possible solutions) in an organisation to avoid critical problems and maximise problem-solving effectiveness. The implementation of quality frameworks make it possible to create a systematic approach that adjusts the organisational systems and processes to the challenges of quality.

The relevance of involving all stakeholders in quality procedures and quality strategy discussions lies in the fact that achieving quality culture is a job for every person in the organisation. Some considerations to implement an organisational quality culture are:

− define outline company values

− train employees in quality culture

− pursue quality instead of chasing compliance

− implement document control

− seek visitor feedback

Quality Culture in Museums In many industries quality tends to be no priority. Moreover, quality management costs are not a minor aspect of why museums can be resilient while adopting quality models as well. In this context, the implementation of a quality culture is also about the restructuration of concepts and beliefs. For example, visitor quantity is usually at the expense of visitor quality, as the more people are in the museum, the shorter is the period people spend in front of an exhibit. This hardly can be called quality. Thus, it is common to see that the quality of management not necessarily coincides with the qualities perceived by the visitor.

The quality culture in a museum can be seen from two perspectives: the public and the private. From the private perspective we could include aspects such as the range and importance of collections; the way they are conserved and the efficiency of their indexing and cataloguing; or the degree of knowledge of the museum staff, among other.

In terms of public quality, we talk about the degree to which a museum meets the needs and wishes of its visitors and provides them with comfort and convenience. In this context, question such as “is the museum immersing itself into the local community? or is it there a visitor research as the way to know and meet people’s real needs?” become relevant. Although, it is important to consider that when evaluating, for example, the collection quality, the visitor’s perception will be strongly dependent from aspects such as the way of education, previous experience and personal background

For Kenneth Hudson, the feel-good factor is the base for the public quality in a museum. This is, if someone feels good after queuing for two hours to see a Picasso exhibition, that is his or her own way of education, previous experience and personal background. This can have little to do with the exhibition quality.

Source: Quality in Museums

http://www.culturalinformatics.org.uk/sites/culturalinformatics.org.uk/files/quality.pdf

http://www.culturalinformatics.org.uk/sites/culturalinformatics.org.uk/files/quality.pdf


TOTAL QUALITY MANAGEMENT

The concept of Total Quality Management (TQM) has been raised as a cross-point between company and public services management. This model has moved from the private sector to the public sector and to cultural organisations such as museums. It consists in a set of concepts, methods and management techniques, aimed at creating and maintaining cultural and working conditions to get excellent results.

This can be made possible through the:

– pursuit of the organisation’s mission and customers’ satisfaction,

– rational use of resources,

– stakeholders' satisfaction (financial investors, clients, personnel, social and professional communities involved by the organisations’ activities)

However, each museum needs to evaluate if any model, not just the TQM, is adequate to its needs or if the organisation is ready to adopt it. In Europe, the European Foundation for Quality Management (EFQM) has also created its own quality model (EFQM Model) for several industry and services, among them museums. The model has a self-evaluation based on 9 Criteria: Leadership; Human Resources Management; Policies and Strategies; Resources (financial, informative, etc.); Processes; Outcomes on personnel; Customer satisfaction; Impact on community, territory, etc.; and Key-issues in organisations (instead of performance outputs). Even though this tool has been adopted by several museums (Guggenheim Bilbao, EFQM), other museums in Europe have preferred to adopt systems based on minimum standards to improve the quality of their work.


HOW TO MEASURE QUALITY

Visitors

NPS: It indicates a general perception of the product or service and a low score is likely to be connected to a poor product or service quality.

Number of complaints per period of time: the idea is to understand and fix problems in the business systems and not in individuals.

Social proof of quality (average rating score): evaluates the clients’ rating of a product or service.

Complexity metrics: including time (how long does it take to perform a certain operation?); number of components/connections between components/levels to deal with from a product or service.

Financial impact

Cost of poor quality: these are all costs associated with problem detecting and solving, including brand impact, customer support and solution costs.

Cost of high quality: achieving high quality also has a cost and it is necessary to understand if achieving it might be more costly.

Prevent quality problems

Near miss metric: measured through the number of near miss issues or events reported or (%) near miss awareness training penetration in order to discover minor quality issues that might lead to more important ones.

Quality self-test metric: to revise how the quality control system works and can be measured taking into consideration the number of self-tests per period and the % of defects detection efficiency.

Quality response

Time to solve quality issue: it is always a WOW experience when issues are promptly solved, and it can be measured with the average time needed to solve the critical/minor problem or time to develop/implement prevention plan.

Time to respond to quality issue: it serves to evaluate not the solving of the issue but the time the organisational team takes to care.

Solution effectiveness: it examines the % of returning problems.



03


CREATIVE SCOTLAND AND THE AUDIENCE AGENCY

Data collection and benchmarking standardised systems

Creative Scotland is a public body supporting arts, screen and creative industries across all Scotland. It aims at enabling people and organisations to experience these industries by helping them to develop ideas and bring them to life. Creative Scotland has partnered in 2020 with The Audience Agency (TAA) to introduce a system of standardised audience data collection, sharing and benchmarking for regularly funded organisations (RFO) in Scotland working for the industries previously mentioned.

The Audience Agency provided data and analytical services to support the audience development through its Audience Finder system. The services include:

− an easy-to-use dashboard helping the organisations involved to better understand their existing audiences and compare them with to broader group,

− a detailed audience segmentation analysis, using MOSAIC Scotland and Audience Spectrum, to segment the UK population by their attitudes towards culture and by what they like to see and do,

− the possibility to benchmark with other institutions in Scotland and the UK.

The participating organisations could engage with the Audience Finder through their ticketing systems or through a standardised survey. For this purpose, TAA platform in Scotland was tailored to the needs of the organisations in Scotland by engaging participants in defining the requirements.

Source: Creative Scotland https://www.creativescotland.com/resources/professional-resources/other-sources-of-support/the-audience-agency

The data included in the Audience Finder system was anonymised and the RFO retained full ownership. Thus, it served for benchmarking purposes only. Those organisations providing data into the system had access to the resulting analytics through a dashboard.

As a result of this collaboration, it is expected participating organisations can start to identify more easily potential for more and diverse audiences, better access, more revenue and greater efficiency. In terms of benchmarking and data collection, standardisation can provide a better picture of Scotland’s cultural sector as a whole and allow sector development organisations to better understand their public.

The ability to compare audiences’ metrics in groups based on geography, sectors, or collaborative groups, such as in the Creative Scotland project is fundamental for data analytics.

https://www.creativescotland.com/resources/professional-resources/other-sources-of-support/the-audience-agency


ACM’S BENCHMARK CALCULATOR AND KPIS

ACM’s Online Benchmark Calculator and menu of KPIs for the children’s museum sector

Benchmarking Calculator creation process In response to the expressed need by the museum members of having a tool through which they could inform about their institutional planning and enable data sharing, the Association of Children Museums (ACM) created an online Benchmark Calculator of 19 reports covering 27 KPIs. This was achieved with the collaboration of the White Oak Institute (WOI) and the Institute of Museums and Libraries Services (IMLS) where each report could compare up to 12 members of the ACM.

The White Oak Institute developed and refined the drafts of reports and KPIs based on the ACM reviews and the input of the members of the ACM through online surveys where they were asked about the most important KPIs and selection parameters.

The aim of the benchmark calculator was to share and interpret museum data efficiently in order to help museum professionals (museum leaders, managers, board members and those participating in the museum’s organisational assessment and planning) to drive sustainability and growth in their institutions by measuring their performance against their peers in the ACM.

The ACM members can generate customised comparable reports, which can be filtered by geography or by operating parameters in the museums.

As a result of the surveys, the main five KPIs were:

1. Admission visits by category (e.g., school, members, free and general public),

2. Percentage of total expenses for selected expense categories,

3. Admissions revenue as a percentage of earned and total revenue,

4. Revenue by category (e.g., earned, private support, public support, endowment/interest income),

5. Personnel expenses as a percentage of total expenses.

Source: The White Oak Institute http://www.whiteoakassoc.com/pdf/Bulletin2-Core-Narrative.pdf


BENCHMARK FOR MUSEUM FACILITIES

The International Association of Museum Facility Administrator Benchmark

The International Association of Museum Facility Administrators (IAMFA) has developed an annual benchmarking programme designed for those in charge of managing the facilities in museums and cultural institutions. More than 130 institutions from nine countries participate in it. It consists of two pillars: a survey and workshop.

SurveyThe online survey seeks to measure performance in key areas of facility management and identify best practices and industry trends. It measures costs of several aspects (e.g., utilities, building operations, building and grounds maintenance, security, etc.), space utilisation, ratios of number of workers, outsourcing, sustainability measures, strategic planning, emergency preparedness planning, project management processes, temperature and humidity, performance-based contracting. Participants need to update their data for the most recent budget year in the system. Checks for data errors are run and progress reports for review are prepared. The final report is presented after edition and refined to the content by the steering committee composed by representatives from museums and cultural institutions across the globe.

WorkshopThe IAMFA also provides a workshop with roundtable discussion on facility issues, best

practice presentations, data review, “Hot Topic” forums where questions are brought for group input and practical take-aways.

Source: The IAMFA https://www.iamfa.org/benchmarking

https://www.iamfa.org/benchmarking


+ Best Practices in Other Sectors

03


ALVA - THE ASSOCIATION OF LEADING VISITOR ATTRACTIONS

The Association of Leading Visitor Attractions represents over 50 historic sites in the UK including museums, galleries, palaces, castles, cathedrals, zoos, historic houses, heritage sites, gardens and leisure attractions. It is a network which encourages members to share information with each other and to learn from trends and insights provided by both, members within the network and external experts. Among the association’s fields of action are training, advocacy initiatives and improvement of visitor experiences but also benchmarking.

The association teamed up with BDRC Continental, UK's largest independent research consultancy, to manage and deliver a visitor experience benchmarking scheme whose goal is to evaluate the visitors’ motivations, their experiences and preferences as well as their willingness to recommend the visited touristic site. The collected data are measured against all 79 scheme participants enabling insights into which of the members are particularly successful in which key performance area. In doing so, the association found out that the seven key motivations for visitors are:

- Social Mindset- Child Engagement- Big Kids- Tick Box- Special Focus* (80% of UK visitors are motivated by a special exhibition, event, performance or activity. 46% check out the attraction’s website before their visit.) - Topic Interest- Broadening Horizons

Besides, the evaluation of key performance indicators in the field of visitor experience, financial aspects are also measured such as average spend per visitor, visitor income, social media followers and engagement, operation costs as well as retail and catering across different attractions sectors. ALVA’s financial benchmarking survey has been conducted for 19 years via an online questionnaire. The results are gathered in a report containing the benchmark analysis as well as providing interpretation and context and a three-year trend analysis. Participants in the survey benefit from being able to compare their financial performance with others in the same sector, enabling them to identify areas of outstanding achievement and those that present a challenge for the future.

A special method of ALVA to evaluate visitor experience is through the eyes of another. The Mystery Guest scheme gives members the opportunity to visit each other and critically assess the visitor experience. Guests are equipped with a comprehensive questionnaire which rates the attraction based on a number of criteria: planning the visit, car parking, welcome, catering and retailing, lavatories, visitor care and safety, comprehensive questions on accessibility for groups with special or particular needs.


ALVA EXPERIENCE BENCHMARKING TOOL

Benchmarking the Visitor Experience in Leading Attractions

The Association of Leading Visitor Attractions has run the Visitor Experience Benchmarking Survey for almost 20 years. More than 80 visitor attractions participate in it and it conducts more than 25,000 visitor surveys annually.

It measures the “Experience Intensity Score”, a measure that brings together three dimensions of the overall visitor experience:

− Service Delivery (comprising dimensions such as cleanliness, catering, retail and orientation)

− Site Content (comprising the added value experience of staff, audio guides and others)

− Emotional Impact (comprising atmosphere, social, relaxing or welcoming)

Using statistical modelling, a weighted level of importance is attached to each of the experience dimensions. This helps to reflect their influence in driving the overall visitor experience at the attraction.

The tool is also helpful to identify the most important aspects driving the overall opinion of the visit and establish priorities for improving the visitor experience. Also, the tool can be used as a visitor summarising tool, to understand visitor motivations and analyse communications effectiveness.

Source: BVA BDRC

Service Delivery%

Emotional Impact

%

ALVA Experience Intensity

Site Content%

https://www.bva-bdrc.com/projects/benchmarking-visitor-experiences/


THE HOTEL SECTOR

Benchmarking tools and KPIs

The hotel sector is perhaps one of the industries where benchmarking is most developed and has a key role in quality management. Several managerial tools have been developed in the industry not only to measure the effectiveness of strategic planning or marketing activities, but also to facilitate an organisation’s benchmarking process. For example, the strategic planning index measures tree dimensions: effectiveness (occupancy percentage, average room rate, and growth in sales per room), efficiency (return on investment and profit margin) and adaptability (number of successful new services/products introduced and % of sales accounting for new services/products).

On other aspects, the International Hotels Environment Initiative, has developed the “Environmental Benchmarking Tool”. Accessing to benchmarkhotel.com, hotels can monitor aspects such as their energy management, freshwater consumption, waste management, wastewater quality, purchasing programs, community relations and bio-diversity improvements. This allows managers to compare the environmental performances against hotels with similar facilities in three major climate zones, and to design a programme to reduce their costs and environmental impact.

The Austrian Hotel and Restaurant Panel AHRP collects financial information annually on a confidential basis from between 1,000 and 1,300 hotels and restaurants. This allows the industry to exchange data on business operations on a global basis and benchmark individual performances.

The sector has also developed uniform benchmarking systems (e.g., USALI) allowing hotels to be able to compare their metrics and divide them in a way they can be compared one to another (e.g., by departments).

The finance aspect for the hotel sector is perhaps the most critical. Thus, the great majority of benchmarks and KPIs tend to focus on aspects such as revenues, costs, products and operations. Here, the most used KPIs for benchmarking in the hotel sector for sales and finances are stated:

− Room occupancy (%) = room sold/available rooms

− Average daily rate (ADR) = room revenue/rooms sold

− Average room rate (ARR) = room revenue per period, segment, or booking channel/rooms sold during that period, segment, or booking channel

− Revenue per available room (RevPAR) = rooms revenue/available rooms

− Total revenue per available (occupied) room (TRevPAR and TRevPOR) = sum of all revenues/available or sold rooms

− Gross operating profit (GOP) (and GOPPAR) = total revenue - operating expenses (/number of rooms)

− Occupancy rate = total number of rooms occupied/total number of rooms available *100

− Average length of stay = total occupied rooms nights/number of bookings

− Cost per occupied room = gross operating profit by total rooms available

Source: Wober, K. Benchmarking for tourism organisations


BENCHMARKING TOOLS IN OTHER SECTORS

Brandwatch Audiences Benchmarking Tool – Measuring Performance in Social Media Channels

Brandwatch is a software tool collecting information from posts, comments and conversations happening in different social media channels. The data are collected from over 300 million people, combining aspects such as bio, content, interests, professions, gender, account type, location, etc. The aim is to provide relevant insights of people perceptions towards brands.

Its Audience Benchmark Tool allows the benchmarking of the online awareness and marketing performance of brands. It provides real-time insight into the size of the online presence of brands, as well as on information about where, when and how customers talked about them. Furthermore, it allows to measure the customer experience through the perceptions and sentiments expressed in those post, comments and conversations. All this information can be later compared with other competitors.

The main insights this Brandwatch Benchmarking Tool provides are:

− Share of voice: amount of mentions and change rate over time

− Response rates: amount of replies per post and change rate over time

− Sentiment context: percentage of positive/negative mentions

− Audience’s interests change: main interests and changes over time

− Audiences’ major trends: main industry trends and changes over time

https://www.brandwatch.com/

https://www.brandwatch.com/blog/introducing-audience-benchmark-reports/


CONCLUSION

This report contributes to the knowledge on finding relevant KPIs and benchmarking processes for museums. Even though the list of metrics in each industry can be extensive, the focus was given into providing the knowledge, tools and methods museums should consider when selecting, developing, measuring and/or monitoring metrics and turn them into KPIs. When properly aligned with the organisation strategy, KPIs play a highly relevant role in goals achievement. Success stories, both in and outside the museum sector, have helped us to understand the relevance of creating the framework in which KPIs can push the organisation to successfully achieve those organisational goals.

Although the decision about which metrics become KPIs is in the hands of each museum, several metrics were presented for key areas (finance, exhibitions, social media, etc.) and for relevant trends (e.g., visitor experience, sustainability, social distance, or emotions) in the museum.

Tools such as dashboards or balance scorecards have also shown to be relevant in providing a better organisation and control while monitoring KPIs, as well as for data collection and clustering. Business analytics platforms have developed powerful tools for these purposes and many of them have been already implemented in the museums sector. Specially valuable are the efforts in museums to provide real-time online dashboards for public access.

On the other hand, benchmarking is the reason behind selecting and monitoring KPIs. Without comparison these figures can lack of meaningful value for organisations. In this context, benchmarking platforms emerge as in depth-knowledge providers that build up benchmarks from their databases in specific sectors. Benchmarking providers can save organisations great efforts and help them to avoid pitfalls related to data reliability or competitors comparison. Also, we learn about the benefit of the establishment of benchmarking methods (e.g., Six Sigma) in organisations to benchmark and tackle aspects related to quality, profitability, customer satisfaction, waste, costs reduction and effectiveness and efficiency in operations.

Finally, we addressed the relevance of creating a quality and measurement culture within organisations. These are a relevant pillars on which KPIs, dashboards, scorecards and benchmarking methods can be sustained and successfully implemented in the working environment. This is possible due to a higher staff commitment and feeling of responsibility, the avoidance of critical problems and the maximisation of the effectiveness in problem-solving.


This report was produced by MUSEUM BOOSTER

EditorsArturo MeloSofia Widmann

Input byMonika MairhoferOlga Kucheruk

WHICH KPIs ARE NEEDED FOR BENCHMARKING AND MANAGEMENT …

Documents

Transcript of WHICH KPIs ARE NEEDED FOR BENCHMARKING AND MANAGEMENT …