Rate the Raters Phase Four The Necessary Future of Ratings ... · Phase Four 2. Rate the Raters. is...

15
Rate the Raters Phase Four The Necessary Future of Ratings July 2011

Transcript of Rate the Raters Phase Four The Necessary Future of Ratings ... · Phase Four 2. Rate the Raters. is...

Rate the Raters Phase Four The Necessary Future of RatingsJuly 2011

Rate the Raters Phase Four 2

Rate the Raters is a four-phase research program in which SustainAbility is working to better understand the universe of corporate sustainability ratings in order to influence and improve their quality and transparency.

We again express our sincere thanks to the ratings organizations which participated in phase three of Rate the Raters, and particularly to those which participated in the phase four workshops in April 2011.

We also thank the sponsors of Rate the Raters — Abbott Laboratories, Autodesk, Brown-Forman, Cisco, Coca-Cola Enterprises, ExxonMobil, Ford Motor Company, GlaxoSmithKline, Ingersoll-Rand, Novartis, Sara Lee, Schlumberger and Scotiabank — whose support made this project possible.

Our gratitude extends to our advisory panel members for their expert guidance throughout the project: Seb Beloe (Henderson Global Investors), Suzanne Fallender (Intel), Allen White (Tellus Institute) and Mike Wallace (Global Reporting Initiative).Lastly, we thank the literally hundreds of individuals who have shared their perspectives informally over the last year by phone, email, in-person, on sustainability.com and elsewhere.

Rate the Raters

Michael SadowskiDirectorSustainAbility Inc.+1 718 210 [email protected]

Alicia AyarsAnalystSustainAbility Ltd.+44 20 7269 [email protected]

Kyle WhitakerManagerSustainAbility Inc.+1 202 315 [email protected]

Mark LeeExecutive DirectorSustainAbility Inc.+1 415 944 [email protected]

In May 2010, SustainAbility launched Rate the Raters in an attempt to better understand the universe of corporate sustainability ratings and to improve the quality and transparency of such ratings. We did this with some precedence — we published Screening of Screening Companies in 2000 and Values for Money in 2004 — and with a sincere belief in the importance of sound ratings.

Over the last decade, there has been a significant proliferation and diversification of ratings. While this has certainly led to confusion and sometimes even griping amongst companies, users and other stakeholders, this is also an indication that ratings are going mainstream. Companies are linking executive compensation to performance on ratings. Asset managers are leveraging sustainability ratings and research in their investment decision making. And citizens and consumers are starting to pay greater attention to sustainability information on products and companies. As said in Rate the Raters phases one through three, we launched Rate the Raters because we see great promise in this mainstreaming, yet we observe too many ratings failing to live up to expectations.

Our research has allowed us to explore key trends and changes in ratings over the last decade (phase one), to inventory over 100 ratings globally (phase two) and to dive deep to understand how 21 prominent ratings work in practice (phase three). The issues we raised, challenges we identified and our constructively challenging approach have contributed to some initial improvements in how ratings work (as evidenced by feedback from raters and companies). Yet, we are mindful of how much more effort is needed as our research has also given us a keen perspective on where and how ratings need to change most.

In this phase four paper we have elected, in part based on the counsel of our advisory panel and sponsors, to succinctly present a vision for the future of ratings along with concrete recommendations for change. We do this because we have already put forth more general elements of strong ratings in previous phases, and we want to make this final paper more actionable.

3Rate the Raters Phase Four

Overview

Date

April 2011

April 2011

June 2011

still in development

still in development

Ratings

FTSE4Good ESG Ratings

STOXX Global ESG Leaders Index family

Environmental Investment Organization ET Index Series

Access to Nutrition Index™

ULE ISR 880

Sample of new ratings since the start of Rate the Raters in May 2010

Figure 1

The perspectives shared in this paper are our own, informed by the many hours over the last year that we’ve spent researching ratings and the hundreds of conversations we’ve had with different stakeholders. In particular, we gained a variety of important insights and perspectives informing this output through two workshops we held with raters, companies and others as part of phase four. We thus give particular thanks to the participating organizations in these workshops (see Appendix).

While Rate the Raters officially concludes with this document, we will continue our research and advocacy in the ratings space. We are particularly interested in the degree of impact and influence ratings have on stakeholder decision-making, the drivers of credibility and usage of ratings, the sustainability of rating business models, the adoption of ratings by stock exchanges and efforts to standardize ratings.

We welcome your feedback on this paper and encourage you to share with us your thoughts on aspects of the ratings agenda that warrant further research.

Rate the Raters Phase Four Overview 4

Look Back and Current State

April 2010 —May 2010

Examine evolution of the ratings agenda over the last decade

Taking Inventory of the Ratings Universe

June 2010 —September 2010

Inventory over 100 ratings and identify key themes and patterns

Uncovering Best Practices

October 2010 —February 2011

Analyze 21 ratings to understand best practices and areas for improvement

The Necessary Future of Ratings

February 2011 — July 2011

Phase One Phase Two Phase Three Phase Four

Dates

Objective Articulate our vision for the future of ratings

Figure 2

Project Phases

5

Many of today’s ratings today are not financially viable, even short term. In phase three we asked whether there was a “right” funding model to ensure objectivity and pointed out that the predominant model today is “client pays” (e.g. investors buy sustainability research, consumers buy magazines). While the “client pays” model may be right for some types of ratings in the future, we know that there is currently too little demand and funding for the number of ratings that exist today. This has contributed to the consolidation we’ve observed in the market (e.g. MSCI’s aggregation of several research firms), and has nudged raters to push the envelope in their marketing efforts (e.g. selling services, advertising).

Going forward, the client pays model will likely prevail, as alternatives present even more dilemmas (with the financial crisis, we’ve experienced the worst of what can happen with a “rated-company pays” model). Raters will continue to test ways to diversify income, including providing advisory services (which we believe can be done without compromising the integrity of ratings so long as there is strong disclosure and separation of the ratings and advisory businesses).

There will be greater demand for fewer, higher quality ratings in the future. At the risk of being caught up in the hype we often see around consumer and investor interest in sustainability, we see enormous potential for ratings to better meet the needs of more diverse audiences. Taking the investor segment as a means of illustration, the growth in investor interest in and commitment to sustainability is astounding — the UN Principles for Responsible Investment now have nearly 900 signatories representing $25 trillion in assets under management. As these signatories move from commitment to implementation, they will need robust research and ratings to help determine how to evaluate the sustainability performance of companies in their portfolios.

We will likely see further growth and diversification in ratings over the next few years, particularly in relatively new markets (e.g. India), as organizations continue to experiment with new approaches and business models. Ultimately the market will settle on a few “winners” in different ratings types. For example there will be a small number of consumer-facing, broad sustainability rankings, a few water ratings, etc.

Quality will be the key driver in determining which ratings survive, and those which do will have greater resources to invest in people, training, quality management and other areas that will result in better ratings (thus creating a virtuous circle).

In phases one through three, we shared hints of our vision for robust, credible and sustainable ratings. We have aggregated and expanded on this vision below. Throughout this section we include anonymous quotes from participants in the April 2011 workshops. In our view, ratings will be:

Rate the Raters Phase Four

What Ratings Must Be in the Future

Financially Viable

“People aren’t paying what this information is worth, so raters are scratching around for different sources of revenue in the same pool.”

“Rather than perseverating over the need for common criteria, quality standards, etc., we (raters, companies, consultants and others) should join together to dramatically boost demand for ratings. Greater demand will help resolve many of the issues we’re talking about today.”

Fewer and in Greater Demand

6Rate the Raters Phase Four Approach

In the phase four workshops we convened in April 2011, there was considerable and energetic conversation around the fact that raters currently compete on the most commoditized part of the value chain — data compilation. Raters each use their own criteria and questions, which is inefficient for all parties involved. We contrast this approach with that of equity analysts who all use the same basic inputs (e.g. data from regulatory filings) yet compete in their analysis of companies.

In the future, we envision a more clearly segmented ratings value chain (see illustration in Figure 3), with a small handful of players specializing in data compilation, others using this data to compete on research and analysis, and others still competing on packaging this research and analysis into rankings and products or services. This specialization will bring greater efficiency (for raters and companies) and will allow the different players to hone expertise in distinct arenas. It will also foster competition in research and analysis, which from our perspective is the most important link in the value chain.

Competing on Analysis not Data Collection

“We need to stop competing on the collection and aggregation of data and instead consolidate our resources. Maybe you have one organization looking at carbon emissions, another looking at water, etc. This would reduce the burden on companies but it would also give raters more time for analysis.”

Data Compilers

Companies Research Firms

Raters & Rankers

Users

Value Increases through the Chain

Figure 3

Ratings Value Chain

Gather, via deskresearch andsurveys, standardsets of data on companies using accepted frameworks (e.g. GRI, CDP)

Thoroughly reporton sustainabilityefforts usingaccepted frameworks (e.g. GRI)

Using inputs fromcompilers and select proprietary criteria, analyze company sustainability performance based on proprietarymodels

Assign ratings or rankings tocompanies based on data compilationand research

Roles Investors and consumersmake decisionsbased on ratings or rankings

7

The ratings that thrive in the future will be those that add the most value to all parties involved in the ratings process (i.e. companies and clients). Raters depend on the participation of and disclosure from rated companies to exist. With companies being inundated by ratings, they will increasingly focus their attention on those ratings which bring the greatest insight and best recommendations — that is, the ones that add value to their work. And, to be clear, packaging company-disclosed information and giving (or selling) that information back to companies is not value-adding.

Ratings must bring new ideas to users as well, for example, giving consumers valuable new information to help guide their purchases or bringing a different investment angle to an asset manager. In the phase four workshops, we had a rigorous discussion around the potential for ESG research firms to be more predictive in their analysis — to identify emerging issues and trends for companies. While we recognize forecasting can be challenging, we maintain that this is how raters will likely generate the most value, for rated companies and for investors and other users.

While sustainability ratings which are more future-focused will bring value to companies and users, they cannot do so at the expense of focus on more immediate and core company operational issues and impacts. We currently observe too many sustainability ratings that pay insufficient attention to core business issues such as ethics, compliance, risk management and financial performance. This is why we see “best of” rankings populated by companies which have experienced significant ethical lapses, health and safety problems, and product recalls. In the future, leading ratings will have at their core these operational measures, as it is these measures that determine whether a company will last — companies with repeated ethical breaches or health and safety incidents won’t be in business for long.

In addition to being centered on these core operational measures, ratings of the future will be designed to evaluate the most material sustainability issues for given sectors. As we said in phase two of Rate the Raters, “…it is always difficult — and meaningless in some cases — to rank companies across sectors and geographies on the same set of issues.” Comparing an investment bank against a food company versus an energy company on climate change is not meaningful — each has unique risks and opportunities. We thus believe that ratings going forward will not be universal or standard across sectors.

Rate the Raters Phase Four

Value-adding and Predictive

“We shouldn’t underestimate the capacity for driving change through an organization simply by asking the right questions. Yes, ratings have been irritating, but they have forced us to make changes that have been fundamental to the future of our business. But we need more of these challenging questions and less common ones.”

Focused on Core Issues and Impacts

“Over the course of ten years, raters have pushed companies to say less about processes and more about the issue du jour. They didn’t want to know about operational integrity — which is fundamental to the sustainability of our business over time.”

8

Future ratings will need to find the right balance between being consistent and adaptive. On the former, companies and users benefit from the certainty that comes with constancy in ratings’ methodologies over time — they want the goalposts to remain the same. Yet, the sustainability agenda is evolving as our understanding of issues improves, and ratings of corporate sustainability performance should adjust to reflect this. For example, when scientists identify new chemicals of concern, these should be reflected in ratings where such discoveries are pertinent.

In our view, raters can strike this balance — and, indeed, some do so today. It is important to be deliberate and transparent about the change, giving companies and users sufficient lead time and explanation. We identified several good practices in this area in phase three.

As we wrote in phase three, we believe that when raters are transparent across a variety of facets — their methodologies, their results, how they manage potential conflicts of interest, etc. — they build trust and stimulate demand for their products. We expect to see greater transparency in the future, and indeed have already observed since phase three improved disclosure amongst some ratings. For example, we like statements such as the one below from the CEO of STOXX Limited upon the launch of a new ESG index family with Sustainalytics (one of the raters which participated in phase three).

The notion that opening the analysis “black box” will create more value is certainly counter-intuitive to some. But we believe ratings will get creative in generating value given their increased disclosure. We applaud those ratings which have become more transparent as a result of Rate the Raters, and we will further push disclosure with our call to action below.

Rate the Raters Phase Four

Consistent yet Adaptive

“It’s going to be difficult to get investors to pay attention to ratings if their criteria change every year. While I understand that issues evolve, raters should be deliberate in their pace of change and better explain the nature of and rationale for the change.”

Transparent

“What raters are fundamentally doing is requesting human capital (i.e. our time) to generate commercial value for them. If we know what goes into their model, what comes out and what their biases are, then I’m comfortable with the fact that their black box [their method of analysis] is unknown.”

“Our new index family will set standards in terms of full transparency and comprehensiveness in the ESG indexing space.”Hartmut Graf, CEO, STOXX Limited, on the April 2011 launch of the STOXX Global ESG Leaders Index family

9

Throughout the last year, one of the most frequent questions we received about this work was: “Why didn’t you actually rate — and rank — the raters?” Some — typically companies! — repeatedly asked us to do so. But, as we’ve maintained throughout, we made the deliberate choice to review a variety of ratings types — consumer-focused, investor-focused, single issue, etc. — and thus do not believe that putting them in rank order would be a meaningful exercise (just as we think comparing companies in different sectors is not a meaningful exercise). Our goal was to identify best practices and areas for improvement across raters, and this informed our decision not to publish the completed questionnaires we received and the rating overviews we compiled for each. This approach allowed us to have candid and constructive conversations with the 21 ratings we reviewed in phase three without the anxious debate that would have occurred if we had intended to assign and publish overall scores.

This said, we do believe that raters need to be more transparent about their practices, and thus with the publication of this paper, we put forth a call to action and disclosure to all raters — the 21 we reviewed, the 87 others we found in phase two and didn’t review and all other ratings. We request that by January 1, 2012, all raters respond publicly to the questions below (the same questions we used in phase three). In doing so, we request that they articulate how they are working to improve their performance in each area and overall. We invite them to reference our methodology and to provide feedback on how we might improve it. In making this request, we are channeling the sentiment of the many stakeholders with whom we’ve spoken during the project who want to see the same level of transparency from raters as we expect from companies.

We commit to compiling the raters’ responses in one place on www.sustainability.com. In addition, we will analyze each report and produce a summary of our findings on progress in 2012. We will also share our perspectives on the degree to which ratings have changed at that time.

With this disclosure, we believe the market will be better placed to decide which ratings will endure.

Rate the Raters Phase Four

A Call to Action and Disclosure

We ask raters to respond publicly to our questions by January 1, 2012

10Rate the Raters Phase Four

1 Describe how you make available the details of your methodology to the public, rated companies and other stakeholders.

2 Make public your methodology.

1 Do you have a policy (or guidelines, standards, etc.) that addresses conflicts of interest? If so, is it disclosed publicly? In answering, please describe the key elements of the policy.

2 Do you disclose the relationships and independence of board members, advisory panel members, partners and other involved third-parties? If so, how? How do you manage conflicts with these entities?

3 Do you offer for-fee services to the companies that you rate (including benchmark reports)? If yes, please describe these services. How do you avoid conflicts of interest in these situations?

4 Do you have any other financial relationships (e.g. sponsorships) with any companies you rate? If so, describe these relationships.

1 Do you have a regular approach to updating your methodology over time? If yes, describe this approach.

2 Describe how you communicate any changes in your methodologies to affected companies and other stakeholders, and how you work with them to adapt to and understand the implications of the changes.

1 Does the rating have an external and / or independent advisory body in place? If so, describe how this body is involved in the rating’s construction and maintenance and the scoring, rating or selection of companies.

2 Describe how you involve / solicit feedback from external stakeholders in the development and ongoing maintenance of the rating. Please provide an example of how you revised your methodology based on stakeholder input.

3 How do you communicate the output / summary from this engagement? For example, do you make public the stakeholder comments?

Questions for Rater Disclosure

1.0 Governance and Transparency

1.1 Disclosure of Methodology

1.2 Conflict Management

1.3 Regular Review

1.4 Stakeholder Involvement

11Rate the Raters Phase Four

1 Please list and describe the information sources that you utilize to develop your rating.

2 Please describe how this information is obtained (i.e. from publicly available documents, partner organizations, directly from companies).

3 How often do you refresh information / inputs on companies?

4 How do you deal with non-disclosure by companies in your rating?

1 Describe the process by which you engage / interact with the companies that you rate (i.e. frequency, nature, format).

2 On average, how much time do your analysts spend engaging with each rated company on an annualized basis?

3 Please describe if and how feedback from rated companies is incorporated into your research and ratings process.

1 Describe your approach to verifying the information used to assess companies. Is this approach captured in a formal policy or guidance document?

2 Is external or third-party verification a consideration in assessing the quality of information sources? If so, please explain.

Questions for Rater Disclosure

2.0 Quality of Inputs

2.1 Information Sources

2.2 Company Engagement

2.3 Input Verification

1 On average, how many years of experience do your analysts (internal or contract) have working in / analyzing the industries they cover?

2 Describe your approach to ongoing training and industry education for your

analysts.

3 On average, how many companies do your analysts cover each year?

1 Describe your approach to ensuring quality control throughout the ratings process. Is this process documented or codified?

2 Have your research systems or processes been certified, assured or verified by an external organization? If so, please describe.

3.0 Research Process

3.1 Experience and Capacity of Research Team

3.2 Quality Management

12Rate the Raters Phase Four

1 Does your rating use the same criteria and weightings for all sectors and companies? If so, explain why.

2 Describe how you take into account industry- and company-specific issues and context.

1 Describe the basis for your rating of companies, including how you determine what constitutes the different degrees of performance. For example, if you grade companies on an A-F scale, how do you determine these levels?

2 Do you factor company-specific context into your rating? If so, please explain how. For example, if assessing a company on water consumption, do you consider the geographic context for this assessment (e.g. some regions are more arid than others)?

3 Do you incorporate external norms, standards or principles (e.g. UN MDGs, GRI, UNGC) into your rating or process? , etc. If so, which ones and how are they incorporated?

4 How do you deal with unforeseen, negative company events in your ratings process (e.g. environmental incidents, ethics breaches)?

Questions for Rater Disclosure

3.3 Sector Specificity

3.4 Basis for Rating

1 Describe the steps you take to verify your rating results (including scores and reports) prior to their finalization.

2 Do you give rated companies an opportunity to review draft scores or reports? If yes, describe this process, including timeframe given for review.

3 Do you have a formal policy and/or process for addressing challenges or disputes presented by companies or other stakeholders? If yes, describe this policy and/or process. If no, explain how you address challenges or disputes.

1 Describe how you disclose and explain your results to rated companies. Do you share the full details of the assessment or results? If no, explain why not.

2 Describe how and to what extent stakeholders (other than rated companies) can access the details and results of your rating.

4.0 Outputs

4.1 Validation of Results

4.2 Accessibility

13

With the publication of the last of the Rate the Raters papers, we end with a few reflections on our work over the last 14 months.

First, it is vitally important that we get sustainability ratings right — this is why we launched Rate the Raters in the first place, and why we will stay engaged in the ratings agenda going forward. We face enormous environmental, social and economic challenges and opportunities, and ratings can help drive capital of all kinds — investor, human, consumer, etc. — to those companies best positioned to deal with these challenges and opportunities.

Second, in observing the growth of ratings over the last decade, it appears we are at an inflection point — the number of ratings may continue to rise in the near term, but it will do so at a lower rate and then soon begin to decline. This trend reflects a few fundamental characteristics of the ratings agenda today. One, there is not enough demand in the market for existing ratings, let alone additional ones. Two, there are already some good ratings in existence. Thus, organizations considering new ratings would do well to first examine the current inventory of ratings (perhaps using Rate the Raters phase two as a guide) so as to not duplicate the good work of others.

Related to this, the introduction of a new rating isn’t going to “solve” the ratings challenge that companies, users and other stakeholders face. Rather, we need to change how the ratings value chain works, for example rewarding more robust research and making the ratings piece of the chain more objective. We would also do well to work to improve the better ratings that exist today, and phase three of Rate the Raters includes recommendations to do just that.

Lastly, companies should play a more proactive role in the ratings agenda. They should understand what various ratings measure and prioritize those ratings which exhibit greater quality and transparency. They should be unabashed about calling out poor quality in raters’ work, but should also be engaged and seek to improve performance based on the results. And, they shouldn’t add to the growing list of ratings by issuing their own — we’ve observed a number of companies developing their own supplier ratings when it is possible they could have utilized existing ratings.

We look forward to receiving and then sharing the results of the call to disclosure in 2012. Until then, we welcome your comments, questions and views on topics that warrant further research.

Rate the Raters Phase Four

Parting Remarks

14Rate the Raters Phase Four Appendix

Bloomberg CiscoClimate Counts Corporate Knights Corporate Responsibility Officers Association CSRHub Ethisphere ExxonMobil Ford Motor Company GlaxoSmithKline Global Reporting Initiative * SustainAbilityTellus Institute *Trucost

Access to Medicine Index ASSET4 AutodeskBloomberg Carbon Disclosure Project Coca-Cola EnterprisesEIRIS FTSE GlaxoSmithKlineHenderson Global Investors *Maplecroft Climate Innovation Indexes oekom SustainAbilitySustainable Asset Management (Dow Jones Sustainability Indexes) Vigeo

April 19, 2011 London

Organizations Participating in Workshops

* Indicates advisory panel members for Rate the Raters

April 28, 2011 Washington, DC

SustainAbility is a think tank and strategy consultancy working to inspire transformative business leadership on the sustainability agenda. Established in 1987, SustainAbility delivers illuminating foresight and actionable insight on sustainable development trends and issues. The company operates globally and has offices in Europe, North America and India. For more information, visit www.sustainability.com

SustainAbility

Sponsors

15Rate the Raters Phase Four