SPTF Annual Meeting 2015 day two notes... · Web viewDay two meeting notes. Opening remarks....

SPTF Annual Meeting 2015

Day two meeting notes

Opening remarks

Frank de Giovanni (Ford Foundation, and SPTF Board Chair)

This is the 10th Anniversary of the SPTF – a very important milestone. We have accomplished a lot over the last ten years – adopted the USSPM and grown to more than 2,600 members in 127 countries. We just developed a new three year strategic plan and conducted a survey in partnership with the MIX and The Global Appeal which indicated that 90% of stakeholders were “aware, familiar, or very familiar” with the Universal Standards indicating that they have achieved high recognition worldwide.

I would like to take this opportunity to reflect on how we have arrived at this point, to say a few words about where we still need to go, and to offer my thoughts on why I think the SPTF is so important to helping the microfinance sector achieve its original vision. I hope that you will indulge me in a bit of personal reflection.

Microfinance began in 1977 when Muhammed Yunus in Bangladesh and other organizations in Asia and Latin America experimented with the notion that poor women could borrow small amounts of money to invest in their livelihoods and repay these loans.

Yunus was motivated first and foremost by a desire to improve the lives of these women, and saw access to credit as a key means to do this.

I know this, because my organization – the Ford Foundation – provided Professor Yunus the funds to conduct the experiment that eventually became the Grameen Bank.

So the original vision of microcredit – and its successor microfinance – was rooted in using financial services to improve the lives of poor women – to move them out of poverty and to empower them.

It was a decidedly people or client-oriented viewpoint married to the development of a financial institution. Thus it had a double bottom line from the outset. Financial services were only a means to an end, not an end in themselves.

But over the years, this original vision of improving the lives of the poor receded as concerns over the financial sustainability of the MFI took on more importance.

Thus, we seem to have shifted from a major concern or focus on poor clients to a dominant focus on the institutions that provide the financial services.

Just think back to the major debates in the field during the 1990s and 2000s. They included: What is more effective – group-based vs. individual lending? Should NGOs convert to regulated FIs? What are the most important or best indicators to use to measure financial

performance? How long should it take an MFI to reach financial sustainability?


These are clearly critical issues, because we need sustainable FIs to continue delivering financial services to poor people.

But the problem was that a comparable level of ferment and energy was not being applied throughout the industry to trying to answer questions such as:

What is the best way to target services? What types of financial services do poor households need to improve their lives? How can we tell if clients’ lives are improving? Can MFIs serve poor clients and become financially sustainable?

In short, ½ of the double-bottom line was being taken for granted and therefore overlooked by many organizations in the field. Not all of them as we saw yesterday in the examples of Cashpor and AMK. And, in the worst case, many organizations began Serving better off clients as they strove to become financially sustainable. This was the proverbial “mission drift” that many observers talked about in the 2000s.

The unspoken assumption seemed to be that if you built a strong microfinance organization, and your heart was basically in the right place, then social performance would happen by itself.

But a number of us were troubled by this state of affairs. We did not believe that MFIs could do a good job of helping clients improve their lives without consciously setting out to do this.

In essence, this would not happen by itself. We believed you could not just luck your way into strong social performance any more than you could stumble into strong financial performance by trying to do the right thing and then hoping for the best. I don’t think anyone here believes you can achieve strong repayment rates without working hard to put in place rigorous systems of due diligence, loan monitoring and collection systems.

Thus, if you want to be equally serious about both halves of the double bottom line, you need to devote as much energy, thought, debate, and resources to social performance as to financial performance.

And it is critical to remember that financial performance focuses on the institution. Social performance emphasizes the client. Strong institutions—with the right legal and governance structure, delivering the right products, through the right channels operating at sufficient scale to be sustainable—all of that is critical to the success of microfinance, but it is a necessary but not sufficient condition. That is, it is not enough.

We also need to care about the client. But we have to do more than that.

All financial institutions care about their clients. They spend lots of resources on market research to identify what their customers or clients want and need. Unless their product offerings are grounded in the needs of their clients, they will not sustain a strong financial performance. Their clients will go elsewhere. We heard that yesterday in the panel on client satisfaction and retention.

But what sets MFIs apart from mainstream FIs is that they should care whether their clients use financial services to improve their lives.

And that is why a few of us came together in 2005 to create the SPTF because we did not believe that the MF industry was paying enough attention to this question – the question that motivated its start – could financial services improve the lives of the poor?


We wanted to explore how the microfinance industry could get intentional and consistent about measuring social performance. And looking back on the last ten years, there are many things I am truly grateful for, and here are just two of them.

First, we started our task force initiative a couple of years before microfinance’s first initial public offering and before the onset of the over-indebtedness crisis in many countries. That first IPO was (and remains) controversial and troubling to many observers, both within and outside our industry because it lead to such a major emphasis on the profit motive in microfinance rather than the social purpose of microfinance.

I think there’s a real possibility that if the Social Performance Task Force had been created after that IPO and the over-indebtedness crises, we might have been perceived cynically as an after-the-fact PR exercise, and there would not have been much we could have done to fight that perception. As it was, the Task Force’s work was already well underway before the IPO and the OI crises. That timing gave us credibility, and that credibility has served the industry well. I am grateful for that.

I am also thankful that we were able to take the time we needed, to develop the Universal Standards. If the USSPM had been a response to externally-driven events, they might not have taken ten years to develop . . . but they also would have been seen for what they would have been: top-down, imposed, and ceremonial. Instead, the Universal Standards are bottom-up, deeply collaborative, and highly practical. We developed them through an intensive process with working groups involving hundreds of people from a variety of stakeholder groups.I mention the Universal Standards because they are such a major milestone. Now that the Universal Standards exist, part of the Task Force’s work will shift to using its collective voice to promote their broad adoption.

We also want to ensure that every institution that wants to implement the Universal Standards has access to the support they need to do so. Those twin goals are at the heart of our new 3-year strategic plan which includes a major focus on developing a long term communications strategy and well as trainings and resource materials to support implementation.

I would now like to shift focus to talk about why I think the SPTF is so special. It has a set of attributes I call the four C’s: client-centric, change-oriented, collaborative, and committed.Focusing on the needs of clients is our WHAT. The essence of “social performance management” is “a management style that puts customers at the center of every strategic and operational decision.”

The second C is change – orientation – asking whether the products and services that we provide are improving the lives of our target population in line with the goals that we have set for these clients. This is our WHY – the reason that we exist. The strategic plan will devote increasing attention to outcomes as you will hear later today.

The Universal Standards themselves reflect this client-centricity – the what and why. Each of the Standards’ six dimensions focuses on the client: from defining the goals an institution wants to achieve with the target clients it wants to reach, to designing products and services to meet those clients’ needs, to ensuring client-centricity on the part of board, management, and staff, to treating clients responsibly—and also, critically, to treating staff responsibly—because it’s the right thing to do, and also because the way you treat your staff is the way they will treat your clients, and to ensuring that the institution’s growth does not imperil its social mission.

We started with AMK on Tuesday and Khushhali Bank and Cashpor yesterday because those organizations live, breathe, eat, sleep, and get up in the morning driven by the why. Thriving customers are the bedrock goal, but it’s not a one-way street. Happy customers are loyal


customers, and that loyalty is good for the “other bottom line,” the financial one.

The third C – collaboration – is the HOW of the Social Performance Task Force. From its earliest days back in 2005, the Task Force was determined to build social performance management for the whole industry. We wanted to do this in a participatory manner.

We didn’t want donors or investors defining success for practitioners, or practitioners deciding for themselves what success would mean, or researchers laboring in isolation to devise evaluation frameworks. Rather, we wanted an understanding of social performance management that would be relevant for the entire industry, developed by all segments of the industry to get buy-in from everyone —hence the word “Universal.”

And that meant involving the entire industry, or at least a representative cross-section of it. And that’s exactly what we have built. The work of the SPTF is largely done through working groups. And this meeting also reflects the high degree of collaboration that characterizes all our work. Over 50 people were involved in designing the content of the sessions.

My final “C”—committed—describes the “WHO,” the people of this Task Force. We started with a small group of committed people, and the fact that many of them are in this room ten years later speaks volumes.

But whether they were present at the creation or whether they joined the Task Force more recently, our members all believe strongly in the Task Force’s vision: of a true double-bottom-line industry where social performance and financial performance mutually strengthen and reinforce each other.

Social performance is nothing more—but also nothing less—than knowing your clients and trying to help them improve their lives. Institutions that regard their poor and low-income clients as capable, multi-dimensional, dynamic human beings will know them—their goals, their challenges, their opportunities, and—yes—as a byproduct of that deeper knowledge, they’ll also know what products and services and delivery channels and marketing messages address their needs.

And I believe that they will know whether the products and services they provide are making a difference in the lives of their clients. And this knowledge will, in turn, enable them to further improve those products and services. This may be more vision than reality for most MFIs now, but examples such as the ones highlighted in this meeting give us hope that they can be a reality for most, if not all, MFIs

And the Institutions that get that right can thrive. Examples such as Khushhali Bank indicate that it is not feel-good spin to say that strong social performance supports strong financial performance. Every successful business in the world needs to understand its clients.

But low-income people in developing countries are not a consumer segment like any other, and financial services are not a consumer product like any other. We’re in a more complex business than many, and our customers’ lives and needs are more complex than many, too.

True client-centricity, under those conditions, simply demands more effort. So we need to be in it for the long haul. It takes commitment from all of us: practitioner commitment, patient capital, and also a commitment to candor—about how implementing social performance management plays out on the ground, about asking the hard questions about what works and what doesn’t, about identifying the challenges, and how to address them.


I want to stress that the Social Performance Task Force – which, again, is a cross-section of the entire industry—is not in the business of creating tools or systems. We’re seeking to create and sustain an industry culture. And like any culture, it depends on consistency, on that commitment I just mentioned.

The best comparison I can think of is a culture that values physical health. You don’t wake up one day and say “I know I am out of shape and today is the day I am going to fix that, I am going to go to the gym and work out for 9-1/2 hours and then I am going to check off that box on my to-do list.”

It just doesn’t work that way!

You make a commitment. You exercise that day. And the next day and the next day and the day after that. And no, you are not going to see results the first day. And the second day, you still won’t see results – plus you’ll be in pain! But what I can tell you is that if you commit to regular exercise, you will be in good shape and you will be healthier and your quality of life will improve.

If your institution commits to social performance, it will be a healthier, client-focused, grounded “in the why” institution. If enough institutions do it, we will have a healthier, client-focused, why-grounded industry. That’s the industry I want to see. That’s the industry the Social Performance Task Force was created to bring about. And that’s the industry that we are committed to working toward, until it is a universal reality.

Thank you.

Recognizing the Warning Signs of Market Saturation: Avoiding a National Over-indebtedness Crisis

This plenary session explored how policymakers and other industry stakeholders can encourage responsible finance through regulation. In particular, it examined the importance of consumer protection practices for preventing over-indebtedness in high growth markets such as Cambodia. The National Bank of Cambodia provided key commentary from the perspective of the local marketplace.

Speakers: Alok Misra, M-CRIL; Rath Sovannorak, National Bank of Cambodia; Jhale Hajiyeva, Azerbaijan Micro-finance Association; Daniel Rozas, MIMOSA

Alok Misra (M-CRIL, Session facilitator)This panel is designed to give a macro-economic perspective on over-indebtedness. In the background of this discussion lay the many crises in different countries over the past ten years – and indeed we still see problems emerging around the world. The impact of the crisis, whether it comes from the client side or the public policy side, can be overwhelming. Institutions fold, clients get denied access to credit. In light of this, the objective of this session is around learning from these crises, asking what we can do proactively. A recent CGAP study considered over-indebtedness crisis and found three common elements: markets that were over-saturated, incidence of multiple borrowing, failure of clients’ businesses, and MFI systems that were overstretched. What’s the local perspective on this? A recent study in Cambodia on over-indebtedness in a sample of villages found that nearly 22 per cent of clients were insolvent or at risk. But does this matter? Many sceptics to this argument, where over-indebtedness has not increased, say that growth is necessary to tackle financial exclusion. It’s important to state that


we here are not anti-growth. Exclusion is clearly a big challenge. But we are for long-term sustainable growth, which is a different matter.

In terms of how this matter link with the Universal Standards – in fact over-indebtedness issues appears all across all dimensions of the Standards.

Q: How is this issue important for the National Bank of Cambodia?A: A bit of background on my country: in Cambodia, we had a long civil war, and microfinance started in the early 1990s (by international NGOs focusing on humanitarian assistance). Since then, the sector grew, and we saw that donor funding was not sustainable. As a result, in 2001 the government passed a law requiring all NGO microfinance programmes with a portfolio over (approximately) $250,000 to transform into private licensed companies. This was a difficult challenge for the NGO MFIs – but in 2005/06, the sector was growing at a rapid pace (20 per cent per annum), and investors entered the market in large numbers. In 2007/08, the market was growing at 70 per cent per annum. Post-crisis growth has slowed down considerably (to about 40% per annum).

In terms of client protection regulation, we have a few key pieces in place at the moment. There are regulations around pricing transparency and interest rate communication. We also work closely with the IFC, who conducted a study on the client protection framework in this country. We’re also looking more at client protection through client literacy.

Q: Jhale, based on your work in Azerbaijan, what is your perspective on this over-indebtedness?A: This issue is very important for us because it’s been part of our mission from the start. We promote effective collective action to advance the interest of the microfinance community and its clients. We shine a light on challenges the industry isn’t looking at yet, and create a dialogue between our members on these challenges. We also take advantage of our membership in international networks to get a global perspective.

In our own market, we’ve been looking at how market saturation is increasing (largely due to downscaling banks) which is leading to cross-lending. In this context, if MFIs pursue aggressive growth strategies (which they do), we’ll have a problem on our hands. Where complaints arose in the field about MFI behaviour, we took the matter to MFI managers, and also involved the Central Bank to investigate the matter directly. As a result, our members (and some brave pioneers among them) are starting to feel that they can talk openly with each other about issues around cross-lending, rather than trying to hide it. To me, this highlights the importance of both early prevention, and also getting top management from MFIs together to discuss the matter frankly. And we need to – because this is part of our mission, and to the benefit of clients and our financial bottom lines.

Q: Daniel – what is your take on over-indebtedness?A: My perspective is that over-indebtedness is not a microfinance problem. It’s a credit problem. We like to talk about crises in India and elsewhere – but what about America? Spain? National over-indebtedness is not even a social performance problem. It’s a business problem. Markets have a certain capacity. If you over-serve it, it will collapse. There is something different about microfinance – and that’s multiple borrowing. If you’re in a developed country, you can get over-indebted – just by taking a loan that is too big or buying a house you can’t afford. But in microfinance, the path to over-indebtedness is generally multiple borrowing. However, it’s hard to say whether multiple borrowing is demand-driven or supply driven, and whether it’s always the path to over-indebtedness. We need to know whether clients take out many loans because the loans are too small to meet their needs, or whether it’s because MFIs are pushing too many loans on them.


The other thing to bear in mind is that we’ll never know whether a multiple borrower is in trouble until a crisis happens – because until it does, they’ll just keep borrowing to stay afloat. In a recent study in one country, we saw that in the moment before the crisis, multiple borrowers and single borrowers had approximately the same performance. The moment the crisis hit – the multiple borrowers started failing.

Q: What does the NBC use to estimate market saturation?A: Based on a sample study (that looked at 6% of the market) we see that around 65% has one loan, 24% has two loans, and around 9% has three loans. And when we travel into the provinces, we see a lot of MFI offices. In terms of saturation, we are very concerned of course, but of course we can’t draw results from such a small sample size. Our focus is to reduce the risk of over-indebtedness. The national association is playing a key role in the dialogue around over-indebtedness here (through a CEO club), as is the new Credit Bureau (launched in 2012). The Credit Bureau is our primary tool for measuring market saturation. However, it only captures data around how many loans a client has, rather than their repayment capacity.

Q: What kind of reporting does AMFA do with its members? A: We started with a few simple indicators – initially at PAR>30. Then we expanded to collect PAR 1>30 days, and PAR 30>180 days. We also look at the performance of loans by currency – because Azerbaijani loans are issued in different currencies. In doing so, we saw that foreign currency loans were the most affected, due to the devaluation. We also look at loan performance by product (e.g. agriculture, business). At the same time, we completed a simple study by selection two regions where we had the highest portfolio at risk (from 1>30 days). We asked our members active in that region to give us the names and IDs of their active numbers, and by cross-referencing we were able to estimate the level of multiple borrowing. In some places it turned out to be as high as 25-30%, and clients had as many as 7 loans. This was an important wake-up call for members. The study results also helped us approach the Credit Registry – because at that time, only banks were required to report. MFIs weren’t reporting, and so didn’t have access to complete data. Two and a half years later, and now MFIs are reporting.

We’re ready to repeat the study, and I’m pleased to report that it’s actually our members that are agitating for this. The study will also look at coping mechanisms when clients become over-indebted. Our members are also more willing to have their names included on the study – rather than the information being reporting anonymously. This shows me that they don’t want to hide the problem of cross-borrowing anymore, and this is an important shift in thinking at the MFI level.

The key lesson in all of this for me is this: this work began because of our own motivation at the start, but we made sure our members saw the benefits. Because the truth is: in the long-term, this work will only succeed if our members believe in it.

Q: We’ve talked about different approaches to looking at over-indebtedness and multiple borrowing. One of them is credit bureau data – which is by definition limited, because it only includes information on numbers of loans, rather than repayment capacity. MIMOSA takes a different approach. Daniel – can you lay out the key facts for us?A: If you blow up a balloon too far, it will eventually pop. You can keep blowing it up, but you never know which breath will be the last. Balloons are also fragile. The same with markets. If a credit market is over-sat, even a moderate shock (political crisis, economic downturn, etc.) will through it into complete chaos. The other thing we know is that it doesn’t have to pop – to make this happen, you just have to stop blowing. In credit markets, the same holds true.

The FINDEX Borrowing Prevalence (2014) study shows that at a country level, as the human development index (HDI) increases, so does the incidence of borrowing. This shows me that market capacity is NOT unknowable.


Let’s talk about MIMOSA. We look at the HDI, population density, and credit bureau data to crunch numbers and come up with our best estimate of the market capacity. A country score is then ranked on a scale. We can then map those results – and in doing so we see that Cambodia has the highlight market saturation score. Looking at the results in 2008, the growth in market saturation is quite marked.

How do we know that MIMOSA works? We piloted in Peru, looking at the difference between penetration and capacity in different regions using our data and credit bureau data, and the results were largely in line. This is great – considering that MIMOSA is quite a simple tool. So at this stage, Cambodia seems like a pretty well-inflated balloon. At this point, the argument that the upsides of another three years of 30% growth outweigh the downsides is hard to make. Some countries, such as Bangladesh, got to the point of market overheat and deliberately pulled back, with great success. Doing the same thing in Cambodia would require collaboration between the Central Bank and the association.

Q: Did you test MIMOSA in markets were crises actually happen, and did you get reliable back-test results?A: Great question! Yes. We went to India (but not AP). We’re looking at Morocco at the moment, and also comparing data with the recent over-indebtedness study in Cambodia.

Q: MIMOSA puts market saturation at higher levels than Cambodian Credit Bureau data.A: We also conduct annual supervision of MFIs. As you can see that the PAR trend of any MFIs is less than one per cent. We head to the field, check documents, and we can’t find many clients that are paying late. So we’re being pro-active to the extent we can in terms of supervision, and hopefully we can get more granular data from MFIs to strengthen the analysis. But it’s important too to recognise that PAR data is a lagging indicator – so we are also looking at client protection regulation, and setting up client financial literacy mechanisms. The problem is of course that we can only control the formal sector – but the informal sector is quite active at all.

Comment:- Over-indebtedness is about making unacceptable sacrifices to repay a loan. I would

dispute the fact that it’s only multiple lending is the one loan can’t lead to this type of stress. However, PAR is a lag indicator – we need to be thinking about the lead indicators that we can monitor. At SEF, we identified stress at a client level to find problems before they turned into default. Suggest that this is an issue the STPF can look at.

Regulation Breakout: Supporting regulators, networks, and financial institutions to respond to client complaints.

This panel examined new insights into client behaviour related to using complaints mechanisms and how regulators, networks, and financial institutions can encourage clients’ rights. Panellists shared lessons learned in complaints mechanism design and how partnerships among industry stakeholders can lead to better results for clients.

Speakers: Alex Fiorillo, GRID Impact; Hema Bansal, Smart Campaign; Aldo Moauro, MicroFinanza Rating

What are the barriers that discourage clients from voicing complaints about their experiences with financial institutions/products?


Alex Fiorillo mentioned that CGAP did research on client behavior to understand how they behave after they have a negative experience with the organization. Some of the key findings from the study were: Many clients accept a negative experience with the FSP/product because they see it as part

of doing business (clients would expect unexpected fees, charges, mal-treatment) Even though complaint mechanisms might be in place, many clients do not use them

because they do not think that anything would come out of it When customers did go to the FSP to express a complaint, many times they were told that

they were” asking too many questions” or “the manager is busy, come back tomorrow” to delay or create hassle factors to customers. So even when customers tried to report a complaint they found that the institution was not receptive.

What barriers prevent FSPs from creating and using effective complaint mechanisms?

Highlights from results of +20 certifications conducted by Microfinanza Rating: Out of 26, only 5 institutions were compliant with having mechanisms Only 30% of institutions had training in place for staff to handle complaints But +61% of institutions used the feedback obtained to improve practices and products Only 22% effectively monitor the effectiveness of client complaint mechanisms But 70% of institutions has dedicated staff (particularly in central office) to deal with client

complaints

According to Aldo Moauro (Microfinanza Rating) the main barriers that prevent FSPs to create, put in place, and use effective client complaint mechanisms are: Cost Confusion between different kinds of mechanisms (complaint vs. client satisfaction surveys)

– when FSPs want feedback from clients they typically rely on client satisfaction surveys

According to Hema Bansal (Smart Campaign) the main barriers are: Institutions do not prioritize understanding clients Selection of complaint channel – Some institutions think that a complaint box is enough.

However, complaint boxes tend to not be effective There is no process to capture complaints at the field level – which means a lot of

complaints are not documented and get lost. Very few organizations have a mechanism to capture complaints at the field level

Client communication – clients need to understand the channels available to capture their opinion. Otherwise, they will not be effective.

Staff training – covering the topic in the code of conduct is not enough, institutions need to provide specific training

What is the role of networks?

While the work is still evolving there is still no clear mandate for networks or clear examples of networks with strong client complaint mechanisms. Smart Campaign and SEEP have worked with some networks to help them understand the need for both networks and their members to set up a client complaint mechanism.

What is the role of regulators?

Networks could share with regulators the most important client complaints for regulators to better understand the state of the market. The main barrier for regulators to get involved has been lack of capacity (from regulators) to dedicate to this matter. So even though regulators have expressed desire to help, they have also mentioned lack of capacity. Many have suggested


third parties taking this role. In Ghana, for example, the regulators are empowering third party actors to handle low-level complaints.

How can we empower customers to seek resolution?

Clients are more likely to seek client complaint mechanisms if they have a close example (friend, neighbor, family member) of having filed a complaint and it having being addressed. Therefore – how can it be signaled to clients on a large scale that it’s important for them to seek resolution?

One thing that will help customers feel empowered is building systems of accountability for FSPs. For example, requiring FSPs to report to regulators certain amount of complains within a period of time, or build incentive mechanisms, encourage spot-checking by third parties (e.g., somebody pretending to file a complaint just to study/experience the process).

How can a FSP overcome the barriers discussed above in order to encourage clients’ right to complain and to respond to complaints?

Aldo Moauro mentioned that a happy client typically shares his experience with 4 people, an unhappy client does so with 9. Further, an unhappy client might leave the institution. Some ways in which FSPs can overcome the barriers are:

Improve communications with clients so clients understand they have the right to complain and are aware of the different mechanisms available.

Train field staff. Many institutions have dedicated and trained staff (to handle clients complain) at the central office but not on the field.

Develop and put in place written policies. Put in place an accountability system with the institution Disclose the timeline to solve a complain (e.g., 1-2 days to give an answer to the client, etc) Have independent controls to check on awareness of clients about the mechanisms and also

check on the effectiveness of the system

Hema Bansal added that institutions need to change their mindsets to see the value of complaints. She also mentioned the importance of having a client complaint mechanism that works for all clients. For example, Ujivan (India) realized that they did not have many complaints so they put in place a role of a person who visits clients frequently and asks them whether they have any complaints. As a result, clients have opened up a lot and the institution has been able to gather valuable data.

How can networks/associations overcome the barriers discussed above?

Networks must understand the value of complaints. Have a best practice example of a network to share with others. In Philippines, for example

the network has a hotline for all members to communicate with this. Rely on partners for funding. Obtain technical support.

How can regulators overcome the barriers discussed above?

Regulators tend to work at a very high-level and lose sight of the field. CGAP worked with regulators in Ghana. They brought customers from rural areas to tell the Central Bank their stories and share their negative experiences with the FSP. This helped the Central Bank better understand the perspective of the client and the need to be customer-centric. The team developed a map of the different levels of complaints to identify what should be managed at the field, what at the branch level, what at the central office, and what should be escalated to the


Central Bank. This map has been turned into a visual that now is being put in place in FSPs offices.

-------------------------------------------------------------------------------------------------------

The room broke into smaller groups to discuss possible solutions or share good examples of good client complaint mechanisms.

The “great ideas” and “crazy ideas” reported were the following:

Great ideas: Empower people to complain – have a campaign with a catchy phrase, e.g., “It’s Ok to

complain” Conduct regular field visits Introduce a hotline number or call center Have multiple channels Identify influencers Categorize complaints Encourage/engage in client education Use IVR on your hotline (interactive voice recording – eg. for x department, press 1)

Crazy ideas: Partner with WhatsApp and similar applications to handle customer feedback Open a platform for complaints with new product launches Change -- complaint jargon Reward feedback

Regulation Breakout: Regulating the industry through partnerships between regulators and associations

This panel examined appropriate roles for regulators and national networks in creating and enforcing regulation. The panel also looked at good models for effective relationships between microfinance regulators and associations and discussed examples of how the two can work together for the mutual benefit of all parties (regulators, associations, financial institutions, and clients).

Speakers: Kinga Dabrowska, Microfinance Centre; Prudence Casala, Central Bank of the Philippines; Jackie Mbabazi, AMFI-Uganda; Javier Vaca, Red Financiera Rural

Prudence (CBP):From our perspective, role of regulator is to create an enabling environment to come up with market-responsive policies, both from the perspective of institutions and the client. How do we do this? We identify who our stakeholders are and listen to them. Our stakeholders, fortunately, all have associations representing them. Financial inclusion is not only about the banking industry – we engage NGOs, consumer organisations, and NBFIs, realising that they all have a role to play in financial inclusion. We have regular policy coordination meetings to bring everyone together to give and receive feedback on our regulations, which have proven to be quiet effective. For example in terms of our client protection framework, we consulted not only the financial services providers but also the clients as well – through local consumer associations.

Jackie (AMFIU):


As an association, we have a good relationship with our regulator, but this was not always the case. In the late nineties, the Central Bank was unconcerned with microfinance, and referred to it as the “informal sector”. We had a microfinance forum that would bring together to local service providers, and when we invited the Central Bank they declined to come. However within that time, the reputation of the sector declined, and many negative stories appeared in the press with clients referring to MFIs as “banks”. The Central Bank retaliated by sending out a press release telling MFIs not to refer to themselves as banks – but this didn’t actually change the situation. So at a network level, we lobbied the microfinance sector to gather strength – bringing all financial service providers into our microfinance forum. We discussed (on a quarterly basis) key issues such as regulation, competition, etc. – and the Central Bank finally started joining us. Regulation for MFIs soon followed, and we’ve never looked back.

Javier (RFR):In Ecuador, we have a mix of credit unions, MFIs and NGOs. The starting point for regulation was to recognise microfinance as being different than traditional banks – because client capacity to repay differs than from traditional banks, because we have a social mission, and because often we don’t have shareholders. Once this was done – that opens the door to creating specific regulation for the microfinance sector that doesn’t apply to the commercial sector, on issues such as risk management, etc. Another advantage of this is in terms of avoiding some of the requirements put upon the commercial banks, in recognition of the fact that many of our clients are operating in the informal sector.

Q: From your experience, what does an effective partnership between the regulator and the network look like?

(Jackie) The regulator usually has specific and rigid roles. As associations we need to work around those. In Uganda, the regulator must enact prudential regulation. They have the expertise, the infrastructure – and it is a full-time job. A network can’t take on this role – we lack the capacity. But where areas of concern arise, both should work in close communication and collaboration.(Javier) We are working with the regulator to develop training and capacity building to help members comply with new regulations, and suggest new regulations when we see they would be appropriate. We also give feedback to the regulator in terms of the practicality of regulations in terms of impact on MFIs. For example, if MFIs don’t have the data to comply with a certain new regulation, and can’t develop that capacity in the short-term – bringing in the regulation in the short-term will only harm MFIs.(Prudence) For me the question is whether there is such a thing as over- or under-regulation. What are we trying to achieve? First, financial stability. Second, financial inclusion. Third, client protection. These goals are sometimes at odds – and each new regulation needs to be evaluated against all of these. It’s not always a question of trade-offs – sometimes there are synergies.Also – there is such a thing as industry convention and self-regulation. For example, sometimes on key issues we go to the industry first to ask them “what can you propose to solve this challenge?”, otherwise we would come in with regulation (which really motivates them!)

Q: Prudence – in your view, what role should associations not play?A: What we see amongst associations is that when we see problems, there is a tendency among associations to close ranks around members. Associations should be more open about issues of doubts – and also be open about possible solutions.

Q: Jackie, what is your experience? Have AMFIU been in the situation of deciding whether to bring an issue of concern to the attention of the regulator?A: Not yet, fortunately. But it is our role to ensure that our members are professional and follow a code of conduct. So when it comes to telling the regulator whether a particular is not abiding by the law – if a particular standard is clearly set out, it’s possible and advisable to approach the


regulator. The association is a professional body, and poor conduct by members weakens our own position within the industry. Of course it’s easier to signal problems at the MFI level when it comes to standards of self-regulation, rather than regulation.

Q: Prudence, what do you most appreciate about your current cooperation with associations? A: We appreciate that we get a real feel for the market, and this serves as a good “reality check” from real-time market data in terms of new proposed regulations.

Q: If you could turn back the clock on your association-regulator relationship, what would you do differently next time?

(Jackie) Engage more, and not just limiting it to quarterly interfaces. For example, a lot of our members had concerns about the new MDI regulation, and we should have taken these concerns to the regulator more often. (Prudence) For me, it’s more of a challenge. We need to engage consumers as stakeholders more. We don’t know whether to organise them ourselves, because we don’t want them to feel like simply a puppet organisation. Other places have stronger consumer movements, and we can learn a lot from them. It’s also challenging to coordinate with co-regulators, because we don’t always see eye-to-eye.(Javier) We’ve proposed many ideas to the regulator, and they don’t always take them up. I think it would have been better to lobby harder for fewer ideas, and be more selective about what we take to them. We also need to identify the key decision-makers within the regulator, rather than engaging them as a whole. In short: a more focused approach.

Kinga: we sometimes forget that regulators too need to build their own capacity. Sometimes they don’t take up ideas because they don’t have the capacity to do so. At MFC, we’ve spent a lot of time building the “know-how” of regional regulators in key topics such as client protection and SPM. For us, policy and peer-learning forums worked well – where we brought together less and more experienced regulators to share ideas.

Questions from the audience:Q: It’s also important to consider what your capacity to take on certain roles that the central bank gives you. To what extent have you analysed the capacity of your team in this way?A: We’ve not done this. But one of our key roles is around information provision, which we are already doing – so when a regulation requires new information, the marginal additional cost can often be small.

Q: In Sa-Dhan, 70% of MFIs are reporting to us. But there are different legal forms, and so different regulators for each. This creates a challenge in terms of coordination. Have you had any experience with this?A: (Prudence): We don’t have one comprehensive law covering all providers. But we do coordinate between regulatory bodies. For example, we have cooperatives offering microfinance services, and we engage the cooperative development authority to ensure that our regulations are consistent across the board (for example on transparency and disclosure).

Q: Jackie: You have a code of conduct in your association – what has been your experience in terms of enforcing it?A: We have a code of conduct, but at the moment it is only applicable to our members. We monitor compliance and build capacity where members need it. However, we are currently lobbying for it to be a national regulation – this is the only way it will have a meaningful impact in the sector.


Q: What has been your experience in terms of integrating social performance and client protection into existing regulation?(Prudence): The Philippines is quite lucky in that social performance and client protection are firmly embedded in our regulatory framework. And we exposed other regulators to this idea in the hopes that they would also adopt this. We have a number of financial inclusion initiatives which offers incentives and paves the way to reaching the most remote areas.(Jackie): In terms of client protection, we are working with the regulator to integrate this currently. However, when it came to social performance, the regulator thought this was overkill, and should be an institutional choice. Our point was “then how can you tell the difference between an MFI and a money lender?” In response to this lobbying, they defined an MFI as an institution with a double bottom line.(Javier): We collaborate on SPM with the government – and in the latest law it’s mandatory that all credit unions need to show a positive “social balance”. There’s no regulation at the moment about the process to arrive at the positive social balance result, but that could come in at a future moment.

Q: What is one piece of advice you’d give to associations and regulators that would improve their overall performance?

(Jackie): In our context, dialogue works best. It promotes an understanding between the two institutions.(Javier): We need a technical committee, because a lot of the issues are very technical.(Prudence): Clear roles and responsibilities are needed. When it comes to sensitive issues, regulation would cause disruption in the market, whereas a quiet informal agreement with top management can work better.

Regulation Breakout: The Future of Responsible Pricing

This panel discussed current efforts by multiple stakeholders to define and encourage responsible prices among financial institutions, as well as future solutions for improving the transparency and affordability of products and services for clients.

Speakers: Tony Sheldon, SPTF; Isabelle Barres, Smart Campaign; Christophe Bochatay, Triple Jump; Wesley Jordan, Vision Fund

Tony Sheldon (Facilitator)Pricing lies at the core of how we think about balancing financial and social performance. Over the last 7 years, MFTransparency helped raise the issue of pricing to prominence by focusing on transparent pricing and then acting on that information. How do we think intelligently and act appropriately when it comes to pricing? To determine this, we need to look at some thorny issues, such as:

Is transparency enough? What does responsible pricing mean? What kinds of issues do we need to take into account when assessing whether pricing is

responsible? How much profit is too much? Pricing is the heart of profitability goals. What can the industry do to promote responsible pricing, particularly as

MFTransparency is winding down?

Q: Is transparency enough? Does transparent pricing create responsible pricing? Does transparency lead to appropriate pricing policies?


Wesley Jordan: Transparency has its limits. MFIs or banks or companies can be transparent about very high pricing. There can be an argument that as long as there’s demand at that price, we’re satisfying that demand.

Isabelle Barres: Transparency is not enough. You could have transparency but pricing that is inappropriately set. An institution needs to be sustainable so it’s there in the long term. At same time, you can’t set prices that are artificially high. The sweet spot allows institutions to grow and be sustainable but allows clients to thrive.

Cristophe Bochatay: Transparency is crucial, and we’re still far from there. Our sector is not completely transparent on pricing. It is indeed not enough. Transparency plus competition should lead to lower prices. Transparency is not the only factor.

Tony: Early on, the hope was that transparency would lead to responsible pricing. Banks and investors would see information that would lead to appropriate action. However, that hasn’t necessarily happened.

Q: Isabelle – in your view do our indicators about responsible pricing reflect what it should be? What are challenges and limitations? What is the definition of responsible pricing?

A: We realize the Smart Campaign definition of responsible pricing not ideal. There’s more work that can and should be done. We have three indicators that measure whether organize are pricing responsibly:

o Pricing of main products (more than 25% of client base). We look at these prices compared to peers. We look for organization that could bring down prices because peers have done it but they are not doing it.

o Efficiency . Organizations can control their operating expense. We look at efficiency related to peers.

o Fees . As we all know, the interest rates are not the only costs. Are fees reasonable? If clients repay early, are there unreasonably high fees?

We are working on improving the responsible pricing principle, and we are taking a number of approaches. We have talked with debt and equity investors and are taking the pulse of the investor community and seeing how they think about it.

Pricing is complex. There are a lot of elements that come into it. Could have a system that red flags MFIs above certain amount of profit and then have to go in and look more closely.

According to market assumptions, prices should go down with increased competition. We’ve found that prices go down over time but not because of competition. Rather, they go down because loan sizes have gone up. We’re looking at markets where prices are much higher in the world and looking at pricing to average loan balance. One main driver is average loan balance.

We are looking at the evolution of pricing over time in key markets and at average loan balance and with competition. We’re hoping to gain more realistic sense of whether the current indicator is sufficient. For example, if we find that in really competitive markets, pricing has not gone down, then we will reopen the discussion.

Q: Wesley, what does the market-based approach leave out?A: It’s a starting point. They’re continuing to refine it. The problem is in markets that are not competitive, with a lack of information in the market. The market-based approach is a natural start and a way to pick out what data are available.

Q: So, transparency is useful, but not quite getting us to what is responsible pricing. Then, next question – do we also have to look at profitability? What’s responsible profit?

Cristophe: We definitely look at pricing and profitability together. Whenever we see high rates, we look at profitability and vice versa. We’ve developed tool that tries to


gather factors to consider when assessing profitability and pricing. It’s helped us frame our analysis on this subject.

o There are different factors that we look at – APR vs loan sizes, pricing vs peers in the country/product peers and added value to clients, although nobody really knows how to assess that.

o We also look at costs and efficiency factors (, i.e.level of salaries in countries, particularly rural MFIs with long ways travel to clients, infrastructure constraints); borrowing costs; inflation; use of profits (distributed to shareholders or stay in institution, equity cushion)

o Then have traffic light to classify: Red – Will not finance Orange – Would require commitment from MFI to lower rates within 12-

18 months Green – Finance with no condition

For now, it’s a due diligence tool, not monitoring. We do track ROE and ROA of clients. If they surpass threshold, we would address the issue. We don’t have loan covenants or formal way to react to that. It would be a discussion in the loan renewal phase.

Wesley: VisionFund Cambodia is part of VisionFund International, and we have a mix of VFI , World Vision and independent board members. A few years ago, after seeing healthy growth and healthy profitability, there was pushback from board to say, “OK, this is good, but we’re here to serve the poor. We’d like to see management take steps to reevaluate its pricing and reassess the pricing structure.” We’ve seen prices come down over years. Part of that is pressure from board, also because of competition. From VFI, it was from the governance level where that came down.

Isabelle: Smart Campaign says a client is affected by price, so that’s what we look at. There can be one institution that has high profit but is inefficient and another that has low profit and is very efficient, and they could have the same price.

But, it’s a perfectly relevant question to ask what profits are used for. A lot of discussion about level of profits, but what are these profits used for? Who benefits from these profits? When we talk about responsible pricing, are we looking at whether MFI is responsible or is the price responsible? MFIs can control some aspects of pricing, and they can’t control others. MFI could have shareholders with certain return expectations, which will drive level of profit they have to set. What if that is not responsible? Can we say the MFI is not responsible? Or is it the shareholders that are not responsible? It ties back to transparency. To get this data, we need more transparency at all levels. It also ties into product design. We could have products that are priced responsibly, but let’s say a client has to have three loans instead of one because the products are not aligned to needs. That leads to a situation where client ends up paying more.

Q: How do we think as industry of gathering reliable pricing information? Without MFTransparency actively involved, how do we keep this momentum going and ensuring that pricing transparency, as a minimum, continues as an industry standard?

Isabelle: It’s a huge challenge, especially now that MFT is phasing out. Even with MFT, we don’t have pricing at a product level for many countries. Now, we often have to use yield as a proxy because we don’t have APR for enough institutions to do the peer comparison.

We also want to make sure clients have access to the institution’s information. There are a lot of discussions. One option would be finding way to collect pricing information through raters and then find a way to store data on a common platform. We can learn more about key drivers and have better benchmarks.


Cristophe: MIVs all gather APR data for MFIs we finance. It’s not as rigorous a methodology of MFT. With help of new SPI4, we’ll manage to actually assess the APR in the same way so it’s comparable. There is a possibility for us to construct benchmarks.

Q: It seems we have need for shared standards on how pricing is approached, a need for more collaboration, the need to define who has access to what information. In the next phase of pricing transparency, what are the roles the different stakeholders need to play to support responsible pricing? What incentives are needed at each element to further promote pricing transparency?

Isabelle: To have responsible pricing, we need to have transparent information. We need to have detailed information on various components of pricing. So, a first element is transparency for all involved. Then, look at what is in control of those various actors and hold stakeholders responsible for what they can control. For incentives, it’s well suited for SPTF to take on and try to tackle together. For investors, what are incentives for MFIs? If investors demand information, it could be a good stick for MFIs to provide information. Or, if MFIs provide information, they could get preferential rates or not get funding. That’s a pretty compelling argument.

Christophe: On debt side, we have to practice responsible pricing ourselves. We don’t have very formalized policies on that in general. We have to have realistic return expectations from our asset owners. There, more data is useful to make that case. Responsible pricing also important for foreign exchange hedging. On the equity side, we have responsibility of setting responsible profitability targets. Here, we don’t have very formalized policies. It’s difficult to have discussion with the other shareholders as well. As investors, we can analyze pricing, contribute data and try to not finance MFIs with irresponsible pricing.

Wesley: From a practitioner perspective, it’s hard to find the right incentives to be transparent on pricing. It could be pressure from regulators, owners or lenders. It’s important to find voice for clients. In more mature markets where you see APR become much more transparent, clearly defined and comparable, it often comes from pressure from the clients. They need to have a voice and that voice can be made louder through country associations or regulators to demand transparency and comparability. MFIs can try to push for transparent information, but it’s hard in competitive marketplace to convince peers to do that. From a network perspective, VFI has encouraged transparency and signed onto MFT and Smart Campaign to endorse. But to enforce with country-level of MFIs, there are risks of being first in transparency.

Discussion: Jürgen Hammer, Grameen Credit Agricole/SPTF: It’s a good idea that this debate takes place. Incentive as industry is survival. This is the most critical point of attack. We need to have a call for involvement at all levels. We don’t do that yet in a sufficient way. MFT was trying to get stakeholders to feed that information. We now have to then use that information. That would be taking the step further. Also, we MIVs need to demonstrate that commitment by applying those things. How many MIVs publish an APR to their MFIs? We have to set up an initiative that collects that info and makes it available, but then we all have to commit to that work.

Audience question. I run an NGO in Bangladesh that provides entrepreneur product, with lots of extra value add, not just credit. In Bangladesh, there isn’t any scope for small organizations to give different rates. Depending on the product (i.e., entrepreneurship rather than just general credit), can you change the pricing?

Isabelle: This is an interesting example. When we look at responsible pricing, we look at average loan balance. It’s not that it’s the only thing, but it’s that when you look at everything else, it’s too complex and there are too many factors. But yes, when you look at all the other factors, if there’s a product with all the extra work and cost, it would make sense to have a higher price.


Wesley: Challenge sounds like you have some sort of restrictions in pricing flexibility. Isabelle: It also illustrates why it’s really hard to have hard and fast rules. The best

approach is to have something that triggers more questions and then look into the various factors that affect pricing. Peer comparison would need to be done by product peers, not country.

Carmen Velasco, SPTF: Twenty years ago, I was thinking that clients didn’t care about the interest rate as long as they had access. But now, they are aware of different interest rates. It’s important to know, what is being done to empower clients to be able to know what they’re paying. We should make it the case that nobody is protecting clients but that the clients are empowered to know and make the decision.

Isabelle: I agree. When we think of client protection, it’s not just what MFIs can do to protect the client. It’s also about what others, including the client, can do. We see it as a three legged stool: regulators, industry (Smart campaign) and financial capability. From Smart Campaign perspective, we look at whether clients understand what they’re going to be paying.

Investor: We need engagement at asset owner level. Asset owners say “We hear microfinance can yield 25% return,” and then we have to model that. They’re being told stories that microfinance is just another asset class and therefore social and financial goes hand in hand and there’s no tradeoff. DFIs are the same. Our business development team struggles to talk to asset owners and explain to them that microfinance is different.. They don’t listen because they look at other MIVs, which say they can get those returns. We need to go to asset owners and raise awareness.

Second, we are asking MFIs for their annual rate, effective interest rate, sample repayment schedules, etc. What about us? We need to be transparent about annual rate, penalty fee, upfront fee. I’m pretty sure most of my competitors would agree. The CEOs of the members of the Cambodian Microfinance Association should get together and write a joint letter that says “You’ve been having us be transparent. We’ve been borrowing from you for the past 10 years, and I also want you to be transparent.” I can’t do it by myself, but if it comes from all the institutions together, it might be more impactful.

Audience comment: There are no regulated prices. It’s the responsibility of the government. Wesley: There’s a role for the regulator and the network association. The customers

have demand it. The MFIs aren’t going to demand it, one by one, because we’re selling. The customers are the best place to demand it.

Tony: SPTF has developed a memo describing how networks can engage with regulators. It’s on the SPTF website, as well as the Annual Meeting flash drives.

Audience question: When is dividend too high, and do you include the profile of the investor? Christophe: We look at level of profitability. If ROE is above 25%, that’s a red flag for us.

We would look at where the money would go. Would it go back to the sector or would it go back to the investor? We don’t have a specific rule on acceptable level of dividends.

Lubna Tiwana, Khushhalibank Pakistan: MFT data is based on utilization of products. What if institution has multiple products and some products are much higher priced but not utilized by clients? Just because they’re not utilized now doesn’t meant they won’t be later.

Tony: Chuck is trying to capture the lessons and hone the tools in terms of simplifying and clarifying their use. There will be a suite of tools available and you can look at a product and determine the APR.

Frank DeGiovanni, Ford Foundation/SPTF: Microfinance organizations exist in a value chain. Some things they control and some they don’t. When you look at pricing as an investor, you are


able to disaggregate an MFI’s cost of funds. How much of this is their own choice and how much of it is because they made unfortunate funding choices themselves? Also, social performance – what are they doing with their clients? Are they tracking outcomes?

Christophe: For cost of funds, we ask the interest rates of loans. There’s a big variety of funders, both local and international. Depending how mature and how much bargaining the institution has, you can find a large variety between the rates. As for social performance, we’re not tracking outcomes on a systematic basis.

Audience comment: In terms of accounting, financial principles and accounting principles, there should be differentiation among scale of operations. In a big org, they should not be charging for travel but could do skype, etc. In a small org, they can charge the traveling costs. It’s very micro, nano things.

Isabelle: That would be taken into consideration in the peer grouping. Jürgen: If I look at other industries, I don’t see a natural trend to be transparent. It has

to be imposed, either by regulator or someone else. Has to be all along the value chain. Tony: Do we need regulation to be transparent?

o Christophe: Regulation seems like the best way. We need laws and regulation on transparency. I’m not sure who the best stakeholders are to advocate for that – DFIs or someone else?

o Isabelle: I agree. There are different levels. A reasonable expectation from the industry is to share data to common database so we could at least have better benchmarks. With sharing individual data, there are clear disadvantages for being first-mover. So MFT method of doing all at the same time was a good one. As for regulators, MCWG and pro-bono lawyers developed model legislation framework. There is a full section in there on transparency.

Wesley: Ideally self-regulation is nice so the industry can take on that responsibility. Challenge is that new actors might have competing interests.

The Last Mile in SPM: What are the credible options for tracking client outcomes?

In the context of the publicity around RCTs, this plenary session explained the work of the SPTF Outcomes Working Group thus far, drawing lessons from leading practitioners and social investors who are exploring client outcomes in microfinance. The breakout workshops on client outcomes (see notes below) take on key themes in more detail.

Speaker: Frances Sinha, EDA Rural Systems

In many ways, the outcomes question is the first one we need to ask, even though it is the last part of this meeting.

Background on the issues“Are we improving clients’ lives? Are we contributing to change? Who is benefiting and how?” Easy questions – but alarmingly difficult to answer. Why is this?

- Diversity and confusion: We all use the word “impact” in different ways, we have different methods and tools that require different levels of skills and resources to apply, we don’t always agree on what “change” means when we set indicators, questions around validity and bias (depending on who has done it), or we can’t bear to wade through another long report…

- Spectrum of tools: Sample curvets, focus groups, case studies, etc.- Spectrum of approaches: Academic studies, RCTs, etc.


RCTs are much in the news these days. We all know what they are: large samples to assess the statistical significance of average results over a short term (6-18 months). This January, a microcredit RCT in 6 countries over 6 years essentially concluded that there was no real impact. Another RCT looked at the impact of graduation programmes, and found promising results. Thinking about RCTs, we need to bear the following in mind:

- Choice of indicators must capture short-term change. Moving out of poverty is a long-term process.

- Income and consumption measures used to estimate poverty can be problematic. The PPI can be more reliable.

- Reports tend to report statistical significance rather than actual numbers of people moving out of poverty. If we have a large sample, and you have a 5% increase for 5% of your sample – that’s significant but not telling us about how many people improved.

- We also see a wide variation in findings, where averages are reported, which masks the range between minimum and maximum amounts (e.g. in terms of business income). Increasingly when we’re thinking about what works for whom – we need to think about variations in change, not just whether there is change.

What counts as good evidence?The Research Unit for Research Usefulness at the University of St. Andrews published a paper in 2013 called “What counts as good evidence?”

- There’s no simple answer, and no clear answer.- It depends on what we want to know, why, and in what context the evidence is meant to

be used.- There will always be dissenting voices and alternative perspectives.

The Outcomes Working GroupWe’re talking about outcomes, not impact. We’re not excluding impact – but we’re not fundamentally focused on it. We’re looking at client-level change that is plausibly associated with our services.

The purpose of the group is to develop practical guidelines for credible measurement and reporting of outcomes, drawing on experience with different approaches and tools. As part of this, we’re looking at the different methods and tools and learning from good examples (around indicators, framing of questions, research methods, quality, analysis and reporting) and also looking at the use of findings (in terms of reporting and improvement).

When we talk about developing guidelines, we want to base this on what we call credibility checks. We want to look at:

- What we measure: what are reasonable expectations in terms of what is practical to measure, and how it links to a theory of change

- How we measure: what are the practical options, what are the quality checks that we need, how does that apply to different methods (qualitative vs quantitative, in-house vs. outsourced, etc.)

- How we analyse: not just averages but segmentation, what types of analysis are the most relevant and useful

- How we report and use data: concise reports, with clear implications

To date, we’ve held three webinars to look at the first of these questions – and follow-up webinars will look at the other three.


Client Outcomes Measurement Workshop: Enterprise development and employment outcomes

This workshop discussed ways to measure enterprise growth and employment outcomes among clients. Participants discussed indicators, data collection approaches, and reporting tools.

Speakers: Alok Misra, M-CRIL, Margaret Richards, Mission Measurement

Margaret Richards introduced the Impact Genome Project (IGP), a comprehensive evidence-based research project to codify the factors that drive social change. It looks at the existing research in many different areas to identify what can be learned from it. Mission measurement is part of the IGP and is focused in microfinance.

The methodology (“Genomic Process”) reviews existing research publications, then breaks them up into efficacy and contextual factors creating a master category for each of these two types based on all the data reviewed. At the same time, effect sizes (how much of certain thing is happening/to what extent what happened caused the outcome in question) is also calculated. When there are gaps in research, these are filled by gathering input from practitioners and experts. All this information is what then enables to identify the factors that contributed to a specific event. There are also impact scorecards that provide actionable data to funders, governments, and practitioners.

When trying to identify leading microfinance outcomes, mission measurement found related 650 terms which were then grouped into broad indicators. The number of existing indicators is being used as measure of interest in the industry (what are people searching for? What are they interested in knowing?).

The group then broke into smaller groups to discuss case studies and then report back to the whole group on specific questions.

The case studies were: Rule of Thumb – study in Dominican Republic looking at the quality of training and how

that leads to changes in terms of business volume Hand in Hand – study in India focused on adoption of good business practices and

effects on enterprises outcomes and jobs supported Tao Yeu May – study in Vietnam focused on the impact of risk management training and

insurance on financial outcomes Business outcomes survey – survey from 1992, to compare how things were measured

before and now

Questions posed to the group: What is learned about the clients through the use of this data?Hand-in-hand

Majority of jobs supported were in rural areas 87% of loan used for productive purposes Split of job creation by gender

Rule of Thumb Program was adapted to clients need and their capacity of understanding As result of one of the program clients started to keep accounting books (among other

changes) which led to increase sales of their businessesTao Yeu May


Insurance training helped reduce OID (instead of borrowing more for emergencies, clients used insurance products) and increase savings of clients

What decisions might an MFI take based on this data?Hand-in-hand

Might decide to continue focusing on rural areas Target family-based enterprises Create another loan size to diversify portfolio

Rule of Thumb When doing a financial education program its good to analyze client capacity before

developing the program (to make sure they understand the content) Identify which clients will benefit the most from this type of program Pilot test the training program before launching it in a big scale

Tao Yeu May Institution might be interested in introducing an insurance product. Also consider

expanding insurance (e.g., to health) Consider training as a core activity (especially with new product launch)

What data do you feel you lack or would like in addition to data presented?Hand-in-hand

Would like to see cash flow analysis of the business, type of business, whether the business included child labor, whether the jobs created were stable

Rule of Thumb Content of the training Whether clients improved their knowledge (beyond change in behavior) Whether clients were satisfied with the training

Tao Yeu May Would like to see changes in before/after Long term impact on livelihood Indication on whether they kept their jobs

Which indicators discussed in your case study do you believe could be broadly applicable and of interest to the industry as a whole?Hand-in-hand

Gender - Number of women in the household working per enterprise Productive use of loan

Rule of Thumb Cost of program Change in client behavior leading to financial outcomes

Tao Yeu May Insurance ratio (to decrease OID and increase savings) Savings ratio

What are good indicators to measure job creation? Number of jobs created (differentiate whether is a new job vs. a recovered job, whether

job is full-time vs. part-time, or whether is seasonal or not) Type of job created Also important is to measure whether growth includes child labor

How can we pick a few good and relevant measurements?Seems like some people are too focused on big data. What about doing a study with a smaller set of people but that goes very in-depth in the case of each people? – Both can be done.


How can quality of training be measured? (not just look at whether a training was provided).

End-of-training survey Measure improved knowledge

Client Outcomes Measurement Workshop: Savings outcomes

This workshop discussed how financial institutions can measure and encourage client savings habits. Examples from CARD bank (Philippines) and EFC Zambia illustrated different ways of measuring savings outcomes using simple yet innovative methods, including using mobile SMS technology, transactions data, and even an RCT.

Speakers: Alex Fiorillo, GRID Impact; Anu Valli, Bamboo Finance; Frances Sinha, EDA Rural

Anu Valli of Bamboo Finance worked with EFC Bank in Zambia.

In 2009, EFC transformed to be able to take savings. The savings accounts had low balances (less than $200) and about 35% of the almost 25,000 savers were women. Most of the MFI’s credit customers were men. Bamboo Finance wanted a robust, low cost methodology to track outcomes rather than outputs. The budget for the research was $5,000 so they decided to use a survey delivered through mobile phone technology. The main research question for the investor was: Do savings products create positive impact on clients? At the same time, the MFI’s objectives for the survey were centered on how to increase savings deposits among their customers. The survey sought to address both these goals.

The mobile phone based survey was offered by TTC (Text to Change) and the mobile operator offered a monetary incentive to users for their participation in the survey of $0.75. The money was provided as a credit on the mobile phone usage of the customer. A mobile phone survey must be short – no more than 12 questions. (For survey questions see the PPT). The survey was delivered to both clients and non-clients of EFC and the survey took approximately 20 minutes to complete. The survey questions were designed through a collaborative process between Bamboo Finance and EFC Bank.

For statistical significance, EFC needed to obtain at least 400 responses and so TTC sent the survey out to over 20,000 people and only 1002 people responded. Of those who responded 400 were clients and 602 respondents were non-clients. Both sets of samples had about 35% of female clients. Because the survey response rate is quite low, the MIV wondered if there could be a bias introduced or a type of mobile customer than was more likely to answer this survey than others e.g. better off, less entrepreneurial, less rural, and these characteristics may be different from their average customer. But given the budget constraints this was as close as they could get and the results were useful so better than not having the data, which was the alternative. EFC Bank was quoted a price of $90,000 for the same type of study but not done through the mobile platform.

Challenges: Budget – there was only $5,000 for the survey. Survey had to be short to accommodate mobile platform, so they were limited to 10

questions. Synchronizing the goals of the three organizations – Bamboo Finance, EFC Bank, and

TCC


TCC included lots of clients in the survey that were in areas where no EFC branches were located. This made the results less useful to the MFI and is a lesson learned – next time make sure they only send the survey to non clients who are potential clients because they are located in a region where the bank operates.

Bamboo Finance was organizing the project and designing the questions but they were not on the ground so it was hard to get close to the client and understand the real situation.

Results: 55% of the users most recent withdraw transaction had been used to invest in the

business. 73% of those considered that their savings had helped to grow their business. Customers’ complaints centered on a lack of understanding of the terms (many did not

know their accounts accrued interest), a lack of respect, and a need for more convenient access to ATMs.

The MFI identified areas for implementation of improvements that included a need for staff training for savings outreach as it differs from credit promotion.

The MIV had to convince the MFI to apply the results of the survey to enhance their operations.

Alex Fiorillo of GRID Impact worked with CARD Bank in the Philippines Grid Impact is a data consulting firm that worked with Grameen Foundation and Grameen’s affiliate CARD Bank to analyze why customer’s were not using their savings accounts. CARD had previously participated in Grameen Foundation’s three-year Gates funded savings project to help them develop two new savings products, including an ATM based account. However most clients did not open savings accounts and many clients (72%) who opened accounts never used them, not even once after opening the account. This study sought to answer why this was happening and what could be done by the MFI to encourage usage of the accounts and more extensive savings by their members.

Because of loyalty to CARD most of the clients who opened saving accounts felt like to be good members they should participate in all of CARD’s products. However since this was the clients only reason for opening the account, they did not use the accounts. Those who used the account even once usually continued to use the account in the future, usually with one transaction every 2 weeks. However even those who used the accounts often did not have a savings goal in mind and many continued to save the majority of their surplus income in informal ways.

Grid Impact carried out a low cost version of a Random Control Trial (RCT) using the MIS data from the MFI as a baseline to save on performing an additional baseline study. CARD’s MIS was greatly enhanced during the Gates Foundation savings project, which made this possible. Grid looked at both analysis variables (transaction and administrative data) and outcomes variables (see PPT for full list of outcomes variables). In addition to the streamlined RCT, Grid Impact performed “immersive behavioral research” - which consists of ethnography, observation, and in-depth interviews - with about 80 staff and clients of CARD. The research methodology uses human centered design to look at the motivations of clients in how and why they open savings accounts for both the clients that used the account to its maximum potential and those who did not use it at all.

The “RCT Lite” was conducted over 6 months to see if the “treatment” designed by Grid Impact would produce the desired results. The treatment was piloted in 3 branches and for these branches the consultants reviewed all transactional data for the previous year as well as monthly data during the study, including end of month summaries and weekly data on new


account openings. The treatment consisted of several interventions bundled together and included:

establishing a savings goal with the client, developing a savings plan to help the client reach this goal, redesigning the application form to simplify it (from a 5 page form in English to a one

page form in the local language), a gift for opening the savings account e.g. CARD branded calendar, and SMS reminders about the savings account and savings goal for both staff and clients.

The treatment was applied for 2 months and then the results were monitored for 4 months. The treatment group was around 1900 people and included three groups: those without savings accounts, those with dormant savings accounts, and a control group.

Challenges: CARD’s MIS is product based rather than client based, so there is no way to look up a

client by name and see all the products they have at one time. Each product has its own database and these are not linked.

Grid required that CARD not introduce any new products in the pilot branches during the 6 month term of the study, however CARD did launch a new product in those branches and it affected the data collection. To manage this problem, Grid selected a subsample of the original sample to retain only those clients that were unaffected by the launch of the new product. This resulted in a smaller sample size than anticipated but due to the original over-sampling the total number of clients in the subsample was still large enough to be statistically relevant.

On an operational level the weekly transfer to transactional data from CARD was often late, which delayed the researchers’ timeline.

There was no point person from CARD to support this research project consistently and learn the researchers’ processes so that the work could continue internally after the project was over.

Results: Grid produced a data dashboard with a summary of the key information from the MIS

data for CARD. Clients in the treatment group were 73% more likely to USE their savings account

(indicated by performing at least one future transaction after initially opening the account). They were also 60% more likely to continue to use the account actively e.g. more than one future transaction.

Clients in the treatment group made 15% larger deposits than the control group. Of the clients in the treatment group, 37% had higher balances at the end of the pilot

than those in the control group. Clients who already had accounts at the start of the pilot made larger deposits and

larger withdrawals, and in general used the account more actively than those in the control group.

Of the set of interventions in the treatment, CARD decided to implement on an ongoing basis the new application form and the instituion retrained staff to talk to clients differently about savings, including the idea of setting a savings goal when opening the accounts.

Client Outcomes Measurement Workshop: Women’s empowerment outcomesThis workshop discussed a set of women’s empowerment indicators developed by Women’s World Banking. WWB shared their experience developing and using the indicators with

katherine, 16/06/15,

Roberta – add notes here.


network members around the world. Participants also provided their own input and experiences with measuring and reporting women’s empowerment indicators.

Speaker: Jaclyn Berfond, Women’s World Banking

Client Outcomes Measurement Workshop: Health outcomes

This workshop discussed a set of health indicators developed by Freedom from Hunger. FFH shared their experience developing and using these indicators with network members around the world. Participants provided their own input and experiences with measuring and reporting client health outcomes and discussed next steps for applying and refining health indicators.

Speaker: Bobbi Gray, Freedom from Hunger

Outcomes Workshop 4: Health OutcomesLeader: Bobbi Gray, Freedom from Hunger

Theories of Change: Improved Health Freedom From Hunger has been doing microfinance+ health to improve households for

about 15 years. Over time, have moved from simple health programs to providing health services, linkage to health providers, health savings and health insurance.

FFH had been doing various evaluations and randomized control trials. Decided it wanted to look at the questions it was asking. Asked, are there any questions that are particularly useful? What indicators could we use that people would be able to measure over time? If MFIs were already using a PPI, could they add a few indicators to measure health?

To do this, took 30-40 plus indicators and examined them to say, what would be good indicators?

Theory of Change: Different pathways to improved health outcome Financial service providers increased income, balancing income with expenses

(consumption smoothing) seek prompt medical care, seek preventive health care, coping with health risks

Health providers client health education, medical services improved health knowledge, seeking prompt medical care, seeking preventive care

Integrating health and microfinance direct provision and linkages between sectors cross-sectoral efficiency gains in provision of financial and health services to poor population improved health knowledge, seek prompt medical treatment, seek preventive health care, coping with health shock

Theories for lack of change in health outcomes: Bobbi asked the group to think about the following questions:

o Why don’t people’s health outcomes change?o What are reasons people don’t change behavior?o What are reasons why people don’t seek prompt medical treatment?o What are reasons why people don’t seek preventive care?

Carmen Velasco, SPTF: It seems so obvious that if you taught them something, they would share behavior. But I learned it has to come from them. It has to be a felt need. They might not feel it’s needed. They might have other needs, live for today.

Ines Arevalo, AKAM: It’s a cultural thing. In China, there were things they were doing for centuries. It’s not going to change overnight. When it comes to behavior change, the message needs to be enforced from different agents.


Glenn Andre, AFD: There’s a huge lack of confidence in the different structures. They don’t think to go to the doctor because they know the quality of the health care is not the best.

Carmen: It’s not just the fear of being ill, it’s also the fear of the cost of being ill. People think, “If I’m sick, I’m just going to worry and it’s too expensive to get care, so it’s just best not to know.”

Bobbi: Projects rarely ask beforehand, what are the things that could go wrong? Knowing all these challenges, how could tracking changes in health outcomes be really challenging? You have to think of what is in your control to measure, and think about all of these other things in the background as you’re considering your indicators.

Choosing Health Indicators Feasibility – Did MFI collect and did it come from survey? Usable – Can they change in the short-term that could be valuable for MFIs? Relevant – Are they relelvant for FSPs? Some indicators are more related to treatment

types of issues. For FSPs that don’t have any health program, could be hard to measure maternal health

Generalizable – more applicable to general microfinance population. Trying to find indicators that apply to men and women

Usability and reliability Consistent measurement over time. Whatever you are capturing provides a consistent

measure. For example, birth control use. Is that by choice or because people can’t afford it?

Current pilot partners (sample sizes vary because FFH integrated data collection into their existing surveys, collections.) ADRA (Peru) CARD (Philippines) ESAF (India) Equitas (India)

Survey Adaptations. Designed it for India and then decided what questions need to be adapted for Peru and Philippines. Poverty measurement – Use of country-specific PPIs Food security and nutrition – Added a focus on food items in India to reflect stronger focus

on nutrition Preventive health care – Varies the most for each country. In India, asked about

institutional birth. In Peru, asked about annual exams and pap tests. In Philippines, asked about annual exams

Curative Health Care – Same question in every country: forgoing medical treatment and purchase of medicines due to cost

Water and Sanitation – In India, asked about open defecation; in Peru, asked about water sources and treatment; in Philippines, asked about treatment of water

Attitudes – Only measured in Peru and Philippines, access levels of confidence related to ability to cover future medical costs and seek adequate medical care

Example of Equitas in India Pilot-tested with cross-section of clients in 4th or 5th loan cycle. Fewer than 20% said they defecated in the open, which is lower than the national average. Identified indicators associated with poverty. How many health indicators are correlated to

their poverty level? Have theory that as people are less poor, they’ll have positive increase in health outcomes.

Two indicators (Water treatment and children under 5 receiving vitamin A) were the only ones that improved with improvement in poverty level. Everything else had a different driver behind it.


Danger in tracking indicators: in this particular region, poorer people treated water. In ESAF, wealthier people treated water. It’s important to ask these questions and understand the context.

Lessons learned Standardization of indicators may be difficult. Maybe the themes will be more useful than

the actual indicators. Providing people with a menu to choose form will be more useful than saying “Use this one.”

Proceed with caution in the interpretation of results. Baseline values will be important to establish. Originally, we asked if we could just do interviews with cross-section of clients over time.

Not going to work – how would you collect data? Baseline values with high levels of performance may not be useful to track. The value of statistical analysis (ex: correlations between indicators of interest with

poverty) can help refine our “Theory of Change,” as well as determine which indicators may be the most useful to help us understand changes in client outcomes.

Who to track and for how long? This is a very important question to answer, as it influences which indicators will be the most useful.

This process requires patience. There’s not a silver bullet for improving poverty.

Final list of HOPI indicator – categories Food security (Do you have enough food and they type of food you want?) Water and sanitation (Main source of drinking water, do you treat water, if yes, how so?) Preventive healthcare (Have you visited doctor for preventive health service? Have you

used a strategy to save money specifically for health?) Curative health care (Have you delayed medical treatment because of cost?) Attitudes (Confidence that you can afford appropriate medical care when needed) Domestic violence (Fear of spouse, is husband ever justified in hitting wife?) Mental health (Hopeful for future, satisfied with life) Maternal and child health (How old is youngest child, did you receive antenatal care with

pregnancy of youngest child? How many times?) Some indicators were not tested in the original HOPI survey, such as domestic violence

indicator on whether you were afraid to your husband/partner.

Audience comment: It’s very important to get the buy-in of the practitioner to use the indicators. Here, none of the indicators help them introduce a financial product. Can we think of adding questions, like “Do you have a toilet at home? If no, would you be interested in a loan product.” That could help generate buy-in. Comments: Bobbi: You could do your baseline with incoming clients and decide to add several

indicators that you could narrow down over time. The issue might be better for a market research department than a monitoring department.

Carmen: It’s important to not project. If you add the health component to your microfinance clients, what’s going to happen? Is it worth it? Everyone is doing huge efforts to include health, but is it to say that I am microfinance+? Is it for the face of the institution, or is it because I want to prove that X and Y indicators have improved in my clients? I don’t want to be too rigorous in attribution. If, after 5 years, the same people that came into the program have changed their health condition and behavior, those are the kinds of outcomes I want to see. I won’t say it’s necessarily because of my institution.

Ines: We may or may not want to enter discussion of attribution. You may want to say we can talk about contribution with a different tool or different intervention. It all depends on what you want to look at.

Case of CARD in Philippines


They’ve been providing credit plus education (health, business, financial education). In 2006, worked to go deeper in health and decided to provide a health loan to pay premium for national health insurance (PhilHealth). They have a network of loan providers that provide discounted health services and have created a micro-franchise pharmacy type structure.

Had client satisfaction survey about to happen when FFH asked them to test indicators. Included the indicators in their survey.

How would you look at these results from the perspective of the MFI?o More than 30% of respondents were food insecure, which is scary. Are these

people able to repay a loan? Are we making things worse? Food security is very sensitive – fluctuates a lot and have to be careful

about interpretation. But statistically, the more food secure you are, the less poor you are.

Food security tends to be an issue in the bottom 75% of income. In ADRA case, clients were above poverty line but many still food insecure.

CARD had more clients enroll in PhilHealth than people in the informal sector enrolled, which shows that they are reaching the clients they want. Part of their education talks about PhilHealth, and they help their clients fill out the paperwork.

Bobbi: Anything missing from list of indicators? o Didn’t include sanitation in all countries because thought health was more

relevant. o In CARD, 10% delayed medical treatment due to cost. In ADRA, more than 60%

delayed, despite a free public health system in Peru. Is it because they want to go to private health care?

Client Outcomes Measurement Workshop: Quality of life (poverty & risk reduction) outcomes

This workshop discussed a set of quality of life indicators used by Opportunity International. OI shared their experience developing and using the indicators with network members around the world. Participants also provided their own input and experiences with measuring and reporting quality of life outcomes.

Speakers: Nitin Madan, EDA Rural Systems; Calum Scott, Opportunity International; Pum Sophy, AMK

Presentations

Sophy Pum (AMK Cambodia)AMK’s 7-person research department aims to create a systematic way to collect, store, and analyse information about AMK clients and the MFI market in order to provide management with outcomes as an input for decision-making. The department asks two types of questions:

- Social research: Does AMK really reach poor people? Is AMK having a transformational impact on them? Used to check alignment with social objectives.

- Market research: Do AMK products fit the needs of the market? What are key market trends? How does AMK perform compared to its competitors? Used to develop new products or modify our systems and products.

The research department gets technical support from the social performance committee (or SPC, which sits at board level) that checks that our research methodology is strong. Findings are communicated to management so they understand current issues regarding AMK’s social


responsibility towards client and staff, and improve current products and develop new products.

Our mission it to help large numbers of poor people to improve the livelihood options of large numbers of clients. We check:

- whether we’re reaching large numbers of clients- whether clients are poor (through our key poverty dimensions – see below)- whether their livelihoods are improving- whether the services are adequate and transparent

Our key poverty dimensions are:- physical assets- expenditures on food, clothing and footwear- social capital: depth/breadth of an individual’s social networks within the village often

depend on poverty level- vulnerability: ability to afford large expenses and the ability to save- food security and quality of diet- human assets: children’s education and affordability of heath care

Our change studies use a rigorous sampling methodology which compares clients with non-clients. It also involves:

- intensive training of data collectors (if enumerators are hired to collect data)- physically cross-checking the questionnaires (at least twice per questionnaire by two

people who were not involved in data-collection)- digital data entry: each questionnaire is entered by two different team members- digital data cleaning

In terms of poverty findings, we focus on the following trends: change in well-being categories, changes in well-being scores, changes in each principle component analysis (PCA) and in other key poverty indicators, and self-assessment. By self-assessment, we mean including a question where we ask clients with a qualitative question to reflect back on their life over the time period, as a way of validating your data.

Lessons learned:

Methodology:- track change over 5 years, not 2. this is because change takes time, and also we

introduced new products during the time period, which might have affected the change. also account for externalities (climate events, political/economic trends)

- incorporate new measures or indices such as PPI- look at where and why and how client have/have not improved

Reporting:- Develop different reports for board and staff. For staff – in local language, and in a way

which they will understand that doesn’t rely on complicated statistics/graphs.

Using findings:- learned about the importance of health insurance, but also realised that we need more

disaggregated analysis

Discussion questions:

How did you manage clients in time 2 that were non-clients in time 1?


These were excluded from the analysis in the second round. Perhaps with a bigger sample size we can have a separate category

Q: Initial survey tracked change over a 2-year period, and you had a 30% attrition rate. If you had a 5-year study, what attrition rate would you expect and how big would your sample size need to be?A: Actually numerous studies have shown that the 30% mark is consistent no matter how long the study length.

Calum Scott (Opportunity International):Our motivation for collecting client data (and that we wanted our partners to collect) – we weren’t happy with our strategy, the data we were collecting and how we were using it. So we started by asking WHY we wanted this information.

- To know how we were performing: for internal reporting and external reporting- To improve performance: to compare performance across partners (and invest more in

them – ie “backing the winners”), and indeed improve the performance of all our partners.

Using data to improve performance For us the process is to define a set of indicators, collect data, manage and analyse the data, use the data, and create better products, services and delivery. But for me, the missing link is between data use and decision-making. Why is this? Because too often the indicators are chosen without considering their practical use – or the choices are driven by external factors/stakeholders.

Social data doesn’t tend to be valued, because it’s unclear, incomplete, unreliable or out of date, etc. Whereas with financial performance information, the data is relevant, complete, available in a timely way, reliable and easily understood. So is it no wonder that this is the data used to improve performance? To bring these two in line, social data collection needs to be “business as usual”.

Our processWe started by breaking down our mission into a long set of indicators, and then What a good list of indicators looks like for a network is different than for a partner MFI – so we recognised that there would be a need for flexibility. We sought feedback in terms of our shortlist of indicators on their relevance/practicality – and within some of those the methodology for collecting the data actually is different between countries. (Eg on vulnerability – it can deal with medicine, or the need to strip household assets to cope with crisis, or food security).

Nitin Madan (EDA Rural Systems)Working with Opportunity International, we came across a lot of challenges in terms of measuring and reporting on outcomes data. We defined an end-to-end approach where we started working with MFIs to improve each link in the data management chain. This includes data collection, data management, data analysis and reporting, and board/management monitoring. At each point in this process, staff training was a critical component. And each and every department needs to be involved in this – data collection involves operations, HR, governance, MIS and internal audit. Each department needs to understand their role and stake in the process. The first step is to have a meeting with the board. Trying to have more than 30 minutes on the board agenda is really difficult – but we persisted and in the end got 2-3 hours. In the end, the boards started to understand that SPM is about more than vaccinating pigs and cows.

In terms of data collection – we found:


- A lot of times, the data being currently collected was rationalised. We didn’t want SPM to add to the cost/time of data collection – so we queried whether they were actually using the data they were collecting currently. often times the answer was no.

- Training was important: it is very important to embed these processes in the MFI itself- Forms often need to be modified as do operational manuals

In terms of data management:- Operations and MIS play a critical role, as does internal audit.- Data auditing: we created basic standards of use for indicators, and created internal

standards and policies.- Data storage: the MIS must mirror the form. Getting changes to MFI MIS systems was a

huge challenge. The other alternative is to use excel – but of course it is limited, fragile, and hard to integrate (so really it’s only a short-term solution)

Data analysis and reporting:- It’s important to sit down with the board to sit down and ask what types of questions

you want answered, and how often they want it to be reported. Not every indicator gets collected with every loan cycle. Eg poverty information – why collect PPI data every year? Ask: who is getting this info, when and at what level?

Challenges/lessons:- Bringing the board on board- Appointing a coordinator. One SPM champion said “no one listens to me because I am

not senior enough.- Pushback: collecting data is more work for staff. What is the incentive? We found that

staff don’t need to know about the theory of SPM. We took the mission and broke it down by staff roles in making good on that mission and collecting the right information to find out what’s happening with clients.

- Staff exit: Make sure SPM is embedded in the organisation- How do we convert data collected into useable information?

Questions

Q: How does Opportunity define and measure spiritual transformation?A: It’s clearly tricky to measure, and it’s actually the most optional in terms of partners reporting. The survey question most often used is “do you feel closer to God?” or “do you feel a greater sense of spirituality since joining the programme?” And we see these indicators really as the starting point for more in-depth research.

Q: There is a real difference between routine data collection integrated into loan application which (perhaps) involves no more training – versus periodic surveys etc. Which is better? Can they be combined?Routine data collection facilitates routine decision-making. If you keep the list of indicators short enough, it’s easy enough to locate it in your operational processes. Easy daily data will also point you to more in-depth analysis/surveys – they give you pointers to where you might focus that less-frequent research.

Breakout group report-back

How did you select indicators (poverty and risk) for your organisation – what criteria did you use to use to identify indicators and what process did you use to finalise them?

- Relevance: needed to be linked to theory of change, need national benchmarks, existing indicators, and easy to collect, verify and analyse


- Pilot testing: short listing, triangulation to make sure we could rely on results, analysing relevance of those indicators, ease of integration into operational processes

How might an emphasis on using data influence your organisation’s chosen indicators?- We might look for indicators that we can segment by poverty status so we can adjust

product offering in response to detailed information about who is improving and who is not

How have you built systems for measuring, reporting and use of social data in your financial institutions?

- Come up with a consensus of indicators based on common social objectives – what does the institution want to do/see and what are the networks/funders asking for us and how can we mesh them together?

- Think about daily decisions that each department is making – and use these decision points to build a data map? Different columns – where does the data come from? How is the data being generated (survey/sample)? Where is this data being stored (MIS/warehouse), Who will use the data (board, HR, management)?

- Important to train board and management to use the data – can’t assume they know what to do with it.

- For some of the indicators – we found we needed to add indicators to the loan application form. For others, we have the data but we can’t see it clearly because it’s not being encoded in the other way (eg. looking for business growth data and not seeing that you have routine cashflow analysis data elsewhere)

What have been the challenges in setting up this system?- Buy-in from different departments – not just the champion, but everyone.- Quality/cost of data- Revision of MIS – excel isn’t useful in the long-term. MIS modification take a long time –

more than you expect.- Training: at all levels.- Even when information is reported to board – it doesn’t mean that they will comment on

it or be interested in it. Board education is important, and they need to critically examine trends against the bigger picture to put them into perspective.

SPTF Annual Meeting 2015 day two notes... · Web viewDay two meeting notes. Opening remarks....

Documents

Transcript of SPTF Annual Meeting 2015 day two notes... · Web viewDay two meeting notes. Opening remarks....