QAT Detailed Overview of QAT Process.ppt
description
Transcript of QAT Detailed Overview of QAT Process.ppt
Quality Audit Toolfor Managing Social Performance
MFC Quality Audit Tool 2
Acronyms Used
• QAT – Quality Audit Tool• ECA – region of Eastern Europe and Central Asia• MFI – microfinance institution (MF Banks, non-
banking financial institutions, non-governmental organizations)
• SP – Social Performance• SPM – Social Performance Management• SR – social responsibility
MFC Quality Audit Tool 3
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 4
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 5
Rationale: ECA Region Context
• Since beginning in 1997 focus and pressure on MFIs’ self-sustainability
• Withdrawal of donors to less developed countries
• New comers – commercial investors (i.e. Deutche Bank, Raiffaisen Bank)
• Post – soviet negative connotation of ‘social’ = for free
• Commercialization of MFIs
• Mission drift observed
• Still huge interest in impact, but no resources to measure
• Difficulties in promoting Social Performance
• No commitment to social goals at the level of operations
• Interest in easy and effective tool bringing quick measurable results
Context Market trends Social Performance reality
MFC Quality Audit Tool 6
QAT Purpose
1. Assessing status and effectiveness of systems in managing SP
2. Helping MFI realize its strengths and weaknesses in SPM
3. Helping MFI prioritize actions for improvement of their SPM
4. Ensuring institution-wide understanding and buy-in
1. To ensure MFI is on track: ‘how effective are your systems in aiming at achieving your mission?’
2. To optimize resource investment: ‘build on what you have and invest in what you really need’
3. To prevent MFI from doing everything at once and burn out shortly – ‘let’s improve step by step and look for quick wins’
4. To create enabling environment for change; to prevent SPM be one person ‘project’
QAT aims at… … in order to:
MFC Quality Audit Tool 7
QAT Target Markets
• MFIs concerned about their Social Performance and quality of their management systems for SP
• MFIs planning social rating exercise and willing to prepare themselves
• Donors / Investors / Partners willing to support their partnering institutions in improving their SPM
MFC Quality Audit Tool 8
SP Dimensions Assessed by QATDim Sub-dimension Content
1.1. Setting SP objectives • Mission translation• Social goals and objectives (outreach, service, change)• SR to clients, staff, communities, environment• gender aware policies
1.2. Strategy for achieving SP objectives
• Systems for developing coherent strategy
2.1. Systems for monitoring and understanding performance
• Operations and marketing: mechanism for collecting clients feedback
2.2. Information system quality and appropriateness
• Marketing, HR, Research, MIS, internal audit: data reliability
2.3. Information analysis and communication
• Marketing, operations: data analysis and reporting
3.1. Information use • Management practice: data use, performance monitoring
3.2. Decision making • Decision making at strategic and operational level
3.3. Organizational culture • Governance, leadership• Internal communication
3.4. Alignment of organizational systems
• HR: Staff Incentive System; recruitment and training• Change management capabilities
Man
ag
em
en
t syste
ms
SP
in
form
ati
on
syste
m
Inte
nt
an
d
desig
n
MFC Quality Audit Tool 9
SPTF Dimensions in QATSPM-mission clarity How well is the mission understood by all staff?
Is mission translated into goals and SMART objectives? How well it is implemented into practice?
SPM-alignment of systems
How does internal systems support achievement of mission?
SPM-decision making Are decision (both strategic and operational) made based on SP info? What is the reliability of SP info used?
Gender approach Does MFI have gender aware policies?How effective are they implemented?
Non-financial services Covered indirectly through overall assessment of institution strategy including range of services and its ability to answer clients’ needs
Responsibility to clients
Does MFI have client SR policies?How effectively are they implemented?
Responsibility to community
Does MFI have community SR policies?How effectively are they implemented?
Responsibility to staff Does MFI have SR towards staff policies?How effectively are they implemented?
Responsibility to environment
Does MFI have environmental SR policies?How effectively are they implemented?
Social Goal--Outreach This is referred through process questions on mission clarity, strategy, objectives:
Does MFI define their target clients properly?How effective it is in reaching the target clients?
Social Goal--Services This is referred through process questions on mission clarity, strategy, objectives:
Does MFI know what are the target clients needs?How effectively are they met?
MFC Quality Audit Tool 10
QAT Uses for MFI Manager
• To check if an MFI is at right track towards their mission achievement
• To identify strengths supporting their SPM• To identify weakness: gaps and areas for
improving their SPM• To use prioritized initiatives for developing action
plan for improvement, allowing step by step SPM institutionalization
• To buy in staff at all levels; to build understanding SPM is overarching theme rather than project or tool
• To monitor progress in SPM institutionalization (in case of repeated audit)
• To prepare for social rating
MFC Quality Audit Tool 11
QAT Uses for MFI Manager - Examples
Horizonti QAT lessons learnt:• „This was very important for
we did the audit before implementing anything: conducting the audit we knew the gaps – where we need to pay more attention.
• One of the outcomes – we do know what we are doing, we are targeting the right people and doing a good job, but we need to work on systems to support this.
• We saw that the MIS cannot support performance, so we will have a new one improved.
• We collect a lot of information which can be used for SP indicators.”
ED, Horizonti, Macedonia
Zene za Zene QAT next steps:• Define more specific objectives
for better understanding what it means for the terms of SP?
• To experiment with MIS reports content
• To improve internal communication
• To measure drop-outs• To use client satisfaction form
across the institution
SPM Champion, Zene za Zene, B&H
MFC Quality Audit Tool 12
QAT Uses for Donor / Investor / Partner
• To help MFI improve in SPM through diagnosing strengths and weaknesses and developing action plan
• To optimize its support through financing or helping implement the initiatives from action plan
• To better understand the institution, its mission orientation and rational behind its translation to action
• To monitor progress in SPM institutionalization (in case of repeated audit)
MFC Quality Audit Tool 13
QAT Desired Impact: MFI Level
• Outcome - setting the stage for change processClearer operational understanding of SPMClear understanding by staff of what needs to be done to improve SPM Internal champion and sponsor identified – infrastructure for change establishedBetter communication of social orientation and required support to stakeholdersQuick wins implemented
• Short term impactProcess of SPM improvement kick-off Consistent effort of MFI to improve SPM in a manageable steps There is on-going buy-in of staff, leadership of champion and support of senior management and board for implemented changes
• Medium term impactInternal systems are improved to be better aligned with double bottom line goals
• Long term impact – expectedInstitution gets all aligned to double bottom line and results are improved
MFC Quality Audit Tool 14
QAT Impact: Examples From MFIs
Gap: description of the target group was vague and mixed with eligibility criteria for the loan
Quick win: through a series of half day workshops defines what is meant by saying „we want to reach economically and socially challenged families” and
comes up with the indicators based on housing materials, asset base, etc.
Gap: At fast growth, new staff don’t understand and commit to mission
Quick win: re-design of training program for newly hired staff to ensure mission perspective
incorporation
Gap: collects lot of information, which is not used later on
Quick win: review the new set of MIX SP indicators to see which information they can start using
immediately
Micro Development Fund, Serbia
CARD, the Philippines
Moznosti, Macedonia
More examples of MFIs’ quick wins in MFC Newsletter Issue 1/2008
MFC Quality Audit Tool 15
QAT Desired Impact
• Investor / Donor / Supporter level:Increased number of partners refocus from assessing SP to improving SPM more supportive and longer term cooperation on SPM improvementMore informed partners make better decision in selecting more likeminded partnersMore efficient allocation of resources to likeminded MFIs and activities that can bring greatest value in improving SPM at MFI
• Industry levelMore MFIs aware of their SPM status and effectivenessMore MFIs improve in their SPMLess mission driftBetter public image of Microfinance
MFC Quality Audit Tool 16
QAT Benefits for Various Stakeholders
• MFI Managers:Clear current picture of SPM systems qualityGood basis for effective implementation planEfforts oriented on mission acknowledgedStaff understands better and is bought-in through discussion about SPMAllows improving before future social rating
• MFI staffBetter understanding of how their tasks fit into social goals and strategy
• Donors / investors / supportersBetter understanding of MFI’s commitment to mission and its translation into practiceAssessment of MFI’s SPM qualityEffective identification of MFI’s needs in terms of supporting their SPM institutionalization
MFC Quality Audit Tool 17
QAT Uses – Extracts from 5th SPM Working Group Meeting Minutes
In March 2008, the representatives of 10 organizations – members of MFC facilitated SPM Working Group, met in Warsaw to discuss lessons learnt in SPM institutionalization. Among other subjects they discussed QAT uses:
• „It can be used before the process of SPM implementation – it is nice to have gaps
• Could be used for process evaluation – a good diagnostic tool• Could be useful for taking on board those who were not directly
involved – e.g. IT people• Gives an understanding where the gaps are and what actions need to
be taken• The format is very useful – the SPM team might be biased – but
through interviews one gets more objective information• Could be used multiple times – especially before the strategic
planning• Provides awareness of different opinions and perceptions• Stimulates additional discussions at the institutions on SPM• Gives better understanding of HR needs – where there are gaps in
assumed and actual way the processes work• Remind and refresh the mission and social objectives• Reveals information flow bottlenecks
MFC Quality Audit Tool 18
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 19
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 20
4 Steps in QAT
• ED• Senior management (Finance, Operations, HR, Marketing, )
• Board• Senior management• Internal auditor• MIS staff• Research staff• Regional level• Branch level• LO• Clients
• All
0,5 day 2 - 3 days1-2 days 0,5 day 1 day
Preparation
Gap analysis
Follow up interviews
Audit panel
Final report
Analysis &draft report
1 day
Sta
ff p
art
icip
ati
ng
Total time: from 6 up to 8 days
MFC Quality Audit Tool 21
External & Internal Assessor
QAT is implemented by the team of two people: external and internal assessor:
Internal Assessor External Assessor
Neutral job position (best from HR or Marketing, with managerial perspective understanding)
Experienced in Microfinance
Internal assessor can help to ensure commitment from management & staff
Skilled in qualitative research skills ensures appropriate probing
Valuable knowledge of the organizational systems and information
Impartiality and independence ensures review of organizational systems and information to check for reliability
SPM understanding is required Skilled in analysis and synthesis of complex information
MFC Quality Audit Tool 22
QAT Implementation Steps (1)
1. Gap analysis: presents detailed questions which are used to assess SPM; sets up a base for more in-depth follow-upSource: ED and senior management
2. In-depth follow-up: gathers more detailed information to verify and better understand gap analysis; checks status and effectiveness of systemsSource: Interviews with senior management, staff &
clients; review of documents and internal reports; and information flows.
MFC Quality Audit Tool 23
QAT Implementation Steps (2)
3. Analysis and draft report: combine information and draft report highlighting strengths and weaknesses of issues in gap analysis; it captures perspectives of different stakeholders to be discussed during audit panel.
Analysis include:Qualitative data analysis using triangulation of different information sources (interviews with different staff, focus groups with clients, review of written documents)Tally sheet is used to identify common themes and divergences around strengths and weaknesses of MFI systems, notes from the interviews are reviewed for evidence and rationale behind emerging themes
Draft report is written addressing strengths, weaknesses, evidence and issues to further discuss/explore that serve as basis for discussion with representatives of different stakeholders during an audit panel.
MFC Quality Audit Tool 24
QAT Implementation Steps (3)
4. Audit panel: Presentation of findings to key organizational stakeholders, including staff from all levels of organization and clients, where appropriate.
Audit panel allows for further verification of the audit results and finalize the report The initiatives for addressing gaps are identified and prioritized in order to develop action plan for SPM improvement
5. Final report: finalized based on audit panel discussion
Includes prioritized activities to be undertaken by MFI in order to improve SPM by addressing the identified gaps and by building on identified strengths
MFC Quality Audit Tool 25
Source of Information and its Validation
Source of information Validation: what for?
In-depth interviews with senior management
Set a base, learn the strategic directions
In-depth interviews with middle management and functional staff
Confirm status, confirm evidence, cross-check data
Focus Group Discussion with Loan Officers and Clients
Confirm status, confirm evidence, cross-check data
Internal documents and reports: Board meeting minutes, codes of conduct, financial statements, portfolio reports, strategic plans, HR policies, MIS reports, clients satisfaction / exit surveys etc
Look for evidence, cross – check data
Audit panel Verifies findings, explore doubtful issues, includes more evidence
MFC Quality Audit Tool 26
Data Collection Methodology
• Gap analysis questions: basic set of questions defining the framework of SPM assessment
i.e.: do you seek to be socially responsible organization in relation to your target clients?
• Status questions: questions validating the status of SPM components: their presence in the institution
Asked to Loan Officer: How do you communicate interest rate to clients? How do you ensure they understand?
• Effectiveness questionsAsked to clients: How do you learn about interest rate? How do you know what is it?
MFC Quality Audit Tool 27
QAT questions examples
Sourc
e:
QA
T H
andbook
MFC Quality Audit Tool 28
QAT Report Structure (1)
• Report is divided according to dimensions and/or sub-dimensions
• Each dimension / sub-dimension includes:[to be discussed during audit panel:]
List of institutional strengths supported by evidenceList of institutional weaknesses supported by evidencePoints for discussion: when there are different and not coherent opinions and evidence, they can be clarified and agreed during audit panel
[after audit panel:] key elements of the action plan
MFC Quality Audit Tool 29
Dimension 1: Intent & designSub-dimension: Setting social performance objectives
MFIs strengths:• xxxx
– EVIDENCE: xxxxx
MFIs areas for improvement:• xxxx
– EVIDENCE: xxxxx
Discussion points:• Something still unclear• various opinions
Action plan:• What are the priority actions?• What are quick wins?• What are immediate next steps?
QAT Report Structure (2)
MFC Quality Audit Tool 30
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 31
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 32
QAT Attributes - Time
• Preparation (documents review)• Gap analysis• Individual interviews• Focus Group Discussions with Loan
Officers and Clients• Analysis• Audit panel• Final report
• 1 – 2 days• 0,5 day• 1,5 – 2 days• 1 day
• 0,5 -1 day• 0,5 days• 1 day
• Depends on skills and experience of assessor• With external facilitator the process is more intensive and
shorter• Internal Assessor can adjust duration to the internal
dynamics
TOTAL: 6 – 8 days
Process steps Duration
MFC Quality Audit Tool 33
QAT Attributes: Cost
• Staff timeInternal AssessorStaff in interviewsStaff in audit panel
• Travel to branches for FGDs
• Snacks for FGDs
• Includes:6 – 8 days1,5 h per person3 h per person
• Travel costs• Costs of drinks and
snacks
• External facilitator time• Lodging• Travel
• 6 days * daily rate• 6 nights * daily allowance• Cost of plane ticket
MFI
cost
sE
xte
rnal
faci
litato
r
MFC estimated cost for external assessment:6000 USD + staff time cost
MFC Quality Audit Tool 34
QAT Attributes: Breadth of Functionality
For whom?
Implementation usage
Results usage
Staff • understanding how their tasks fit into strategy and mission• securing their buy-in
• acknowledgement of up-to-date efforts
Management and Board
• ensuring all staff is on the same page
• assessment of effectiveness in SPM• effective identification of areas and actions to improve SPM
Supporters / Investors
• understanding the organization better (it’s social mission and its translation to practice)
• managerial systems quality assessment• effective identification of areas and actions to support SPM institutionalisation
MFC Quality Audit Tool 35
QAT Attributes: Infrastructure and Resource Requirement
• Resources required:Staff timeFunding for external facilitator budget
• Infrastructure required:Computer for assessors, recorderSeparate room for conducting interviews (preventing noise, securing proper concentration)Convenient room for 2 FGDs (with Clients and Loan Officers)Transportation to branch (if needed)Conference room for holding audit panel (including: LCD projector, flipcharts, markers)
MFC Quality Audit Tool 36
QAT Attributes: Ease of Use
• In case of external assessment:MFI needs only to set up logistics and select internal assessorAdditional benefit: MFI is trained on how to conduct the assessment to do it independently next timeChallenge: securing the funding
• In case of self-assessment:Pre-assessment training is requiredTiming adjusted to the internal dynamics of MFIChallenge: the selection of internal assessor must be more careful
MFC Quality Audit Tool 37
QAT Attributes: Customer Satisfaction
• Assessing both strengths and weaknesses makes the audience appreciate the objective assessment: they are motivated to improve as their up to date efforts to achieve the mission are acknowledged
• Involvement of many stakeholders into process and allowing them commenting during audit panel make them owe the findings in opposite to external findings (external assessor becomes perceived rather as facilitator)
• Developed action plan reflects the real needs and does not irrationally overburden staff
• Clear and easy to follow structure of the report makes it a handy document for further work
MFC Quality Audit Tool 38
QAT Attributes: Customer Satisfaction in Citations
• I am pleasantly surprised how everything is now clear – the role and the goals of SPM
• External assistance in the audit is very helpful. I wonder whether to us people would say what the external auditor found out. I have a feeling they tell us what they think we want to hear.
• It was good that loan officers and branch staff were included in this audit. They gave a needed perspective for the understanding of the issues they face in their work. The fact that all staff was included means we will have full support for the changes
• I am the SPM team member. During this week it became clear that we did a lot of stuff, discussed and decided on many points. I now see what will be the result of the entire SPM effort.
• Social audit made me realize that we are already working on many aspects of SP in our organization, but have not given it the right emphasis
I am now better aware of the importance of the right loan for the client. I will pay more attention to how I do my work, how the loan is used and whether the client is advancing. This is giving me an encouragement for my future work
Partner, Bosnia and Herzegovina
In our case, this tool revealed many weak points in the systems and management processes and resulted in creating an institutional strategy with a holistic approach
AgroInvest, Montenegro, Serbia
MFC Quality Audit Tool 39
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 40
Agenda
1. QAT Overview2. QAT Process3. QAT Attributes4. QAT vs. Other Tools5. QAT Implementation
MFC Quality Audit Tool 41
QAT vs. Other Tools (1)
• Gathers perspectives of various stakeholders• Discussion on findings in held
• Assesses quantitative outputs: social accounts• Focused on outputs, not internal systems • QAT allows assessing the effectiveness of systems
Similarities with QAT
Key differences
Compliance audits: social accounting by Social Audit Network (UK)
• Assesses status and effectiveness
• Uses a scorecard to assess SPM, which is less flexible – comparison to desirable characteristics of MFI• Is not aligned with SP framework• Allows for comparison between organizations to greater extent• Does not encourage immediate management reaction and commitment to improvement
Social Performance Audit Tool (SPA) by USAID
MFC Quality Audit Tool 42
QAT vs. Other Tools (2)
• Methodology of data collection is the same (interviews, documents)
• The greater focus is put to outputs: outreach, client service, information transparency, customer protection, • Less focused on internal management systems
Similarities with QAT
Key differences
SOCIAL Tool by ACCION
• Goal: to determine the abilities of MFI to achieve social objectives
• More quantitative questionnaire• Covers less areas: outreach to poor and excluded, adaptation of products and services to target clients, improving social and political capital, Corporate SR
Social Performance Indicators by CERISE
MFC Quality Audit Tool 43
MFIs Implementing QAT Worldwide
• ECA regionAgroInvest and MDF, SerbiaAregak, ArmeniaFinDev, AzerCredit and Viator, AzerbaijanHorizonti and Moznosti, MacedoniaPartner and Zene za Zene, Bosnia and Herzegovina
• The PhilippinesASHI, CARD
• MENA RegionTamweelcom and DEF, JordanTill end of 2008: ARDI and AMOS, Morocco
MFC Quality Audit Tool 44
Contact Details
For more information on QAT, please contact:Ewa Bankowska, Project CoordinatorE-mail: [email protected]: +48 22 622 34 65Skype: bankowska