SEDA IMPACT EVALUATIONWESTERN CAPE(SOUTH AFRICA)

Varsha Harinath (the dti)Francisco Campos (World Bank)

Finance and Private Sector Development IE Workshop

Dakar, SenegalFebruary 2010

Content

Background

Objective of the impact study

Impact Evaluation methodology

Process and timelines

Challenges and lessons

Next steps

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

South Africa background

Unemployment is high: 19% for males 25% for females

South Africa has identified the development of small, micro and medium enterprises (SMMEs) as means to mitigate the unemployment crisis in the country, as well as to ensure a more equitable distribution of income (Department of Trade and Industry, 1995).

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

SEDASmall Enterprise Development Services

The Small Enterprise Development Agency (seda) offers non-financial business development services to small businesses in South Africa.

The agency, overseen by the Department of Trade and Industry (the dti), was created in 2004 and operates in all 9 provinces of South Africa.

Business assessments, training as well as assistance with business plans, promotional material, technology systems, registering process are among a range of non-financial services that seda offers to clients.

Eighty percent of its support to enterprises is focused on businesses with less than 20 employees.

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Objective of the impact evaluation

The impact evaluation seeks to test if seda services in the Western Cape Province lead to improved business knowledge and practices by SMMEs

and hence have a positive impact on growth, competitiveness, sustainability and formality.

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Key outcomesImpact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Outcomes Indicators

1. Growth 1a. Increase in turnover1b. Increase in number of employees

2. Competitiveness 2a. Increase in productivity2b. Increase in net profits2c. Improvement in operational margins2d. Increase in number of customers

3. Sustainability 3a. Survival rate

4. Formality 4a. Registered4b. Certification of tax payment

Methodology

To measure the impact of seda services, we would like to observe the performance of the same enterprise with seda’s intervention and without seda’s intervention over the same period in time……….

Both states of the world do not exist at the same time!

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Methodology

We “construct” these states of the world by creating an “experiment”, whereby we randomly assign a representative group of small business to either:

A treatment group - those that receive seda’s services. A control group - those that do not receive seda’s services.

The treatment and control group is obtained from a representative sample of formal and informal small businesses in the Western Cape Province of South Africa.

To date, there is no extensive list of formal and informal small businesses in the Western Cape. Hence, for this study a multi-stage sampling technique was used , including the sampling of households and the identification of small enterprises within those households.

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

MethodolodyImpact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Populations of SMMEs

SMME Sample(906 businesses)

Control group(453 businesses)

Treatment group(453 businesses)

Average outcome for treatment group

Average outcome for control group

Impact of seda’s intervention

_ __

Process

1. Sample identification - ~9,500 sampled households surveyed to identify SMMEs owned by household members

2. Baseline survey - ~900 SMME owners surveyed on business characteristics, financial accounts, support from government, registration status

3. Intervention - seda contacts the businesses in the treatment group to complete an office visit, assess their needs and provide appropriate services.

4. Follow-up survey - information collected on outcomes for treatment and control group 12 months after the intervention

5. Analysis - impacts measured with difference in differences

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

TimelinesImpact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

2008-2009 2010 2011

Set Objectives Roll out of intervention Follow-Up survey

Design Methodology Baseline survey reports Impact evaluation final report

Budget and Implementation Plan

Design Questionnaire

Sampling

Baseline survey

Roll out of intervention

Baseline reports

Baseline data can be useful for decision-making by assessing the status of entrepreneurship, as well as differences between sub-groups and their corresponding needs

Growth Series: Profile of businesses Profile of entrepreneurs Awareness of SEDA Informality Access to finance Performance (financials) Labor Markets Challenges to growth

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Baseline dataImpact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Variable Treatment Control Difference

Male business owners (%) 53.0% 53.0% 0pp

Non-white business owners (%) 80.6% 79.7% 0.9pp

High-school business owners (%) 50.0% 52.5% -2.5pp

Average age business owner 40.9 40.2 0.7

Average number of employees 5.2 5.1 0.1

Wholesale & retail (%) 54.1% 53.4% 0.7pp

Unregistered (%) 72.9% 72.1% 0.8pp

Number of observations 453 453 0

Challenges and lessonsImpact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

Challenges Lessons

First rigorous IE study for the dti:

buy-in for a large study is difficultPolitical head lobbying for IE fast tracks the process of buy-in

IE can at first glance look expensive (especially if multi-stage sampling is done)

Multi-stage sampling improves external validity and provides rich data at the household and enterprise level

Pilot encouragement design, inviting treatment group to an event by seda at cinemas in the Western Cape yielded low turn-out rates and even lower take up rates of seda services

Significant brainstorming with programme managers & marketing teams is crucial and flexibility when implementing an encouragement may be necessary. Encouragement can be a challenge, especially when invitees are dispersed geographically. Randomised outreach intervention techniques are also usually more successful

Next steps

Office visit intervention for the Western Cape to be completed by March 2010.

Following the interest of this study in the Western Cape, the dti was also requested by seda to do an impact evaluation for the agency on a national scale. The baseline data for national seda study is piggybacking of a national small business study that is presently being conducted for South Africa.

Follow–up survey for Western Cape and for the National study in 2011.

Impact Evaluation South Africa

WB FPD Workshop, Dakar

February 2010

SEDAImpact Evaluation

Finance and Private Sector Development IE WorkshopDakar, SenegalFebruary 2010