EFFECTIVENESS OF TRADITIONALEFFECTIVENESS OF TRADITIONAL BIRTH ATTENDANTSBIRTH ATTENDANTS

A META-ANALYSISA META-ANALYSIS

Presented by: Lynn M. Sibley, CNM, PhD

American College of Nurse-MidwivesEmory University

Theresa A. Sipe, CNM, PhDGeorgia State University

With support from:

World Bank, Safe Motherhood Special Grants Program USAID Bureau for Global Programs, Office of Population, Bureau for Africa, Office of Sustainable Development

GOAL AND OBJECTIVESGOAL AND OBJECTIVES

This meta-analysis1 of traditional birth attendant (TBA) training effectiveness was undertaken by the American College of Nurse-Midwives with support from the World Bank, the United States Agency for International Development and Emory University Department of Anthropology. It summarizes the available published and unpublished studies and describes the effect of training on TBAs, their clients and on pregnancy outcomes. The goal is to provide information that may be used for more informed policy decisions about future TBA training, and about evaluation research needs. Objectives are to: Examine differences in knowledge, attitudes, and

practices of trained and untrained TBAs; Examine whether, and to what extent, trained and

untrained TBAs differentially impact on maternal and neonatal mortality;

Explore the influence of sample characteristics on observed effects; and

Describe the quality of and gaps in the evaluation literature.

1 Meta-analysis is the “analysis of analyses” (Glass, 1978, p. 352). The meta-analytic approach to research synthesis combines primary research studies on a particular topic and uses many techniques to integrate findings from all of the studies (Glass, McGaw & Smith, 1981). Meta-analysis summarizes large numbers of studies efficiently and analyzes studies with statistical methods rather than impressionistic literary review methods (Guskin, 1984).

2

METHODSMETHODSSamplingSampling

The sample included all studies that met the following criteria: Treatment was TBA training; Data were derived from situations of TBA

training; Treatment group data were derived from trained

TBAs and/or women and neonates whose health care was provided by trained TBAs;

Comparison group data were derived from untrained TBAs (or TBAs before training) and/or women and neonates whose health care was provided by untrained TBAs or persons2;

Dependent measures were related to TBA knowledge, attitudes, behavior and/or maternal and neonatal health outcomes;

Documents were in English, completed or published between January 1970 and June 1999;

Research design was either experimental or quasi-experimental; and

Data were sufficient to calculate an effect size.

2 In some instances, the comparison group was other health care providers. These were not included in the overall calculation of effect size.

3

Sampling (cont.)Sampling (cont.)

A broad search of potentially eligible studies initially yielded citations for 1,190 documents, which included books chapters in books, technical reports, conference papers, theses, as well as published journal articles. These were independently screened against the criteria in stages by three investigators. Inter-rater reliability was established as 87% using percent agreement among two-out-of-three investigators. All differences were resolved. Sixty-three (n=63) studies met the inclusion criteria.

Data collectionData collection

Two doctoral students, using pre-tested coding forms and supervised by one of the investigators, coded variables related to study context, methodology, characteristics of the intervention TBA training, sub-sample identification, demographic data, pre-test comparisons and outcome measures. Inter-rater reliability was established as 89% using percent agreement (Glass et al, 1981). All differences were resolved. The data were double entered and validated using Epi Info Version 6.04b (1997).

4

Data collectionData collection (cont.)(cont.)

One investigator coded variables related to outcome and effect size, which was calculated on proportions using the arcsine transformation3 (Lipsey, 1990). A second investigator verified a large sample of variables. A positive effect size indicates that the treatment

group scored “better” on the outcome measure than the comparison group

A negative effect size indicates that the treatment group scored “worse” on the outcome measure than the comparison group

A zero effect size indicates equivalence between the two groups.

Effect sizes were computed using Quatro Pro Version 7.0 (1996) and formulas were verified by hand. All data were imported into SPSS Version 9.0 (1998) for the analysis.

Analysis Analysis 3 The following formula was used to calculate effect size: let pt be the success proportion for the treatment group; let pc be the success proportion for the control group; let t be the arcsine transformation 2arcsine ( pt) and correspondingly, c =

2arcsine ( pc). The effect size index for differences between pt and pc is expressed as follows: ESp = t - p. A large effect size includes values > 0.70 (> 25% difference), medium effect size includes values 3.0 – 7.0 (20%-25 % difference), and small effect size includes 0.1-<3.0 (5%-10% difference) (Cohen, 1988).

5

The following effect sizes were calculated: Effect size mean for each study Effect sizes for each category of outcome

measured (study-weighted) Effect sizes for each attribute (study-weighted) Grand total effect size mean (study weighted)

Homogeneity tests on the distributions of effect sizes were conducted to check for variability. Where variability existed, the source was explored through sensitivity analysis.

6

RESULTSRESULTS

This poster presents findings for the following aspects of the sample of studies: Type of document, country of origin, and year of

publication; Research design and method used to evaluate TBA

training; Intervention, TBA training, including curriculum

content, intensity and productivity of TBA training programs, training modalities, approaches, and follow-up of trained TBAs; and

Effect of TBA training on TBA ‘Knowledge’ and on TBA client or maternal ‘Knowledge’.

The effects of TBA training on TBA and maternal ‘Attitude’, ‘Behavior’, ‘Advice’ (a subset of behavior), and on ‘Pregnancy Outcomes’ are forthcoming.

7

Type of publication

Per

cent

age

100%

80%

60%

40%

20%

0%

Fig. 1. Distribution of studies by document type (n=63).

The majority of the studies were journal articles, followed by dissertations and theses. About two-thirds (64%) were published documents.

8

Year of Publication

1995-1999

1991-1996

1985-1990

1981-1986

1976-1980

1971-1975

Per

cent

age

100%

80%

60%

40%

20%

0%

Fig. 2. Distribution of studies by five-year interval, 1974–1998 (n = 63).

The studies span three decades. Most were completed during the 1980s and 1990s— since the time that emphasis had been placed on TBA training by the WHO post Alma Ata Declaration.

9

Table 1. Distribution of studies by country (n=63).

COUNTRY (n) COUNTRY (n)

Bangladesh 7 Indonesia 2 Belize 1 Kenya 2 Brazil 2 Liberia 1 Burkina Faso 1 Malawi 2 Burma 1 Malaysia 1 Cameroon 1 Nepal 2 Ethiopia 1 Nigeria 3 Gambia 3 Pakistan 4 Ghana 5 Philippines 2 Guatemala 2 Senegal 1 Haiti 2 Thailand 3 India 13 Uganda 1

The distribution of studies by country shows that the region most represented was Asia (57%), followed by Africa (32%) and Latin America-Caribbean (11%).

10

Table 2. Research designs used (n=63).

DESIGN STUDIES (n)

PERCENT (%)

POST COMPARISON Post comparison, 2 groups 28 44% Post comparison, > 2 groups 5 8% Post comparison, 2 groups, multiple observations

1 2%

PRE/POST COMPARISON Pre/post comparison, 2 groups 10 16% Pre/post comparison, 1 group 4 6% Pre/post comparison, gain score 4 6% Pre/post comparison, 2 groups,

multiple observations3 5%

Pre/post comparison, 1 group, multiple observations

1 2%

Baseline/post comparison, 1 group 1 2% Baseline/post comparison, 2 groups 1 2%

OTHER Mixed1 4 6% Baseline comparison, 2 groups 1 2%

1 Mixed designs occurred in some studies having multiple data sets.

Post comparison was used most often (54%), followed by simple pre/post or baseline/post comparison without a control group (33%). The

11

rigorous pre/post comparison with a control group was used infrequently (6%). Table 3. Sampling procedures used (n=63).

SAMPLING STUDIES (n)

PERCENT(%)

Population 13 21%

Random, multistage, stratified or cluster

15 24%

Random, simple 8 13%

Non random, multistage, stratified or cluster

4 6%

Non random, simple 7 11%

Mixed1 10 16%

1 Mixed sampling procedures occurred in some studies having single data sets. An example of a study having a single data set is where the treatment group is sampled randomly and the comparison group is sampled non-randomly or it includes the entire population. Several studies having multiple data sets also used mixed procedures.

Fifty-seven authors reported sampling procedure (90%). Random sampling was used most often (37%), followed by population (21%) and non-random procedures (17%). Mixed sampling was also used, both one procedure for the treatment group and another for the comparison group and also different procedures for different data sets. Authors

12

reported some form of matching between treatment and comparison groups in 10% of studies.

Table 4. Curriculum content reported (n=63).

CONTENT STUDIES (n)

PERCENT (%)

Maternal risks, problems for referral

28 44%

Intrapartum care, practices 27 43%

Antepartum care, checkups, practices

22 35%

Infection prevention, hygiene 22 35%

Infection prevention, cord care 17 27%

Neonatal risks, problems for referral

19 30%

Referral, how to safely transport 18 29%

Family planning, general, methods

19 30%

Postpartum checkups, practices 16 25%

Over half of authors (59%) provided information on curriculum content. Twenty-five percent or more of authors reported maternal risks-problems for referral, antepartum care, intrapartum care practices,

13

infection prevention (i.e., basic hygiene and cord care), neonatal risks-problems for referral, safe referral, family planning, and postpartum care.

Table 4. Curriculum content reported (cont.).

CONTENT STUDIES(n)

PERCENT(%)

Neonatal problems, management

15 24%

Neonatal care, nutrition 15 24%

Neonatal care, checkups, practices

14 22%

Infection prevention, causes 12 19%

Infection prevention, immunization

12 19%

Record keeping 9 14%

Antepartum care, nutrition 8 13%

Child growth monitoring 6 10%

Intrapartum care, harmful practices

6 10%

Between 15%-24% reported neonatal care (i.e., checkups, nutrition, breastfeeding, weaning, immunization), as well as management of neonatal

14

problems (i.e., asphyxia, low birth weight, ARI, diarrhea).

Table 4. Curriculum content reported (cont.).

CONTENT STUDIES (n)

PERCENT(%)

Home visiting 5 8%

Reproductive anatomy and physiology

5 8%

Other primary health activities 3 5%

Postpartum care, nutrition 2 3%

STD/HIV/AIDS 2 3%

Maternal problems, management

2 3%

First aid 2 3%

Relations with other MCH care

providers

2 3%

Abortion 1 2%

Only 3% of authors reported content related to first aid care or management of maternal problems (i.e., breech delivery and hemorrhage).

15

Table 5. Intensity and productivity of training (n=63).

MEASURE

STUDIES(n)

M SD MIN MAX

INTENSITY Duration/wks 23 8.1 7.0 1 32 Contacts/wk 5 1.6 0.4 1 2 Hours/wk 2 6.0 0.0 6 6 Total hours 8 43.6 32.3 5 100 Cohort size 9 25.3 23.3 4 59

PRODUCTIVITY TBAs/yr 10 400 920 4 3000 TBAs overall 35 18,246 77,594 4 450,000

Intensity and productivity of TBA training programs is quite variable. Twenty-three authors (37%) reported duration of training, which ranged from 1 to 32 weeks. At program level, ten authors (16%) reported number of TBAs trained per year, which ranged widely from as few as 4 TBAs to as many as 3,000 TBAs. Thirty-five authors, however, reported

16

the number of TBAs trained overall, which also ranged widely from as few as 4 TBAs to as many as 450,000 TBAs.

Table 6. Modality, approach and follow-up (n=63).

TRAINING CHARACTERISTIC

STUDIES(n)

PERCENT(%)

MODALITY Theoretical 22 35% Practical 20 32%

APPROACH Mostly didactic 2 3% Mostly participatory 2 3% Mixed didactic + participatory 18 29%

FOLLOW-UP Supervision 31 49% Follow-up support 27 43% Continuing education 25 40%

Over one-third of authors provided information about the modality and approach used in TBA training. Modality is best characterized as a

17

combination of theoretical and/or some form of practical training. The approach is best characterized as mixed didactic and participatory.

TBA ‘Knowledge’TBA ‘Knowledge’

Twenty-five studies, 27 data sets and 316 outcomes are included in the analysis of TBA ‘Knowledge’, reflecting 18 of 24 MCH content areas. Large effect sizes (values > 0.70) were found for

TBA ‘Knowledge’ of maternal and neonatal nutrition, need for maternal and neonatal health check-ups, infection prevention measures and management of neonatal problems.

Medium effect sizes (values 0.30 – 0.70) were found for TBA ‘Knowledge’ of reproductive anatomy-physiology, maternal and neonatal risks-problems for referral, family planning, intrapartum care practices, and umbilical cord care.

Small effect sizes (values 0.10 – < 0.30) were found for TBA ‘Knowledge’ of preparation for

18

confinement, postpartum care practices, and management of maternal problems.

The grand weighted mean effect size for TBA ‘knowledge’ across 18 MCH content areas was 0.66 + 0.48. The grand mean effect size was computed from a study-weighted pooled sample of 1,392 subjects in the treatment group, and 1,702 subjects in the comparison group. Absent data are shown as green in the table.

Table 7. Effect size means (study-weighted) for MCH contentby TBA knowledge.

MCH CONTENT AREA STUDIES(n)

OUTCOMES(n)

EFFECT SIZE

MEAN

EFFECT SIZE

STD. DEV.

TXGROUP

(n)

CXGROUP

(n)

19

Infection prevention, causes 2 9 0.90 0.50 143 165

Infection prevention, hygiene 3 3 0.75 0.64 276 226

Infection prevention-cord care

2 5 0.41 0.41 238 118

Infection prevention-immunization

5 11 1.05 0.51 421 636

Antepartum care, checkups 5 6 1.08 0.55 179 470

Antepartum care, practices

Antepartum care, nutrition 2 3 1.29 0.91 116 364

Intrapartum care, prep for confinement

1 2 0.27 -- 26 48

Intrapartum care, practices 1 2 0.60 -- 57 305

Postpartum care, checkups

Postpartum care, practices 1 6 0.13 0.13 57 305

Postpartum care, nutrition

Family planning, general 3 5 0.67 0.25 273 302

Family planning, methods 8 105 0.58 0.33 551 780

Abortion

Newborn care, checkups 2 6 0.84 0.53 174 212

Newborn care, practices

Newborn care, nutrition, breast feeding

3 6 0.71 0.87 168 452

Maternal risks, problems for referral

8 86 0.50 0.46 498 761

Maternal problems/management

2 5 0.26 0.06 128 130

Newborn risks, problems for referral

6 20 0.69 0.63 371 680

Newborn problems, management

3 3 0.91 1.07 185 221

Relations with health care providers

Anatomy and physiology 7 35 0.42 0.41 533 690

Maternal ‘Knowledge’ Maternal ‘Knowledge’

20

Six studies, 6 data sets and 46 unique outcomes were included in the analysis of maternal ‘Knowledge’, reflecting 9 of 24 MCH content areas. No large effect size means (values > 0.70) were found. Medium effect size means (values 0.40 – 0.70)

were found for maternal ‘Knowledge’ of immunization and family planning.

Small effect sizes (values 0.10 – 0.30) were found for ‘Knowledge’ of basic hygiene, need for antepartum checkups, antepartum care practices, specific family planning methods, neonatal risks-problems for referral, management of neonatal problems, and relations with other health care providers.

The grand weighted mean effect size for maternal ‘Knowledge’ across all 9 MCH content areas was 0.32 + 0.17. The grand weighted effect size mean was computed from a study-weighted pooled sample of 2,014 subjects in the treatment group and 1,882 subjects in the comparison group.

Data were absent for most of the content areas, shown as green in the table.

8. Effect size means (study-weighted) for MCH content9. by maternal knowledge.

21

MCH CONTENT AREA STUDIES(n)

OUTCOMES(n)

EFFECT SIZE

MEAN

EFFECT SIZE

STD. DEV.

TXGROUP

(n)

CXGROUP

(n) Infection prevention, causes

Infection prevention, hygiene 1 1 -0.03 -- 228 101

Infection prevention, cord care

Infection prevention, immunization

1 2 0.33 -- 244 120

Antepartum care, checkups 1 6 0.26 -- 190 172

Antepartum care, practices 1 3 0.18 -- 228 101

Antepartum care, nutrition

Intrapartum care, prep for confinement Intrapartum care, practices

Postpartum care, checkups

Postpartum care, practices

Postpartum care, nutrition

Family planning, general 3 5 0.37 0.25 1,122 1,097

Family planning, methods 2 15 0.26 0.01 797 842

Abortion

Neonatal care, checkups

Neonatal care, practices

Neonatal care, nutrition, breast feeding Maternal risks, problems for referral Maternal problems, management Neonatal risks, problems for

referral1 5 0.10 -- 190 172

Neonatal problems, management

1 2 0.24 -- 244 120

Relations with health care providers

1 6 0.26 -- 190 172

Anatomy and physiology

CONCLUSIONSCONCLUSIONS22

These preliminary results suggest that: TBA training improves overall ‘Knowledge’ of

MCH since the overall grand weighted effect size mean for the attribute ‘Knowledge’ is a medium value of 0.59 + 0.46. This was computed from 28 studies, 32 data sets, 362 unique outcomes, and a weighted pooled sample of 3,406 subjects in the treatment and 3,698 subjects in the comparison groups.

Several findings are relevant to the debate that TBA training can or cannot contribute to a reduction in maternal mortality. Training has a medium effect on TBA

‘Knowledge’ of maternal risks-problems for referral, but only a small effect on management of maternal problems.

The effect on maternal ‘Knowledge’ of maternal risks-problems for referral, and on management of maternal problems, is unknown due to a lack of research or reporting in this important content area.

This is consistent with the finding that only 3% of programs describe curriculum content covering first aid or management of maternal problems.

23

The effects are not uniform between TBAs and their clients, nor are they uniform across MCH content areas, and there are many content areas in which there are no data.

We know little about the intervention, TBA training, due to insufficient reporting, making it impossible to assess which elements of programs lead to better outcomes.

The quality of the studies were often lacking in methodological rigor. Nonetheless, the final results, which will include the attribute ‘Knowledge’ and also ‘Attitude’, ‘Behavior’, ‘Advice’, and ‘Pregnancy Outcomes’, may be used to inform policy decisions about future TBA training and evaluation research needs. The findings may be important in the light of the recent programmatic shift to skilled attendance at delivery (Fortney and Smith, 1997; Starrs, 1997; WHO, 1992)-- a distant reality for some.

Two by-products of this meta-analysis are a taxonomy of studies on TBA training spanning three decades and an extensive electronic bibliographic database of the TBA literature. For information about these products, please contact:

lynn_sibley@learnlink.emory.edu or theresa@gsu.edu.

24

ACKNOWLEDGEMENTSACKNOWLEDGEMENTSWe thank individuals who have encouraged, supported or directly worked on this meta-analysis. Deborah Armbruster, Catonsville, MD George Armelagos, Emory University, Atlanta, GA Caroline Blair, AED/PRB, Washington, DC Wilda Campbell, PRIME/Intrah Regional Office, India Julia Cleaver, Ipas, Chapel Hill, NC Karen Cristiani, WHO, Geneva, Switzerland Erin Finley, Boston University Hospital, Boston, MA Judith Fortney, FHI, Research Triangle Park, NC Zahid Huque, UNFPA, Sudan Vinay Kamat, Emory University, Atlanta, GA Lynn Knauff, Chapel Hill, NC Phyllis Long, Lakewood, NJ Steven Morreale, Emory University, Atlanta, GA Charlotte Quimby, Meriden NH Jinny Sewell, USAID, Washington, DC Katherine Sieck, Emory University, Atlanta, GA Ann Tinker, Save-the-Children, Washington, DC

We especially thank the World Bank (Safe Motherhood Special Grants Program), USAID Office of Population, Bureau for Global Programs (PRIME I Project, Contract No. HRN-A-00-99-00022-00) and USAID Bureau for Africa, Office of Sustainable Development (SARA Project, Contract No. AOT-C-k00-9900237-00) for their support. The views expressed in the document are those of the authors and do not necessarily reflect the views of the World Bank or US Agency for International Development.

25

REFERENCES CITED

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, New Jersey: Lawrence Erlbaum.

Dean, A.G., Dean, J.A., Coulombier, D., Burton, A.H., Brendel, K.A., Smith, D.C., Dicker, R.C., Sullivan, K.M., and Fagan, R.F. (1997). Epi Info (Version 6.04b). A word processing, database, and statistics program for epidemiology on microcomputers [Computer software]. Stone Mountain, Georgia: USD.

EndNotes (Version 3.0.1) [Computer software]. (1998). Niles Software Corporation.

Fortney, J.A. and Smith, J.B. (1997). Training of traditional birth attendants: issues and controversies. Research Triangle Park,N.C: Family Health International and United Nations Children’s Fund

Glass, G.V. (1978). Integrating findings: The meta-analysis of research. Review of Research in Education, 5, 1978.

Glass, G.V., McGaw, B. and Smith, M.L. (1981). Meta-analysis in social research. Newbury Park, California: Sage Publications.

26

Guskin, S.L. (1984). Problems and promises of meta-analysis in special education. The Journal of Special Education, 18 , 73-80.

Lipsey, M.W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, California: Sage Publications.

Quatro Pro for Windows (Version 7.0) [computer software]. (1996). Ottawa, Ontario: Corel Corporation.

SPSS (Version 6.4.1) [computer software] (1996). Chicago, Illinois: SPSS.

Starrs, A. (1997), the Inter-agency Group on Safe Motherhood. The safe motherhood action agenda: priorities for the next decade. Report on the Safe Motherhood Technical Consultation, 18-23 October 1997, Colombo, Sri Lanka. New York, NY: Family Care International.

World Health Organization. (1992). Traditional birth attendants: A joint WHO/UNICEF/UNFPA statement. Geneva: WHO.

27