1

Evaluation of quality of HIV clinical care in

wollayta soddo Hospital

SNNP Regional state, Ethiopia

Prepared by: Marshet We/yohanns

Jimma may, 2009

2

Out lines

• Introduction• Statement of the problem• Purpose of evaluation• Objective of the program• Program activities• Stage of development• Social context of the program• Logic model of the program• Stakeholder identification and analysis• Objective of evaluation• Evaluation question• Study Area and period• Study unit and Sampling unit

3

Out line cont..

• Sample size and sampling procedures• Focus of evaluation• Dimension of evaluation• Approach of evaluation• Designs of evaluation• Theoretical modal of evaluation• Inclusion and Exclusion criteria• Data collection procedures• Data management and Analysis• Limitation of the study• Report writing & dissemination plane• Result and Discussion• Conclusion& Recommendation• Reference

4

Introduction

program evaluation is applied research used as part of the managerial process.

• programs are evaluated basically because administrative decision have to be made and it is important to know (or to show) that our program are good programs.

5

statement of the problem

• We are in the third decade of what has become the most important infectious disease epidemic (pandemic) in the last century.

• UNAIDS fact sheet, estimated that, at end of 2007,

33.2 million people world wide were living with HIV/AIDS, 2.5 million become newly infected with HIV and 2.1 million lost their lives due to AIDS.

6

Statement cont...

• According to the single point HIV prevalence estimate, SNNPR region is one of the areas with high prevalence and Incidence of HIV/AIDS.

• The adult HIV/AIDS prevalence is estimated to be 1.5% with 0.2% Adult incidence.

7

statement cont…

• To get the required out comes of HAART treatment care with standardized quality is mandatory .

• Quality of care remains an area for improvement, despite the increased attention it has received in recent years.

8

Purpose of Evaluation

formative evaluation was planed for this evaluation and its main purpose is:

• Planning and management of the intervention which helps to improve the quality of HIV clinical care

• Assist in an efficient allocation of resources

• Report on the achievements of the intervention (i.e. accountability)

• Ensure optimum use of its funds and other resources

9

General Objective of the HIV clinical

service

The objective of care and treatment program in Wolayta Soddo Hospital is :

To improve the quality and length of life of PLWHA in Wolayta zone, by providing medical care and treatment service and thus contribute to the well being of their families and development of the country.

10

Program Activities

• Preparation of ART clinic and establishing information system in the hospital

• Training ART teams by communicating with the Regional Health bureau

• Searching for and establishing a link with care and support group

• Collecting necessary drugs and supplies from the regional health bureau

• Provision of ART and pre ART chronic HIV care for eligible PLWHA

• Conducting on site sensitization on ART and its referral system for staffs of health centers in the districts of wolayta zone.

11

Stage of development

• Soddo hospital was started HIV clinical care program in

June 2005. till the end of June 2008,

There were

792 PLWHA gets pre ART care

985 AIDS patients started on ART

12

Logic model of the program The basic logic model components of this Evaluation Illustrates the

connection between the planned activity and intended results.

Resources: Program Activities: Out put: Out comes:

Impact :

Final Evaluation proposal Marshet word doc.doc

13

Stakeholder Identification& Analysis

Stakeholders are individuals, groups, or organizations

that tended to have significant interests on how well a program functions.

Stakeholders are identified during evaluability assessment and proposal development.

14

Objectives of Evaluation

General objectives The general objective of this study is to assess the

quality of HIV clinical care provided by wollayta soddo hospital.

Specific objectives• To assess HIV clinical practice compliance with the

national guidelines.• To assess the Availability of human and material

resources required to provide quality HIV clinical care in Wolayta Soddo hospital.

• To assess adequacy of resources to provide HIV/AIDS clinical care in Wolayta Soddo hospital?

15

Evaluation Questions

Evaluation Questions Is the HIV clinical care provided by Wolayta Soddo

hospital according to the national guideline? if not why?

Are there adequate resources to provide HIV clinical care in Wolayta Soddo Hospital?

16

Methods

Study area and study period Wollayta zone is one of the biggest zones in SNNPR state. The

total population living in the zone for the year 2008 is 1.7 million.

In the zone there are 30 health center and 1 Government hospital, 1 private hospital and 1NGO hospital. Among these 21 health centers and the three hospitals are giving VCT services, ART services are given in 2 health centers and 2 hospitals.

The study was conducted in wollayta soddo hospital

The study pried was from December 1, 2008 – May 1, 2009 and data was collected from data was collected from December 1-20, 2008.

17

Methods cont…

Target population All patients receiving chronic HIV care with or without

ARV regimen In the hospital during the study period was target for this study.

Study units and sampling unit. Adult patient with HIV who attend chronic HIV care and

treatments was the sampling & study units for the study.

18

Methods cont… Sample size and Sampling Procedures Chart review Epi Info version 3.3.2 statcalc was used to calculate the

required sample size. According to the hospital records, the total number of patients on chronic HIV care and treatment was 1777

The resulting sample size was 316Were p. was the proportion of HIV patients who attends chronic HIV clinical care =50%

C.I= 95%. Margin of errors = 5% The sample size for pre ART: 316* 792/1777 =141 The sample size for ART: 316*985/1777 =175

19

Methods cont…

The study units was selected by using systematic random sampling. the first document was selected randomly by lottery system

Expert interview

The sample size for in depth interview was determined based on the adequacy of the information needed. (Using saturation sampling methods).

20

Methods cont…

Focus of Evaluation This evaluation primarily focuses on measuring processes of quality of HIV

clinical cares in wollayta soddo hospital Dimension of the evaluation

This study uses 3 dimensions of evaluation. Availability Measures the presence of resources interims of

infrastructure, equipment and supplies, and trained human in sufficient amount.

Compliance This dimension helps to measure HIV clinical care practice in the hospital comparable to the national guide lines.

Accommodation Measures the providers operation is organized in way that meet the constraints and preferences of the client.

21

Methods cont… Approach to evaluation

The evaluation approach is formative evaluation because its purpose is to help form or shape the program to perform better.

Design of evaluation

Case study design was used for this study.

The study was conducted through face to face interview, document review, and observational methods.

22Figure 2 theoretical Modal of Evaluation

Availability of resources

Accommodation

Increase Service utilization

Compliance to guideline

Decrease drug resistanceReduction of HIV related morbidity and mortalityIncrease productivity

Client SatisfactionIncreased treatment complianceIncrease service up-takeIncrease adherence to ARV DrugsDecrease loss to follow-up

Improved quality of service

23

Data Collection Procedures

Before the Data Collection, training was give to make the data collectors familiar with the information to be collected and to see the clarity of the different questionnaire

The questionnaires was pre-tested for its reliability and validity before it is

being used. In the Data Collection 5 Physicians 1 Pharmacy technician, 1 Laboratory

technologist and 5 senior ART nurses was participated.

To insure quality of data collection daily supervision was done by the P.E.

Availability of the necessary resources.

HIV clinical care compliance with the National guidelines Expert Interview

24

Inclusion and Exclusion criteria

Inclusion criteria for chart review All clients who have a minimum of two visits before the study period Exclusion criteria for chart review. Patients who have less than two visits before the study period. Patients who was lost to follow up and death are excluded from the

study

Inclusion criteria for expect interview. Experts who workers for more than 2 months on ART clinics

25

Data Management and analysis

• The data collector was recruited out of the study health facility. • During data collection period data was rechecked every time at

the end of each day. • After data collection & editing, quantitative data was entered and analyzed by using SPSS version 16.

• Both descriptive and analytic statistical technique was used to describe the final findings

• By identifying the thematic area of qualitative information, the data

was analyzed manually .

26

Limitation of the study

HIV clinical care offered to children is not evaluated by this study.

Some service providers may not respond honestly.

Some data collectors on observation check list may not feel the form honestly

27

Report writing and dissemination plan

The final report of this evaluation will be commented by Jimma university research comity, then the report was distributed both in hard and electronic copy to all stakeholders and interesting organization.

The copy of this evaluation report will also be submitted to Jimma University.

28

Results and discussion

AvailabilityHuman resource• Implementation of Effective HIV clinical services is

successful when staffs at program management level are trained

• In Wollayta soddo Hospital, there were four MD, ten ART nurses, five laboratory technician and eight pharmacy technicians are trained in ART and all of them are working in HIV clinical care department.

• Due to shortage of human power the four physicians have multiple responsibilities in addition to HIV clinical care. Compared to the national guideline the availability of trained human resource achievement resulted was 100 %.

29

Availability of OI and ART Drugs

• Highly active antiretroviral therapy (HAART) can reduce the amount of HIV in some ones body and restore their immune system

• The availability of OI drug in the hospital is found as expected. Among the required OI drugs, all are found to be available during the data collection period. This is the most heartening result seen in this evaluation and it has to be sustainable.

• Regarding the availability of ART drugs only 50% of the drugs

are available. when we observe the availability of ART drugs specifically, the resulting judgment criteria become fair.

30

Availability of OI and ART Drugs cont…

• In HIV infected patients on HAART 80% to 90 % adherence has been associated with failure to complete viral Suppression in 50 % of patients. one of the factors associated with non – adherer to ART& OI drug is poor drug supply.

• From this specific evaluation finding It is possible to hypothesize that, In Wollayta soddo Hospital patient’s on HIV clinical care are highly exposed for none adherence to ARV drugs due to stock outs of the drugs.

31

Number of rooms for ART SERVICES

• Facilities infrastructure are required to be accredited before starting to provide ART service.

• Regarding infrastructure for HIV clinical care: there were one examination, one separate counseling, one onsite pharmacy, one storage space and one specimen collection rooms are available.

• According to the Judgment criteria, the availability of rooms for ART services Compared to the national guidelines found to be excellent (100%)

32

Number of rooms cont…

• In similar evolutions study conducted in Addis Ababa, The physical set up compared with the national Guideline was found to be 80 % which is in adequate.

• This study result was better when compared to the above evaluation findings.

33

Availability of Laboratory Services

• The HIV clinical care department of the Hospital uses laboratory services from the laboratory departments of the hospital.

• Some of the laboratory reagents for the service, which are not available include: Indian ink, BUN and cretinin. According to the above result the availability of Laboratory service in the hospital was judged as good (66.6%)

34

Availability of The necessary Laboratory and pharmacy supply and equipment

The availability of laboratory and pharmacy supply and equipment was found as expected. Out of the 8 necessary Laboratory and pharmacy supplies and equipment recommended by the national guidelines, 6 of them are found functional.

Hematology auto analyzer and clinical chemistry auto analyzer were not available in the Hospital. According to the judgment criteria, the result is found to be 75 % (Good).

35

Compliance to the National Guideline

To assess compliance of clinical care for patients with HIV infection in Wollayta Soddo Hospital with the national treatment guideline, a total of 316 charts of patients attending HIV clinical care in the hospital were reviewed.

Out of these 175 (55.3) were on ART and the rest 141 (44.6) were on pre ART HIV clinical care.

36

Socio demographic characteristics of study population for chart review

• The mean age of the study population was found to be 30.6 years (SD + 8.2). More than half (57%) are between the age of 18 and 30 years.

• The majority of the study population in the sample (37.9%) had primary school education about 26.5% were not able to read and write at all. only 1.9% completed college level education.

37

38

Socio demographic cont…

• Marred make up significant proportion (49.2) of the sample, Followed by single 26.2 and widowed 17.4 percent. 6.9 percent of the study populations were divorced.

• More than half of the study subjects (61.5%) were female and the rest are male. Table of socio demographic characteristics. final draft of Evaluation thesis marshet word doc.doc

39

Evaluation and follow up of HIV clinical care of the patients

• To provide, continuum of HIV care at different level trained man power is an essential entity that needs to be strengthened and highly coordinated.

• Accordingly Wollayta soddo Hospital has adequate trained man power and more than 96% of the patients are evaluated by trained physician.

• Compared to the national guidelines this result is considered to be excellent and needs to be continued in the future.

40

Evaluation and follow up cont…

• Patient monitoring and evaluation is an important part of high quality of patient care.

• Out of 316 patients who have at least two follow up on HIV clinical care in the hospital,180(56.8%) of their clinical status was monitored on every visit for the last 6 months.

• This shows that the proportion of clients whose clinical status is monitored every visit is judged to be faire.

• which is not accepted according to the agreed up on evaluation parameters & by the national guideline so It needs argent correction.

41

42

Cotrimoxazol prophylactic therapy (CPT) and TB screening

• Several clinical trails in Africa have shown the impact of CPT in reducing morbidity and mortality in PLWHA

• Studies in West Africa suggest that Cotrimoxazol prophylaxis reduces the morbidity and mortality related with opportunistic infection specially patients with TB HIV Co infection.

43

CPT prophylactic cont…

• According to the national TB /HIV implementation guideline criteria, in this evaluation study 91.6 % of the eligible Patients are prescribed with CPT prophylaxis therapy.

• In similar evaluation study conducted in one referral Hospitals of the Amhara National Regional state, only 45.9% of eligible patients were taking Cotrimoxazol prophylaxis therapy.

• The acivments of Wolayta soddo hospital with regard to this components of HIV care was found to be better when compared to the above study findings.

44

CPT prophylactic cont…

• Based on TB/HIV implementation guideline, all HIV positive clients should be informed about the advantage of being screened for TB& they should under go screening for TB.

• This evaluation finding identifies, in Wollayta soddo Hospital only 56.2% of the sampled HIV positive case are screened for TB which is far lower than the national TB/HIV implementation Guideline. This needs argent Correction.

45

ART and OI Adherence Counseling

According to the national guidelines at least 95% adherences required for ART and OI regimen to be fully effective and avoid the emergence of resistant strains of the viruses.

Accordingly this evaluation study revealed that adherence counseling and evaluation of adherence status was done for 52.7% & 88.5 % of patients respectively.

Since Adherence counseling do have direct relation with adherence status of the patient the above result is not acceptable according to the agreed evaluation criteria.

46

ART and OI Adherence cont…

• None adherence in patients on anti HIV/Therapy is the strongest predictor of failure to achieve viral suppression below the level of detection and faulty adherence to anti HIV drugs most often underlies treatment failure.

• From this point of view the above evaluation findings needs urgent correction

47

Staging and follow up of patients with CD4 count

Out of 316 patients included in this evaluation study CD4 count was performed at least once during the last 6 months for 173 (54.6) patients.

This makes the proportion of patients who have CD4 counts at least once during the last 6 month to be judged as Fair.

Wollayta soddo Hospital uses the WHO staging criteria for the follow up of disease progression in each patient.

48

Staging and follow up cont…

According to this criterion, from a total of 316 patients who are on chronic HIV clinical care 239(75.4) of the patients are staged correctly, the rest 77 (24.6) are not staged correctly.

This result concludes the proportion of patients who are correctly staged during there last visit to be judged as Good.

49

Initiation of ART treatment

• From a total of 211 patients who are eligible for ART 174(82.4) of them are started ART, The rest 37(17.6) even thou they are eligible for ART they didn’t started ART with in 2 weeks of their eligibility time and they are not currently on ART.

• After the initiation of the ART treatment 9 of the 10 patients declared that there health condition is being improved from time to time.

50

INITIATION OF ART TREATMENT

• In similar study undergone in Amhara national region the result showed that only 76.8 % of eligible patients are started ART.

• According to agreed evaluation criteria even thou this results is Judged to be very good, from the point of view of benefit gained by timely Initiation of ART treatment and the need for the result to be nearly 100 % this finding needs Improvement.

51

Evaluation of ART adherence and functional status of the patients

• Out of 175 patients who started antiretroviral treatment, adherence status was evaluated for 155(88.5%) of the patients, in each of there visit.

• This makes the proportion of eligible patients whose adherence status is estimated every visit to be judged as very good.

52

Evaluation of ART adherence cont…

• According to the standard operational procedures of ART treatment manual, the functional status of the patient needs to be evaluated every visit to asses the progress of the patient in line with the treatment.

• From a total of 316 patients included in this evaluation study, the functional status was assessed every visit for 206 (65%) of patients. This results the specific Indicator to be judged as Good.

53

Accommodation

Service time• According to national ART guideline, all health facilities

are expected to provide chronic HIV clinical care 5 days per week and eight hours per day.

• In Wollayta soddo Hospital the service was given to the patient 6 days per week and eight hours per days. This is in line with the national guide line.

• Based on the judgment criteria set with the stakeholders this indicator is jugged as excellent (100%)

54

Accommodation cont…

Waiting time

• One of the reasons for increased defaulter rate and loss to follow up is long waiting time. To avoid this the waiting time should have to be shortened.

• From a total of 10 exit interview, 8 of them responded that the length of waiting time is short; the rest responded the waiting time is long.

The level of achievements for judgment matrix of evaluation final draft of Evaluation thesis marshet word doc.doc

55

Conclusion and Recommendation

Conclusion• The average achievement of availability was 81.8%

which is judged to be very good.

• The least achieved indicator in this dimension is presence of laboratory service, according to the requirements of the national guidelines which is only 66%.

• From this we can conclude that even though the average achievement on the availability is said to be very good it needs some improvement on the list achieved parts of this indicator

56

Conclusions cont…

• Based up on the national guideline and agreed up on judgment criteria, compliance with the national guide line achieved only 72.7 % of the expected results.

• Providing adherence counseling for eligible patients is the least achieved indicators in this dimension. This needs urgent improvement.

• On concrete term the HIV clinical service in Wollayta soddo hospital is not fully complained to the national guideline and the level of achievements for more than 40% of the composed indicator is only faire. This needs special attention and consideration.

57

Conclusions cont…

• The findings from this evaluation revealed averagely near to 80% of the expected quality indicators are achieved.

• In conclusion the overall quality of HIV clinical care implementation in the hospital was found to be good, but not more than average level of the expected achievements. In general the final result of this evaluation needs improvement.

58

Recommendation

1. Wollayta soddo Hospital needs to establish sustainable and reliable ART supply chain system in collaboration with the Regional health Bureau and zonal health Department.

2. The Regional health Bureau should have to provide timely & sustainable procurements and distribution of ARV Drugs.

3. Wollayta soddo Hospital is expected to avail all laboratory service According to the recommendation of the national Guidelines.

4. Since patient care at regular interval is one of the important quality indicator that needs to be followed by the clinician, all patients’ clinical status needs to be monitored during there initial and follow up visits.

59

Recommendation

5 TB is the most important opportunistic infection in PLWHA, to avoid the risk of TB in HIV infected individual, all HIV positive individual in the hospital needs to be screened for TB.

6 In Wollayta soddo Hospital ART &OI adherence counseling and evaluation of adherence should be done for all eligible patients in accordance with the national guideline.

7 Timely Initiation of ART treatments is vital in effectiveness of HAART. Wollayta soddo Hospital needs to give due emphasis for timely Initiation of ART treatments for all eligible patients, since the level of achievements for this Indicator is very Low than expected.

60

Recommendation

8 Wollayta soddo Hospital should establish regular and sustainable self assessment & Improvement mechanism for HIV clinical care given in the hospital.

9 Since there is memory Laps and high attrition of health

workers, The Regional health Bureau in collaboration with the zone health Department should plan &provide sustainable basic& refresher training for the health workers.

10 The Regional health Bureau and the zonal health Department should establish a system to mentor quality of HIV clinical care provided to the patients.

61

Meta- evaluation To assure the quality of this evaluation different Activities have been done in

addition to those activities, the evaluation was evaluation by using the following four standards of evaluation.

Utility: The over oll procedure of the evaluation is based on participatory approaches. The intended stakeholders was involved in every aspects of the evaluation to ensure that an evaluation will serve the information needs of its users.

Feasibility: This evaluation follows scientific evaluation procedures with a great political supports. the information produced in this evaluation was

sufficient value ,so that the resource expended was justified

Propriety: - after the evaluation proposal is assured by Jimma University, the evaluation was conducted legally and ethically by fulfilling oll the ethical procedures.

Accuracy: to determine the merit or worth of the program being evaluated, this evaluation will reveal technically adequate information by giving grate care in every activity of the evaluation.

62

References CDC frame work for program Evaluation in public health, September,

1999/volume 48/No.RR-11 Program Evaluation an introduction, David Royse, Brace A.thyer, Deborah K.

Padgett, T.K. Logan. Fourth Edition. Chan Kam, c., Good ridge, G., and Moodie, R"strategic planning, program design

and management" in P.Lamptey and H.Gayle (eds), HIV/AIDS prevention and care in resources constrained settings: a hand book for the design and managements of program .Arlington, va: family health international, 2001.

UNAIDS report on the Global HIV/AIDS epidemic, 2006 Tony Barrnett and Alan white side "AIDS in the 21st century Disease and

Globalization " pal grave Macmillan, 2003 AIDS in Ethiopia sixth Report, federal ministry of health national HIV/AIDS

prevention and control office September, 2006 National strategic planning for multi sectorial response, MOH 197-2000

63

Reference cont…

Guide line for the implementation of antiretroviral therapy program in Ethiopia, Federal HIV/AIDS prevention and control office, Federal ministry of health, July 2007.

National AIDS program, A Guide to monitoring and Evaluating HIV/AIDS care and support, UNAIDS, WHO, UNICEF, 2004 scaling up antiretroviral therapy in resources limited setting, Guide line for using antiretroviral agents among HIV infected adults and adolescents. Retrieved on 15 September 2007 Peride M. An introduction to quality Assurance in health care. International Journal for quality in health care 2003 Evaluability Assessment: A primer, Michael S Trevisan and Yi Mim Huany

Washington state university, 2003 Meta Evaluation by Daniel Stufflebeam, December 1974

64

Reference cont…

AID Ethiopia and Intermon-Oxfam, participatory project planning, Monitoring and Evaluation, Training manual, 2005

Prism Evaluation consulting services. Retrieved from

www.collegs.org/techcenter/if/2004 conference/ Stakeholders participation, Stakeholders of program in planning and

implementing Evaluation activities,Retrieved from www.cdc gov/STD /program/papesttd/step. text. Accessed on

June 10, 2008 Focus the Evaluation Design Retrieved from www.cdc.gov/Drugresistance Evaluations our Approach Retrieved from www.irvine.org/evaluation/approuch.shtml.

65

Reference cont… Bick man, the function of program theory. in L.Bickman(Ed) using program theory in Evaluation. New direction for program Evaluation,no.33 san Francisco.Jossey-bass,1987. AIDS Institute of the new York state department of health protocol for the

primary care of HIV/AIDS in adult and adolecent,1995

FHI Antiretroviral therapy (ART) 2006 http:// www.fhi.org/EnTopics/Topics Antiretraviraltherapy. Htm Retrieved on

June 2008-06-23

Donabidian A. Exploration in quality assessment and Monitoring. Annarbor, Michigan: health administration press, 1980

Quality of care indicators for HIV/AIDS. A discussion paper of the foundation for Accountability (FAC Allbert wu,

Allen Gifford, Steven Asch, 1998

66

Reference cont…

• Expanded response guide to indicators for monitoring and reporting on HIV/AIDS program,

• November 2002• Treating opportunistic infection Among HIV infected adults and adolescent’s

CDC, 2004.• Bond W& Hussar, D,A(1991) Detection methods and strategies for

improving medication adherence. American journal of public health 1987 – 1988.

• Brigido, L.F ,Veiga A.P,D. Ambrosio,A.C, Bueno, A casseb, J.& Galbitti, F.F.(1998). Low adherence in ART users at sao paolo, Brazil l2th world AIDS conference, Geneva,

• African health, 1999 volume 21, Number3• Meseret demise, Evaluation of quality of ART program implementation in

federal police hospital, Addis Ababa, Ethiopia, May 2008.• Effectiveness of CPT, kate Grimwade: A.willem sturn: Andrew J,Nunn:

AIDS, 2005,

67

Reference cont…

• TB/HI implementation Guideline, Federal Ministry of health Ethiopia, July 2005.

• Yibeltal kiflie, An Evaluation of HIV/AIDS clinical care quality in Amhara national Regional state. The case of Felege Hiwot referral hospital, north western Ethiopia, March 2008.

• Brgido, L.F.reiga, A.P d ambrosia, A.C Buena, A.casseb.J& oliteF.F (498)low adherence in HAART users at.sao. Paola, Brazil 12th world AIDS conference, Geneva

• Bond.W&Hussaf, D.A (1991) Detection methods and strategies of improving mediation adherance.American journal of public health, 1978 – 1988.

• Broers, B. Morabia, A.Iilschel, B(1994).A cohort study of drug users compliance with Zidovudine treatment.

68

Acknowledgment

Professor Abebe Gebremariam Mr. Yibeltal kiflie Dr. Jemal Aliy Professor Elithabeth Moreira Dossantos Professor Carl Kendal Dr Wuleta lema and Tulen University. wollayta zone health department & wollayta soddo hospital

69