Monitoring and Evaluation Module 12 – March 2010.
-
Upload
colin-marshall -
Category
Documents
-
view
215 -
download
0
Transcript of Monitoring and Evaluation Module 12 – March 2010.
Monitoring and Evaluation
Module 12 – March 2010
Project Partners
Funded by the Health Resources and Services Administration (HRSA)
Module Overview
Monitoring & Evaluation (M&E) framework and components
Records, registers and reports
Recording and evaluating response to TB treatment regimens
Supervision
International Standards 13 and 21
Learning Objectives
At the end of this presentation, participantswill be able to: Describe what is meant by “Monitoring and
Evaluation”
Discuss the importance of collecting data and ensuring the accuracy of the data
Explain ways in which the data are used to evaluate treatment
Describe how M&E activities can benefit both TB and HIV/AIDS programs
Monitoring & Evaluation System
A key element of the Stop TB Strategy
Allows programs to:
• Monitor progress and treatment outcomes of individual patients
• Evaluate the overall performance of the TB program at various levels (local, district, national)
• Identify areas of program improvement and weakness
• Ensure accountability
Monitoring
What is it?
• Routine tracking of services and program performance
Monitoring (2)
How is it done?
• Through information collection, data input, analyzing the data, and reporting what is found in that analysis outcome report
Why should we do it?
• To better assess how well a policy or program is achieving its intended target
Evaluation
What is it?
• Episodic assessment of results that can be attributed to program activities
• Types of evaluation related to M & E:
➜ Process evaluation: assesses the progress in program implementation and coverage
➜ Outcome and impact evaluation: measures the effect of the program activity on the target population
What are the Targets?
Stop TB Partnership/WHO• 70% TB case detection and 85%
cure rate by 2005
Millennium Development Goals (MDG):• Halt, and begin to reverse, the incidence of major
diseases such as HIV/AIDS and TB by 2015 Decrease TB prevalence and death rates to 50% of
the 2000 estimates
United Nations General Assembly Special Session (UNGASS) – global targets
Where do Indicators Fit In?
Indicator: a specific, observable, and measurable characteristic or change that shows the progress a program is making toward achieving a specific outcome
Indicators may be expressed in terms of:• Number
• Rate
• Proportion
• Percentage
Limitations of Indicators
Indicators DO NOT:
Measure everything
Tell us why a problem may exist or how to fix it
Determine if problems identified are amenable to intervention
Tell us which interventions are most cost effective
What are some possible uses of data collected by the National HIV/AIDS and
TB Programs?
Using and Disseminating Data
M & E can improve and enhance NAP and NTPwork by: Identifying areas of strengths and weaknesses
Helping plot progress toward program goals
Allowing a program to see trends and to identify high risk groups in order to better target TB control efforts
Providing justification for needed resources
Identifying training and supervision needs
Increasing public awareness about TB
Advocating for policy changes and allocation of funds
Monitoring & Evaluation Framework
CONTEXTEnvironmental, cultural, political, and socio-economic factors external to the programme
Monitoring/Process Evaluation Outcome/Impact Evaluation
INPUT
Basicresourcesnecessary
• Policies• People• Money• Equipment
PROCESS
Programmeactivities
• Training• Logistics• Management• IEC/BCC
OUTPUT
Results at theprogrammelevel(measure ofprogramme activities)
• Services• Service use• Knowledge
OUTCOME
Results atlevel of targetpopulation
• Behaviour• Safer practices
IMPACT
Ultimate effectof project inlong term
• TB incidence• HIV prevalence• Morbidity• Mortality
Types of M & E Activities
Staff training
Supervision
Completion of reporting forms
Discussions during staff meetings
Ensuring medicine and laboratory stock supplies
Quality control activities
Analysing data and preparing reports
Standard 13: Recording & Reporting
A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients
Standard 21: Recording & Reporting
All providers must report both new and re-treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies
Form 3: BASIC MANAGEMENT UNIT TB REGISTER – LEFT SIDE OF THE REGISTER BOOK
Type of patient 4 Date of
registration District TB No.
Name Sex
M
/F
Age
Address Health facility1 Treatment supporter2
Date treatment
started
Treatment category 3
Site P / EP N R F D T O
1 Facility where patient’s treatment card is kept. In case several copies are kept, the most peripheral facility should be entered. 2 including community worker/volunteer, family members or friends. 3 Enter the treatment category: CAT I: New case
CAT II: Re-treatment e.g. 2(HRZE)S/1(HRZE)/5(RHE) Chronic:patient sputum positive at the end of a re-treatment regimen. Chronic cases still alive and not started on Category IV treatment should be re-entered at the beginning of each year. Patients who are started on Category IV treatment should be entered in a separate Category IV register and separate Category IV treatment cards should be used for them. 4 Tick only one column : N=New – A patient who has never had treatment for TB or who has taken antituberculosis drugs for less than 1 month. R=Relapse – A patient previously treated for TB, declared cured or treatment completed, and who is diagnosed with bacteriological (+) TB (smear or culture).
F=Treatment after failure – A patient who is started on a re-treatment regimen after having failed previous treatment. D=Treatment after default – A patient who returns to treatment, positive bacteriologically, following interruption of treatment for 2 or more consecutive months. T=Transfer in – A patient who has been transferred from another TB register to continue treatment. This group is excluded from the quarterly report on registration. O=Other previously treated– All cases that do not fit the above definitions. This group includes smear-positive cases with unknown outcome of previous treatment, smear negative previously treated, EP previously treated and chronic case (i.e. a patient who is sputum positive at the end of a re-treatment regimen)
Form 3: BASIC MANAGEMENT UNIT TB REGISTER – RIGHT SIDE OF THE REGISTER BOOK
Results of sputum smear microscopy and other examination Treatment outcome
& date TB/HIV activities
Before treatment 2 or 3 months 1 5 months End of treatment
Smear result
Date/ Lab. No.
X-ray Date/ Result
4
Smear result
Date/ Lab. No.
Smear result
Date/ Lab. No.
Smear result
Date/ Lab. No.
Date
Outcome in text
2
HIV result3
/ Date/
No. HIV reg
ART Y/N
Start date/ No. ART reg
CPT Y/N Start date
Remarks
1CAT 1 patients have follow-up sputum examination at 2 months; CAT II patients have follow-up sputum examination at 3 months. CAT 1 patients with extended phase 1 to 3 months have follow-up sputum examination at 2 AND 3 months with results registered in the same box. 2Enter the code (1-6) as follows:
1-Cure: Sputum smear positive patient who was sputum negative in the last month of treatment and on at least one previous occasion. 2-Treatment completed: Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure. 3-Treatment failure: New patient who is sputum smear (+) at 5 months or later during treatment, or who is switched to Category IV treatment because sputum turned out to be MDRTB. Previously-treated patient who is sputum smear positive at the end of his retreatment or who is switched to Category IV treatment because sputum turned out to be MDRTB. 4-Died: Patient who dies from any cause during the course of treatment. 5-Default: Patient whose treatment was interrupted for 2 consecutive months or more. 6-Transfer out: Patient who has been transferred to another recording and reporting unit and for whom treatment outcome is not known.
3 + positive, - negative, U unknown, ND Not Done. Documented evidence of HIV test performed during or before TB treatment is reported here. 4 + : suggestive of TB, -: not suggestive of TB, ND: not done.
Why is accurate reporting and record keeping important?
Data Quality Assurance
Ensures that the information collected adequately represents the program’s activities
Accurate data – measuring what it is intended to measure
Reliable data – collected and measured the same way by all program personnel over time
Reporting Forms and Registers
Request for Sputum Examination
Tuberculosis Treatment Card
Tuberculosis Identity Card
Basic Management Unit TB Register
TB Laboratory Register
Quarterly Report on TB Case Registration
Quarterly Report on Sputum Conversion
Register of TB Suspects
Quarterly Report on Treatment Outcomes
Yearly Report on Program Management in Basic Management Unit
Tuberculosis Treatment Card
Tuberculosis Treatment Card BMU TB Register No._____________
Name: ________________________________________________________ Disease site (check one) Pulmonary Extrapulmonary, specify ___________
Type of patient (check one) New Treatment after default Relapse Treatment after failure Transfer in Other, specify ___________________
Sex: M F Date of registration: ____________________________
Age: ________ Health facility: _________________________________
Address: ________________________________________________________ ________________________________________________________________
Name / address of community treatment supporter (if applicable) ________________________________________________________________
Sputum smear microscopy Weight (kg)
Month Date Lab No. Result 0
TB/HIV Date Result* HIV test
CPT start ART start
* (Pos) Positive; (Neg) Negative; (I) Indeterminate; (ND) Not Done/unknown
I. INITIAL PHASE - prescribed regimen and dosages
CAT (I, II , III):
Number of tablets per dose and dosage of S: (RHZE) S
Referral by :
Self-referral Community member Public facility Private facility/provider Other, specify --------------------------------
Cotrimoxazole ARV Other
Tick appropriate box after the drugs have been administered Daily supply: enter . Periodic supply: enter X on day when drugs are collected and draw a horizontal line ( ) through the number of days supplied. Ø = dr ugs not taken
Day Month
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Please turn over for continuation phase
Tuberculosis Treatment Card (2)
Comments: _________________________________________
___________________________________________________
___________________________________________________
___________________________________________________
___________________________________________________
II. CONTINUATION PHASE
Number of tablets per dose Daily supply: enter . Periodic supply, enter X on day when drugs are collected and draw a horizontal line ( ) through the number of days supplied. Ø = d rugs not taken
Day Month
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
X-ray (at start) Date: Results (-), (+), ND
Treatment outcome Date of decision ____ Cure Treatment completed Died Treatment failure Default Transfer out
HIV care Pre ART Register No. CD4 result ART eligibility (Y/N/Unknown) Date eligibility assessed ART Register No.
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
______________________________________________________________________________________
Name and address of contact person: ______________________________________________________________________________
(RH) (RHE) (Other)
Preparing a TB Treatment Card
Activity
Tuberculosis Identity Card
This card contains an extract of information on the treatment card
It is given to the patient at the start of treatment
It is used to record daily DOT and must be used during the intensive phase of treatment
It also serves as a reference document for TB status after treatment
It should be presented to the doctor whenever the patient falls ill in the future
Tuberculosis Identity Card (2)Tuberculosis Identity Card
Name __________________ BMU TB Register No. _____ Address __________________ Date of registration: _______ Sex: M F Age ______ Date treatment start _______ Health facility: ______________________________________ Supporter (name and address) __________________________
Sputum smear microscopy Weight (kg)
Month Date Lab No. Result
Disease site (check one) Pulmonary Extrapulmonary, specify _______
Type of patient (check one) New Treatment after default Relapse Treatment after failure Transfer in Other specify ______________
I. INITIAL PHASE CAT (I, II , III): (RHZE) S Other Drugs and dosage:
II. CONTINUATION PHASE (RH) (RHE) Other Drugs and dosage:
Appointment dates: ________________________________ __________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________ __________________________________________________
REMEMBER
Basic Management Unit TB Register
This revised register is the cornerstone of an NTPs monitoring & evaluation system
It records essential information for notification & treatment outcome by district
It should always be kept up to date with data on sputum smear examinations and treatment outcome
Where electronic data collection systems are available, the information from the register should be entered into the database at least once every month
Basic Management Unit TB Register – Left side of the register book
Date of registration
BMU TB No.
Name Sex
M
/F
Age
Address Health facility1
Date treatment
started
Treatment category2
Site P/EP
Type of patient3
N R F D T O
Footnotes appearing on first page of the register only. 1 Facility where patient’s treatment card is kept. In case several copies are kept, the most peripheral facility should be entered. Use standardized type of health
facilities according to block 2 of the Yearly Report on Programme Management in BMU. Health facility is defined as any health institution with health care providers formally engaged in any of the following TB control functions (DOTS): referring TB suspects/cases, laboratory diagnosis, TB treatment and patient support during treatment.
2 Enter the treatment category: CAT I: New case of sputum smear microscopy positive, severe
sputum smear microscopy negative PTB & EPTB e.g. 2(RHZE)/4(RH)
CAT II: Re-treatment e.g. 2(RHZE)S/1(RHZE)/5(RHE) CAT III: New sputum smear microscopy negative PTB and EPTB
e.g. 2(RHZE)/4(RH) 3 Tick only one column:
N=New – A patient who has never had treatment for TB or who has taken antituberculosis drugs for less than 1 month.
R=Relapse – A patient previously treated for TB, declared cured or treatment completed, and who is diagnosed with bacteriological (+) TB (sputum smear microscopy or culture).
F=Treatment after failure – A patient who is started on a re-treatment regimen after having failed previous treatment.
D=Treatment after default – A patient who returns to treatment,
positive bacteriologically, following interruption of treatment for 2 or more consecutive months.
T=Transfer in – A patient who has been transferred from another TB Register to continue treatment. This group is excluded from the Quarterly Reports on TB Case Registration and on Treatment Outcome.
O=Other previously treated– All cases that do not fit the above definitions. This group includes sputum smear microscopy positive cases with unknown history or unknown outcome of previous treatment, previously treated sputum smear microscopy negative, previously treated EP, and chronic case (i.e. a patient who is sputum smear microscopy positive at the end of re-treatment regimen)
TB Register in Basic Management Unit using Routine Culture and DST – Right side of the register book Results of sputum smear microscopy and other examinations Treatment outcome & date TB/HIV activities
Remarks
Before treatment 2 or 3 months 1 5 months End of treatment
Date
Outcome in text 7
ART Y/N Start date
CPT Y/N Start date
Sputum smear micros-
copy date/No./ Result
2
HIV result3/
Date
X-ray Result
4/ date
Culture
date/No./ Result
5
DST date/No./ Result
6
Sputum smear micros-
copy No./
Result2
Culture No./
Result5
Sputum smear micros-
copy No./
Result2
Culture No./
Result5
Sputum smear micros-
copy No./
Result2
Culture No./
Result5
Footnotes appearing on first page of the register only 1 CAT I patients have follow-up sputum smear microscopy examination at 2 months; CAT II patients have follow-up sputum smear microscopy examination at
3 months. CAT I patients with initial phase of treatment extended to 3 months have follow-up sputum smear microscopy examinations at 2 AND 3 months with results registered in the same box.
2 (ND): Not done; (NEG): 0 AFB/100 fields; (1-9): Exact number if 1 to 9 AFB/100 fields; (+): 10-99 AFB/100 fields; (++): 1-10 AFB/ field; (+++): > 10 AFB/ field 3 (Pos):Positive; (Neg):Negative; (I):Indeterminate; (ND):Not Done / unknown. Documented evidence of HIV test performed during or before TB treatment is
reported here. Measures to improve confidentiality should accompany recording of HIV status. 4 (Pos): Suggestive of TB; (Neg): Not suggestive of TB; (ND): Not Done. 5 (Pos): Positive; (Neg): Negative; (ND): Not Done. 6 (ResistR): Resistant to Rifampicin; (ResistH): Resistant to Isoniazid; (ResistE): Resistant to Ethambutol; (ResistStrept): Resistant to Streptomycin;
(ResistRH): Resistant to Rifampicin and Isoniazid; (Suscept): Susceptible; (ND): Not Done. 7 Write clearly ONE of the following outcomes per patient:
Cure: Patient with culture or sputum smear microscopy positive at the beginning of the treatment who was culture or sputum smear microscopy negative in the last month of treatment and on at least one previous occasion. Treatment completed: Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure. Treatment failure: New patient who is culture or sputum smear microscopy positive at 5 months or later during treatment, or who is switched to Category IV treatment because sputum smear microscopy turned out to be MDRTB. Previously-treated patient who is culture or sputum smear microscopy positive at the end of his re-treatment or who is switched to Category IV treatment because sputum turned out to be MDRTB. Died: Patient who dies from any cause during the course of treatment. Default: Patient whose treatment was interrupted for 2 consecutive months or more. Transfer out: Patient who has been transferred to a health facility in another BMU and for whom treatment outcome is not known.
Treatment Outcomes
Cured
Treatment completed
Treatment failure
Died
Default
Transfer out
Treatment Outcomes
Cure Patient whose sputum smear or culture was positive at beginning of treatment but who was smear- or culture-negative in the last month of treatment and on at least one previous occasion
Treatment Completed
Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure
Treatment Failure
Patient who is sputum smear-positive at five months or later during treatment– or –Patient found to harbor a MDR strain at any point of time during treatment, whether smear-negative or -positive
Treatment Outcomes (2)
Died Patient who dies for any reason during the course of treatment
Default Patient whose treatment was interrupted for 1 month or more
Transfer Out
Patient who has been transferred to another recording and reporting unit and whose treatment outcome is not known.
Treatment Success
A sum of cured and completed treatment (smear-positive or culture-positive patients only)
Supervision
How is supervision used in your TB and HIV/AIDS Prevention and Control
Programs?
Role of Supervision in M & E
Supervision is a process of guiding, helping, training, and enabling staff to improve their performance in order to provide high quality health care services
Purpose of Supervision
Provide leadership and direction to staff
Ensure effective program implementation
Monitor operations and evaluate achievement of goals
Ensure adherence to laws and policies
Avoid confusion or duplication of efforts
Purpose of Supervision (2)
Monitor that all necessary tasks are properly performed
Ensure that resources are properly used and are available to staff, including training and supplies to carry out their duties
Ensure accountability
Barriers to Effective Supervision
Lack of commitment
Lack of proper planning and time management
Lack of tools for Monitoring & Evaluation
Insufficient staff
Problems with transportation
Lack of confidence or preparation
Roles and Responsibilities
National Level – NTP and NAP
• Planning, implementing, monitoring, and evaluating Program at all levels
Regional/District/Parish Levels
• Coordinating, supervising, planning, implementing, monitoring and evaluating all aspects of TB and HIV/AIDS Programs in the region
How Can M&E Information be used in TB and HIV Programs?
Identify gaps in performance
Monitor treatment outcomes
Measure the impact of an intervention or policy change
Identify populations for enhanced control and prevention efforts
Identify local problems as they arise
Ensure high quality TB and HIV prevention and control strategies are consistently provided
Summary: ISTC Standards Covered
Standard 13: A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients
Standard 21: All providers must report both new and re-treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies
Summary
Several approaches are used to monitor and evaluate TB and HIV/AIDS programs including supervision, training and the keeping of records and registers
Accurate and timely reporting and record keeping is important. It allows true assessment of Program achievements