Surveillance EvaluationAssist Prof Dr. Kwankate Kanistanon, DVM, MS,

PhD

What is health surveillance

“The tracking and forecasting of any health event or health determinant through the continuous collection of high quality data, the integration, analysis, and interpretation of data into surveillance products and the dissemination of surveillance products to those who need to know a specific public health purpose or policy objective”

Why evaluate surveillance

To improve and support a health surveillance

Evaluation helps answering the questions

1. What are the success and deficiency of the surveillance system?

2. Is the surveillance system meeting its piblic health objective?

3. How does surveillance support and benefit stakeholders?

4. What measures could improve performance and productivity of the surveillance system?

What to evaluate?

1. The quality of the information that their systems produce

2. The effectiveness in supporting the objectives of the programs that they serve

3. The effectiveness in supporting informed decision-making

4. The efficiency of their systems

Effectiveness

How well surveillance can produce purposed outcome

Is the information use? Response to the objectives of

surveillance

Efficiency

Cost, time, and labor Resources consumed and the

value of outcomes

6 Steps for surveillance evaluation

1. Establishing the context of the surveillance 2. Developing evaluation questions 3. Designing the process for data collection

and management 4. Summarizing and presenting the findings 5. Reviewing an evaluation report 6. Following up on the use of findings

Step 1: Establishing context

Why evaluate? You must be able to identify the reasons for evaluation

Who are stakeholders for evaluation? You must identify the persons or organization that will benefit from the evaluation

What is this surveillance system? You must find information of surveillance before you evaluate it

What is this surveillance?

Purpose Population under surveillance What data is collected and

summarized Who uses the surveillance What action is expected from data

analysis and interpretation Activities of surveillance

Example of surveillance activities

Identify the disease, case, population Develop data collection tools, active or

passive surveillance Data collection Data audit and analysis and Interpretation Report writing Dissemination of results

Step 2: Evaluation questions

Questions are designed to ask for 2.1. Acceptability 2.2. Simplicity 2.3. Flexibility 2.4. Data quality 2.5. Positive predictive value 2.6. Sensitivity

Step 2: Evaluation questions

2.7. Representativeness 2.8. Timeliness 2.9. Stability 2.10. Compliance

2.1 Acceptability

= Willingness of persons and organizations to participate in the surveillance Are the data collected within the normal

course of operations? Is there a mutual understanding of

jurisdictional mandates, addressing security, and privacy

2.2 Simplicity

The ease of operation Appropriate technologies and labor

2.3 Flexibility

Accommodate changes in operating conditions Can the system respond to new

conditions, such as changes in case definition, labor, equipment

Can the system accept, process and forward another system’s information

2.4 Data quality

Can data describe the health event according to the objective of the surveillance?

Is the quality of data acceptable? Missing data Correct conversion Historical and current data having different meaning Same data have more than one definition

2.5 Positive predictive value

= The percentage of cases reported to the system that actually have the health problem How much the diagnosis depend on

skill of observers? Is there a process that the diagnosis is

confirmed?

2.6 Sensitivity

= Percentage of cases with disease detected by the surveillance Ability to detect outbreak

in a timeframe correct number of diseased animals

2.7 Representativeness

The surveillance can explain the health problem by person, place, and time Can describe population

characteristics Can describe diseased animals

characteristics

2.8 Timeliness

Appropriateness of interval between disease occurrence and Report to health agency Identifying the incidence by health

agency Implementation of control measures

2.9 Stability

Stability is measured by amount of time required to manage and disseminate the information to decision makers

2.10 Compliance

Comply with law or policy Ethics Confidentiality Security Privacy

Step 3: Data collection/Management

Collect data for evaluation Observation Interview the surveillance staff Reviewing the surveillance report

Step 4: Findings

Summarizing information gathered from the evaluation

Step 5: Reviewing the report

Presentation and dissemination of results How and whom should the report sent to? What evaluation report should say?

Clarify the weak and strong point of the surveillance

Any suggestion for surveillance?

Step 6: Follow-up

Activities after the evaluation Allocation of resources Creation of a policy Technical assistance to the

surveillance

How to describe the evaluation results

1. Scoring system 1 = Must be improved 2 = Poor 3 = Fair 4 = Good 5 = Excellent

2. Descriptive results: บรรยาย 3. Combined system: Scoring and text

at some points

Goal of evaluation To improve and support a

health surveillance