Health andConsumers

NAS Network feedback on reference document

"Objectives and Indicators"

MANCP Network meetingFVO, Grange

25-26 November 2014

Health andConsumers

In plenary, members analysed the document "Objectives and Indicators", Version 1, Revision 5 of 15/07/2014Replied to three questions:1.Is the document useful for NAS auditors?2.Is the approach followed for the structure of the document adequate?3.What can be improved?

NAS Network meeting of 24-25/09/2014

Health andConsumers

• Auditors don’t write the MANCP but if people who write the MANCP follow the reference document then it will be more easily audited.

• Yes, it is useful but if it is to be used in an audit context it needs an annex with specific examples (e.g. validation of indicators).

• Provides important background information on “objectives and indicators” in the MANCP context.

Is the document useful for NAS auditors?

Health andConsumers

• Helps an auditor understand the auditees’ intention and direction of their control systems and how they define objectives and indicators (common terminology), but is not a tool for audit performance.

• Encourages MS to have objectives and indicators and is a good reference for CAs that have to design systems.

• Not relevant for audits directly but allows understanding of how indicators are developed/created.

Is the document useful for NAS auditors?

Health andConsumers

• Outcome of the use of the reference document is useful for the auditors (useful for improving the MANCP).

• More useful for management level, but still useful for auditors as a background.

• It needs to be completed with national audit instructions and translated into MS languages.

Is the document useful for NAS auditors?

Health andConsumers

• In general yes. The parts “purpose and use” and “characteristics” (although the acronym RACER was new for some participants) are useful. The part “principles” is confusing.

• Does the scope include “fraud” too? (Example on Annex 1.2).

• More effective to have objectives and indicators together in one reference document.

Is the approach followed for the structure of the document adequate?

Health andConsumers

• Impact of indicators in terms of behaviour/risk management needs to be highlighted.

• Needs to be included in the wider context of the purpose of the MANCP.

• Structure and logics are adequate but the diagrams need more explanations (for instance diagram 2 and its legend).

Is the approach followed for the structure of the document adequate?

Health andConsumers

• Strategic objectives cannot be SMART. On the other hand, operational objectives are easier to fit in the SMART characteristics. A hierarchy of objectives should be defined, where indicators are linked specifically with operational objectives.

• Part “Types and Examples” could be clearer and include more examples.

• A procedure for acceptance of objectives and indicators is missing (top-down, down-up).

• Better examples linked with diagram 2.

What can be improved?

Health andConsumers

• More diverse examples (quantitative and qualitative indicators).

• In the definition of indicators change “tool” to “feature”.

• Title of the document doesn’t reflect content.• Reference to “Effectiveness” reference document –

link the two documents.• Examples in annexes could be further developed

(bearing in mind that examples for each sector could undermine a final agreement on the reference document).

What can be improved?

Health andConsumers

• Keep the categorisation of indicators in the definitions - “output”, “outcome” and “impact” indicators.

• Clarify the part “principles”.• Include a methodology to build a good indicator (use

a diagram?)• Change title of the document – “Principles for

objectives and indicators and good practice”.• Add examples of key indicators.• Prescriptive indicators – list of indicators that could be

used by all MS?

What can be improved?

Health andConsumers

Thank you!

Unit F7Operational and Administrative services

European CommissionFood and Veterinary Office

Grange, Dunsany, Co. Meath, Ireland