Post on 29-Dec-2015
REVĪZIJAS IESTĀDES JAUNĀ LOMA – REZULTĀTU UN IZPILDES RĀDĪTĀJU AUDITSOlga Guza, vecākā eksperte, ES fondu revīzijas departaments, Finanšu ministrija
Content of the presentation
• Output and result indicators
• Goals at different stages and performance reserve
• Auditing approaches and requirements
• Available data, IT system, reporting and information from beneficiaries
• Timetable and extent of auditing
Output and result indicators
• Indicators and corresponding targets established in order to assess progress in programme implementation aimed at achievement of objectives as the basis for:
– monitoring
– evaluation
– review of performance
• Those indicators include:
– output indicators relating to the operations supported
– result indicators relating to the priority concerned
• Defined in the programme
Indicators in the programme
In Latvia:
1 OP
12 priorities
33 investment priorities
72 objectives
Structure of indicators
121 result indicators in OP of Latvia
135 output indicators in OP of Latvia
Example
Performance review
• EC in cooperation with MS, shall undertake a review of the performance of the programmes in each MS in 2019 (the 'performance review')
– with a view to monitoring progress towards the objectives and targets set for each priority over the course of the 2014 - 2020 programming period
– to ensure that the budget of the Union is not used in a wasteful or inefficient way
• Performance review will be carried out with reference to the performance framework set out in the respective programme
Performance framework
• Performance framework consists of up to four different types of indicators: financial, output and result indicators and key implementation steps
121 → 0
135 → 31
Performance framework in the OP
Performance framework
Performance framework
• Progress is reviewed twice during the programming period against the milestones and targets set in the programme.
Goals at different stages
2018:
•Milestones are intermediate targets, directly linked to the achievement of the specific objective of a priority
•The milestones are intermediate targets set for indicators to be achieved by 31 December 2018 and to be assessed in 2019
2023:
•Targets
•The targets themselves are set to be achieved by 31 December 2023 and their accomplishment will be assessed at the closure of the programme period in 2025
Performance reserve
• 6 % of ERDF, ESF and Cohesion Fund under the Investment for Growth and Jobs goal, EAFRD and to measures financed under shared management in accordance with the EMFF Regulation
• Established in the Partnership Agreement and programmes
• Established in order to facilitate the focus on performance and attainment of the objectives of the Union strategy for smart, sustainable and inclusive growth
Allocation of Performance reserve
Consequences of poor performance (after Performance review)
At the end of 2018:If 2 indicators, any of indicators < 65% of milestone valueIf > 2 indicators, any 2 of indicators < 65% of milestone value
Consequences of poor performance (after EC examination of the final implementation report)
At the end of 2023:If 2 indicators, any of indicators < 65% of target value If > 2 indicators, any 2 of indicators < 65% of target value
Auditing approaches and requirements – Designation procedure
• 2 ways of carrying out audit work:
– Full audit work in accordance with EC guidance and checklist
– But if management and control system is essentially the same as for the 2007-13 programming period + audit evidence of its effective functioning during that period is present → conclusion that the relevant criteria are fulfilled without carrying out additional audit work
• In both cases - for the new criteria (including procedures to ensure reliability of data on indicators/milestones/progress of the OP in achieving its objectives), audit work will have to be performed in order to assess the compliance in these areas
During designation procedure
• Check whether following procedures are in place and address following issues?
Auditing approaches and requirements – system audits and audits of operations
• Wording of the Opinion:
–«In my opinion, and based on the audit work performed … the management and control system put in place function properly»
–«The audit work carried out does not put in doubt the assertions made in the management declaration»
• Wording of the Management declaration – «Furthermore, I/we confirm the reliability of data relating to indicators, milestones and the progress of the operational programme required under article 125(2)(a) of Regulation (EU) No 1303/2013»
System audits
• The audit scope include verification that the relevant authorities properly ensure reliability of data relating to:
– output indicators
– milestones (financial and output indicators, key implementation steps)
– progress of the programme in achieving its objectives
• Audit Authority should assess:
– whether effective controls are implemented over collecting, summarizing and reporting the related data
– whether the reported compiled figures reconcile with the source data
Relevant key requirements – Key requirement 2
• AC 2.1:The MA drew up, for approval by the Monitoring Committee, appropriate selection procedures and criteria that ensure the contribution of operations to the achievement of the specific objectives and results of the relevant priority
• AC 2.4: All applications/projects are evaluated in accordance with the applicable criteria
No Main risks (examples) Expected controls Testing procedures1. Approved selection procedures
and criteria are not appropriateAll selection procedures and criteria prepared by IB are reconciled by MA All selection procedures and criteria are approved by the Monitoring Committee
Control test for process of preparation and approval of selection procedures and criteria
2. Approved selection procedures and criteria are changed during the ongoing selection process resulting in non equal treatment of applicants
3. Decisions taken on the acceptance or rejection of applications are not adequate resulting in ineligible expenditure
Applications are evaluated using the selection criteria /scoring approved by the Monitoring CommitteeEvaluators of the applications possess the required expertise and independence
Control test for a sample of applications
Substantive testing (re-performance) for a sample of applications (take both approved and rejected applications)
Review submitted appeals
Relevant key requirements – Key requirement 4
• AC 4.1: The management verifications include Administrative verifications and On-the-spot verifications of operations
• AC 4.3: Written procedures and comprehensive checklists should exist to be used for the management verifications in order to detect any material misstatements
No Main risks (examples) Expected controlsTesting
procedures1. Risk of suspension of
payments if management verifications do not timely reveal a serious failure in achieving milestones / targets relating to financial and output indicators
Beneficiaries' application for reimbursement templates are adjusted in order to enable for timely and correct reporting on indicatorsAdministrative verifications include verification on:•application for reimbursement includes check on the progress in the attainment of indicators•final application for reimbursement includes check on information on the actual contribution to the output and results indicators provided by the beneficiaryOn-the-spot verifications include verification on:•correctness of the data communicated by the beneficiaries in relation to the indicators•correct understanding of the indicator by the beneficiary and the values reported •correctness of inputting information on indicators into the IT system by the beneficiary
Review of the checklists
Control test for a sample of administrative and on-the-spot verifications performed
Substantive testing for a sample of administrative and on-the-spot verifications performed
2. Written procedures and checklists for verifications are not detailed enough in order to ensure quality and reliability of the data on indicators
Relevant key requirements – Key requirement 5
• AC 5.1: The detailed supporting documents for operations are kept at the appropriate management level (including progress in achieving outputs and results and monitoring reports)
• The audit trail shall allow data in relation to output indicators for the operation to be reconciled with targets and reported data and result for the programme
No Main risks (examples) Expected controls Testing procedures
1. The data, aggregated related to indicators and target values at investment priority, priority or programme level is not timely, complete and reliable
Reconciliation performed for the data submitted to the monitoring committee and EC (AIR)
Re-performance of reconciliation
Relevant key requirements – Key requirement 6
• AC 6.1: The existence of IT system capable to collect, record and store data on each operation, including data relating to indicators and milestones and on the progress of the programme in achieving its objectives
• AC 6.2: Adequate procedures are in place to allow for the aggregation of the data where this is necessary for the purposes of evaluation, audits, as well as for payment applications and accounts, annual summaries, annual implementation and final reports, including reports on financial data, submitted to the Commission
Thematic audits
• System audits relating to key requirements targeted to specific thematic areas should be indicated in the Audit Strategy, which may include:
–Thematic audit on the reliability of data relating to indicators and milestones and on the progress of the operational programme in achieving its objectives provided by the managing authority to the monitoring committee
–Thematic audit on the functioning and security of IT systems
Audits of operations
• The audits of operations should cover:
–that for expenditure declared to EC outputs and results underpinning payments to the beneficiary have been delivered
–participant data or other records related to outputs and results are consistent with the information submitted to EC
–required supporting documentation demonstrates an adequate audit trail
Available data
• Ex-ante evaluation
• IT system
• Information from beneficiaries
• Reports at National level (including information submitted by the managing authority to the monitoring committee)
• Annual Implementation Reports
Ex-ante evaluation
• Information on appraisal of:
–the relevance and clarity of the proposed programme indicators
–how the expected outputs will contribute to results
–whether the quantified target values for indicators are realistic, having regard to the support from the Funds
–the suitability of the procedures for monitoring the programme and for collecting the data necessary to carry out evaluations
–the suitability of the milestones selected for the performance framework
IT system
Data on indicators:
•common and programme specific output indicators (including measurement unit, target value, achievement level for each calendar year)
•common and programme specific result indicators (including measurement unit, baseline value, target value, achievement level for each calendar year)
Member States shall ensure that no later than 31 December 2015, all exchanges of information between beneficiaries and authorities can be carried out by means of electronic data exchange systems
National reports (Latvia’s example)
MAResponsible Bodies
(10 entities)Cooperation Body
(CFCA)
Monitors achievement of output and result indicators
at the level of specific objectives
Monitors achievement of output and result indicators
at the level of projects
Carries out administrative and on-the-spot checks
Participate in on-the-spot checks
Enters data to MISEnters data to MIS on result indicators
MIS
Reports to MA Reports to MA
Reports
Provides info to Monitoring Committee
Prepares annual plan on achievement of outputs and results
(milestones and targets)
Prepares AIR
Annual Implementation Reports
Extent of auditing
ISA 330
•Controls that have not changed from previous audits:
The auditor’s can make decision on whether to rely on audit evidence obtained in previous audits for controls that:
(a) have not changed since they were last tested; and
(b) are not controls that mitigate a significant risk,
But the auditor shall test the controls at least once in every third year, and shall test some controls each year to avoid the possibility of testing all the controls on which the auditor intends to rely in a single audit period with no testing of controls in the subsequent two audit periods
Timetable of auditing – multiannual (example)
Timetable of auditing – one year
Conclusions
• Auditing approach should follow a common audit methodology for the Funds
• Key focus on data related with financial and output indicators
• Include specific information in the Audit Strategy
• Audit during:
– designation process
– system audits (including thematic audits, IT audits)
– audit of operations
• Part of the audit opinion of the audit authorities
Thank you for your attention!
Olga Guza
(+371) 6 708 3864
olga.guza@fm.gov.lv