NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS 1 NATIONAL ASSOCIATION FOR STATE...
-
Upload
darrell-quinn -
Category
Documents
-
view
216 -
download
1
Transcript of NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS 1 NATIONAL ASSOCIATION FOR STATE...
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
1
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
(NASCSP)
2011 NASCSP WAP Orientation Training
State Monitoring
March 1, 2011
Bob ScottDirector, Energy Services
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
2
Grant Guidance Requirements• State must conduct comprehensive monitoring of
each subgrantee at least once a year
• DOE recommends at least quarterly visits for monitoring and oversight of subgrantee’s work
• A minimum of 5% of completed units must be inspected by the Grantee
• Higher % strongly encouraged
• Additional inspections if significant problems are found
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
3
Grant Guidance Requirements
• Grantees are encouraged to also inspect jobs “in progress”
• Safe work practices, LSW, and other relevant factors
• Can be very revealing
• Exemplary agency status suspended until after 3-31-2012
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
4
Grant Guidance Requirements• Subgrantee should be briefed on the observations
and findings generated by the visit.
• Within 30 days, State will prepare a written report on its findings and send it to the subgrantee for corrective action
• Noncompliance findings unresolved within 45 days should be reported to the PMC.
• Sensitive or significant noncompliance findings should be reported to the PMC immediately
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
5
Grant Guidance Requirements• Major findings should be tracked by the State to
final resolution.
• DOE recommends tracking to include: findings, including success stories, recommended corrective actions, deliverables, due dates, responsible parties, actions taken, and final resolution.
• State will summarize and review each subgrantee's audit, program assessment reports and findings. Results should be considered during annual planning and available for review during DOE visits.
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
6
Monitoring Approach• Part of State Plan On-File Section
• Overview of procedures and guidelines for Grantee monitoring of local agencies
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
7
Monitoring Procedures Overview
• Goals of Monitoring
• Guiding Principles
• Desktop Reviews
• Subgrantee Monitoring Visits
• Monitoring Procedures
• Local Agency Monitoring Responsibilities
• Agency Discipline/At-Risk Policy
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
8
Goals of Monitoring• Analyze whether best possible program
services are being delivered to low-income population
• Determine program compliance and accountability
• Analyze program performance
• Analyze quality of work
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
9
Goals of Monitoring• Identify problems, deficiencies, and areas
that need improvement and advise on how to correct
• Assist agencies in their program operations and compliance with DOE and State regulations
• Assess T&TA needs
• Be a major tool for program improvement
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
10
Guiding Principles• Intended to be constructive process,
conducted with consistency, fairness, respect, and timeliness
• Foster positive, open, and constructive working relationships
• Two-way process to promote interaction, feedback, and improvements to Grantee and Subgrantee
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
11
Guiding Principles• Understanding that Grantee and Subgrantee
share the same goals of program effectiveness and improvement
• Promote improvement by providing technical assistance, reinforcing strengths, and sharing successes, innovations, and practices seen at other agencies
• Reports will be based on established policies, procedures and standards
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
12
Types of Monitoring
•Desktop Monitoring
•Subgrantee Visits
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
13
Desktop Monitoring• Not an agency visit• Based on reported information to the State by
subgrantees• Can save time of actual subgrantee visits by
preparing and organizing information in advance• Performance analysis• Review of expenditures• Certain compliance issues• Identify specific focus areas for agency visit,
including problematic jobs
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
14
Subgrantee Visits• Fiscal Monitoring
• Program Specific Administrative and Management Review
• Field Procedures Monitoring and Job Inspections
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
15
Fiscal Monitoring• Often part of overall agency fiscal
monitoring conducted by specialized Grantee fiscal monitor
• ARRA has necessitated more focused WAP Fiscal monitoring
• Sometimes a distinction between fiscal/procurement monitoring and specific WAP issues like job costing and reporting
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
16
Fiscal MonitoringKey issues identified during ARRA
•Procurement
•Invoicing techniques
•Accurate job costing
•Proper designation of expenses between Admin. and Program Operations
•Property records and inventory control
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
17
Administrative and Management Review
• Review production, expenditures, and related compliance issues with agency
• Review Client Files
• Review client eligibility and income verification documentation
• Review material, tool, equipment, and vehicle property and inventory records
• Review client flow charts and scheduling
• Review safety policies and practices
• Review insurance policies
• Inspect warehouse
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
18
Field Procedures Monitoring and Job Inspections
• Analysis of quality of field practices, including Compliance with audit protocols and work orders Diagnostics Overall effectiveness Compliance with State Field Standards Workmanship Appearance Thoroughness Health and Safety issues
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
19
Field Inspections
• Will probably have several more field inspection visits that fiscal and administrative visits ARRA production levels will necessitate Frequent visits recommended
• Should be a random selection of jobs within reason
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
20
Field Inspections – Best Practice
• Focus on agency audits, work orders, work done, and the agency final inspection
• If any of these are not consistently done well, the agency will likely have problems with call backs
• Grantee monitoring should analyze effectiveness of subgrantee inspection process
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
21
Monitoring Procedures• Agency visits are typically scheduled in
advance by notification from Grantee Representative
• Upon arrival, Monitor(s) should conduct entrance interview
• Local WAP Coordinator should be available at all times during visit
• Exit conference at conclusion of visit
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
22
Monitoring Procedures• Report should be sent to subgrantee
within 30 days (preferably sooner)
• Agency has to submit corrective actions, if necessary, within specified time frame
• Grantee reviews agency response
• Grantee writes agency to note acceptance of response or any unresolved issues
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
23
Best Practices - Approach• Be constructive – conduct yourself
professionally, be fair, treat agency personnel with respect, be on time
• Never monitor with “I gotcha” intentions
• Always act as a partner and team player – after all, you and the agency should share the same goal of program improvement
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
24
Best Practices - Approach• Help find solutions to problems
• Understand the agency and local program environment and set realistic expectations
• Offer constructive criticism based on your knowledge, experience, and the fact that you have observed other agencies that have dealt with similar circumstances
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
25
Best Practices - Approach
• States need to tell subgrantee in advance when you are coming, what you will be doing, and what documentation will be needed
• Don’t be afraid to pat someone on the back, they are in the Weatherization Program and probably need and deserve it
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
26
Best Practices - Reports• The report is the written final word – make sure
monitors have time to carefully write report
• Remember that some of the readers of the monitoring report may not have the same expertise as the monitor
• Make sure monitoring reports are clear and objective, and major findings are listed first and most prominently
• Findings should include specific references to Weatherization rules and standards
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
27
Best Practices - Reports
• Either report or cover letter must make clear corrective actions needed and possible consequences
• State Program Managers must insure their monitors are thorough and correct and then must back the reports
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
28
Best Practices - Reports• Remember – monitoring reports are
often the only assessment by State an agency receives
• Should be a thorough assessment that also recognizes good practices, not just deficiencies
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
29
Best Practices - Reports
• Monitor(s) should draft reports (or at least an outline of findings) and conduct exit interview so agency is not taken off guard by report
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
30
Best Practices - Reports• Decide on terms and have a clear distinction for
each
Common termsFinding – an issue that may be a violation of rules or standards and needs to be addressed
Concern – an issue that may not be in violation of rules or standards but needs to be addressed
Recommendation – notation based on monitor’s experience and expertise on how to improve certain aspect of the program
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
31
Best Practices - Reports• Sometimes a thin line between a finding
and a concern
• Finding – more significant, such as misuse of funds, eligibility issue, major health and safety issue, consistently failing to follow rules or standards
• Concern – Smaller issue, such as minor file omissions, or field observation that doesn’t significantly impact overall results but needs to be addressed
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
32
Other Best Practices• State needs to have clear, printed up-to-
date administrative and field standards to base monitoring
• Make monitoring flow between visits – do follow-up of last visit
• Document progress from last visit
• Confirm whether cited deficiencies were corrected
• Note areas of improvement
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
33
Other Best Practices• Do frequent monitoring – agency tends
to be more responsive to findings and suggestions
• Consider team monitoring – can cover a lot more ground and can capitalize on individual strengths
• Focus on issues – not peopleo Be tough on issues, but (where possible) sensitive to
people
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
34
Other Best Practices• Use digital cameras to document and
illustrate field work
o Make digital pics part of the exit conference and report
• Talk to clients – assess their satisfaction, ask if they can tell difference in comfort and/or fuel bills, but be careful about discussing findings with them
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
35
Other Best Practices• Make monitoring part of the Training
Program and use it as an opportunity to provide T&TAo Include agency field staff in field monitoring
o Consider asking agency to select their best jobs and their most problematic ones and visit with them
o Have them set up blower door, do diagnostic tests, heating system tests, etc.
Analyze their expertise and practices
See if equipment is calibrated and maintained
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
36
Other Best Practices• Strive to be consistent between
monitoring staffo Insure your monitoring forms/tools are as objective
as possible
o Have monitors do “practice” monitoring together. Inspect same files individually and then compare notes. Go to same house, inspect individually and compare findings and discuss.
o Consider developing library of “common findings” so there is similar wording and explanations between monitors in reports
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
37
Other Best Practices• And perhaps most important – due to
DOE policy on call backs
Formulate specific criteria that spells out criteria for agency call backs Have open discussions with State Managers, Monitors, and local agency staff
NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
Contacts
Bob Scott
202-624-5867
38