Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.
-
Upload
kaitlyn-barnwell -
Category
Documents
-
view
218 -
download
1
Transcript of Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.
![Page 1: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/1.jpg)
Quantitative On-Site Evaluation of Energy Efficiency Programs
Jackie Berger
August 16, 2010
![Page 2: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/2.jpg)
MotivationQuotes from summary of on-site visit:
“The core finding was that the auditor’s and crew’s engagement with the client and with one another increased as a function of their familiarity with their tools and their tasks. To some degree there was more engagement within the crew when the logistical challenges were greater, causing crew members to turn to one another to work out solutions.”
2
![Page 3: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/3.jpg)
MotivationQuotes from summary of on-site visit:
“The agency routinely uses good-quality silicone caulk for air sealing interior and exterior cracks, particularly around windows. While this is probably satisfactory, an easier-to-apply caulk with a long lifetime and substantially lower cost is in widespread use in weatherization and other retrofit work. It’s an acrylic latex with silicone. After dispensing (using the pull or push method), it can be easily worked with a wet sponge, rag, or finger and has a lifetime rated at 35 to 50 years…
3
![Page 4: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/4.jpg)
Motivation
Quotes from summary of on-site visit:
…It is available in a number of standard colors including white and clear. (The clear comes out of the tube white and gradually turns clear during curing, a process that takes at most a day unless humidity levels are particularly high.) It cleans easily with soap and water and costs about $2.25 per tube in case quantities at Home Depot and other home supply stores...
4
![Page 5: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/5.jpg)
Motivation
Quotes from summary of on-site visit:
…Finally, since installing air sealing measure on the inside of the conditioned envelope is usually more effective in limiting convective losses than is installation on the exterior, use of an acrylic latex product with silicone usually results in a more aesthetically appealing result than does pure silicone. When fully cured, it also takes paint well (Figures 22 and 23).”
5
![Page 6: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/6.jpg)
Motivation
6
Figures 22 and 23. All caulks have their place and matching the right one to the right job is part of the weatherization craft.
![Page 7: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/7.jpg)
Motivation
• Goals for on-site observation and inspections– Collect the targeted information.– Summarize findings over all visits.– Understand how findings relate to the program
overall.– Make recommendations for program improvement.
7
![Page 8: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/8.jpg)
Session Outline
• Introduction
• Process Evaluation Research
• On-Site Evaluation
• Sample Selection
• Data Collection Instruments
• Findings from On-Site Evaluation
• Summary
8
![Page 9: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/9.jpg)
INTRODUCTION
9
![Page 10: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/10.jpg)
INTRODUCTION• Impact evaluation
– How much energy is saved?– Is the program cost-effective?– Which measures are cost-effective?
• Process evaluation – Why succeeding or not meeting goals?– How can the program be improved?
• Documenting service delivery– Anecdotal – cannot be generalized.– Quantitative – how pervasive are the issues.
10
![Page 11: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/11.jpg)
PROCESS EVALUATION RESEARCH
11
![Page 12: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/12.jpg)
Research Tasks
• Background research purpose– Understand program policies and procedures.– Identify potential improvements.
• Background research activities– Review program documentation.– Conduct in-depth interviews with program
designers and managers.– Review program performance statistics.
12
![Page 13: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/13.jpg)
Research Tasks
• Contractor survey purpose– Understanding of program
– Program implementation issues
– Barriers with program procedures
• Contractor survey activities– Review delivery statistics.
– Review contractor characteristics.
– Select sample.
– Conduct survey and analyze data.13
![Page 14: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/14.jpg)
Research Tasks
• Client survey purpose– Program experiences– Energy education delivered– Energy education impact– Safety and comfort impact– Satisfaction
14
![Page 15: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/15.jpg)
Research Tasks
• Client survey activities– Identify key segments.– Select sample.– Conduct survey and analyze data.
15
![Page 16: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/16.jpg)
ON-SITE EVALUATION
16
![Page 17: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/17.jpg)
On-Site Importance
• Were protocols followed?
• How well did they work?
• Were important savings opportunities missed?
17
![Page 18: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/18.jpg)
On-Site Importance
• Protocols – provider compliance– Program information, diagnostic tests, measure
installation, client education.
• Protocols – applicability– How well protocols work, consistent application.
• Use of equipment– Tools available, adequately maintained and used
correctly.
18
![Page 19: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/19.jpg)
On-Site Importance
• Provider adaptability– Protocols properly adapted to individual
circumstances.
• Comprehensiveness– Extent to which cost-effective procedures are
addressed, potential modification of procedures.
• Quality of Work– Conformance to program standards, respect for client
home, safety.
19
![Page 20: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/20.jpg)
On-Site Importance
• Client Education– Focus on high potential areas, ability to
motivate clients.
• Client Interaction– Explanation of program and work, use of client
feedback as information input.
20
![Page 21: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/21.jpg)
SAMPLE SELECTION
21
![Page 22: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/22.jpg)
Sample Selection Challenges• Limited sample size.
• Many contractor/job/home/client characteristics to consider.
• Difficult to draw inferences about program implementation.
22
![Page 23: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/23.jpg)
Sample Selection Methods• Probability sampling
– Random selection techniques are used.– Each job has a known probability of selection.– Results can be weighted to represent the program.– Confidence intervals can be developed.
• Purposive sampling– Random selection techniques are not used.– But can allow for assessment of program performance.
23
![Page 24: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/24.jpg)
Sample Selection Utility Program Evaluation• 5 service delivery areas, as specified by utility.
• Baseload jobs – customers without electric heat or hot water.
• Full cost jobs – customers with installed electric heat and >3,600 kwh seasonal heating or cooling usage.
• 16 contractors provide services.
24
![Page 25: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/25.jpg)
Sample Selection Utility Program Evaluation• One dominant contractor in each service area
for each job type.
• Budget only allowed for observation/inspection of one contractor in each service area for each job type.
• Dominant contractor for each area was chosen.
25
![Page 26: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/26.jpg)
Sample Selection Utility Program Evaluation
26
Region Contractor# Baseload
# Full Cost
Region Contractor# Baseload
# Full Cost
1 A 6 272 3 I 135 1021 B 0 23 3/4 J 46 1281 C 1 107 3/4 K 153 12 D 81 115 4 L 38 272 E 36 58 4 M 2 1062 F 44 102 4 N 63 02 G 53 0 5 O 4 4122 H 28 46 5 P 57 0
Previous Year Service Delivery
![Page 27: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/27.jpg)
Sample Selection Utility Program Evaluation• Selected contractors represent 60 percent of
Baseload jobs and 68 percent of Full Cost jobs.
• Contractors use only 1-3 auditors.
• Observed auditors do 30 to 40 percent of the audit work.
• Findings represent significant percentage of job experiences.
27
![Page 28: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/28.jpg)
Sample Selection National WAP Evaluation• Select 20 local agencies in 20 different states
28
Stratification Factors
Climate Region
Crews or Contractors
Urban/Rural/Suburban
Agency Size
Client Education
Community Action Agency /Other Type
Baseload Measures
TrainingDOE Funding/ Substantial
Other Resources
QADominant
FuelComputerized
Audit/Priority List
![Page 29: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/29.jpg)
PROCEDURES AND FORMS
29
![Page 30: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/30.jpg)
Procedures and FormsDevelopment• Understand
– Program goals– Program design and implementation– Roles/responsibilities of providers
• Source of problems– Protocols– Division of responsibilities– Implementation
30
![Page 31: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/31.jpg)
ProceduresUtility Program Evaluation• Baseload Observations, Full Cost
Observations, Full Cost Inspections– Procedures: how to conduct visit, forms to
complete, visit write-up, forms to collect from contractor.
– Data Collection Form: home, electric usage, services provided.
– Potential Big Users: lights and appliances that could lead to high baseload usage.
31
![Page 32: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/32.jpg)
Baseload ProceduresUtility Program Evaluation
1. During visit– Record auditor work and customer interactions.– Every 10 minutes record time and auditor actions. – Record if auditor addresses the Potential Big Users.
2. During/after visit– Complete Baseload Observation Form. – Complete Potential Big Users Form.
3. Immediately following visit– Debrief customer and auditor. 32
![Page 33: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/33.jpg)
Baseload ProceduresUtility Program Evaluation
4. Evening or next day– Complete customer debriefing.
5. Narrative– Chronologically document audit.– Estimate length of key parts of audit.– Assess interaction between auditor and client.
33
![Page 34: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/34.jpg)
Baseload ProceduresUtility Program Evaluation
6. Forms – obtain copies of all forms used during the visit.– Customer usage history– Program application– Core assessment form– Refrigerator data form– Water heater checklist– Customer profile– “Your Electric Bill” form
34
![Page 35: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/35.jpg)
FormsUtility Program Evaluation• Selected parts of baseload observation form
35
VISIT INTRODUCTION
1. Did the customer expect the visit? YES NO
2. Did the auditor describe the program to the customer? YES NO
3. Did the auditor review and explain the electric bill? YES NO
4. Did the auditor discuss whether there were any health and safety issues?
YES NO
5. Did the auditor discuss whether there were any comfort issues?
YES NO
6. Did the auditor discuss whether there were any problems with energy usage?
YES NO
![Page 36: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/36.jpg)
FormsUtility Program Evaluation• Selected parts of baseload observation form
36
PRIORITY LIST – AIR CONDITIONERS
1. Did the auditor inspect all window air conditioners? YES NO
2. Did the auditor do sealing around the air conditioner? YES NO
3. Did the auditor talk about seasonal storage? YES NO
4. Did the auditor determine that the air conditioner had EER of <6 or was in poor condition?
YES NO
5. If yes, did the auditor offer to replace air conditioner? YES NO
6. If yes, did customer accept AC replacement? YES NO
7. If no, why did the customer refuse
![Page 37: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/37.jpg)
FormsUtility Program Evaluation• Selected parts of full cost inspection form
37
THERMOSTATS (FILLED OUT FOR EACH ONE)
Location Program Temp
System controlled Day Heat
Programmable Night Heat
Condition Day Cool
Accuracy Night CoolExceptional (Ex): No improvements identified Good (Gd): Minor improvements possible Satisfactory (Sat): Some improvements recommendedFair (Fr): Extensive improvements necessaryPoor (Pr): Substandard in all respects
Same as recorded
Effort_________ Quality _______ Appropriateness ___________
![Page 38: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/38.jpg)
ProceduresWAP Evaluation• Observe audit, measure installation, and final
inspection.
• Observe job from start to finish in one home.
• Conceptual home – observe all aspects.
38
![Page 39: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/39.jpg)
ChallengesWAP Evaluation• Observe up to 480 homes from 20 providers in 20
different states.• WAP implemented differently in every state.
– Audit procedures– Eligible measures– Type of education provided
• Develop general forms and procedures that collect detailed quantitative data.
• Assess work based on whose standards?
39
![Page 40: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/40.jpg)
FormsWAP Evaluation• Audit observation form
• Audit write-up assessment form
• Measure installation observation form
• Final inspection observation form
• Client debriefing
• Weatherization staff/contractor debriefing
40
![Page 41: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/41.jpg)
Forms - WAP Evaluation• Selected parts of audit observation form
41
Combustion Safety Tests
Test done? In agency standards?
Should have been done?
Performed correctly to obtain needed info?
Ambient CO in CAZ
Y N Y N Y N Y N
Ambient CO outside CAZ
Y N Y N Y N Y N
Gas/propane/ fuel oil leaks
Y N Y N Y N Y N
Spillage testing
Y N Y N Y N Y N
Heating system CO
Y N Y N Y N Y N
Water heating CO
Y N Y N Y N Y N
![Page 42: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/42.jpg)
Forms - WAP Evaluation• Selected parts of audit observation form
42
Combustion Safety Tests
Test done? In agency standards?
Should have been done?
Performed correctly to obtain needed info?
Heating system draft
Y N Y N Y N Y N
Water heating draft
Y N Y N Y N Y N
CAZ – combustion air
Y N Y N Y N Y N
Range CO Y N Y N Y N Y N
Stove CO Y N Y N Y N Y N
Addressed immediate safety issues
Y N NA
![Page 43: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/43.jpg)
Forms - WAP Evaluation• Selected parts of measure installation observation form
43
Air SealingMark NA if not in work scope and skip section
Mark NA if not done this visit and skip section
Blower door used to guide air sealing Y N NA
Sealing at top and bottom of envelope prioritized Y N NA
In Work Scope
Done This Visit
Rate Quality
Attic floor penetration sealed Y N Y N 1 2 3 4 5
Knee walls sealed Y N Y N 1 2 3 4 5
Second floor rim joist sealed Y N Y N 1 2 3 4 5
![Page 44: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/44.jpg)
Forms - WAP Evaluation• Selected parts of measure installation observation form
44
Air SealingIn Work
ScopeDone This
VisitRate
Quality
Basement penetrations sealed
To outside Y N Y N 1 2 3 4 5
To conditioned space Y N Y N 1 2 3 4 5
Crawlspace penetrations sealed
To outside Y N Y N 1 2 3 4 5
To conditioned space Y N Y N 1 2 3 4 5
Garage penetrations to conditioned space sealed
Y N Y N 1 2 3 4 5
![Page 45: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/45.jpg)
Forms - WAP Evaluation• Selected parts of measure installation observation form
45
Air SealingAll major opportunities sealed Y N
Lowest Highest
Overall quality of air sealing work 1 2 3 4 5
Overall cleanliness of air sealing work 1 2 3 4 5
Comments
5 – Exceptional: No possible improvements identified4 – Good: Minor improvements possible3 – Satisfactory: Some improvements recommended2 – Fair: Extensive improvements necessary1 – Poor: Substandard in all respects
![Page 46: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/46.jpg)
FINDINGS FROM ON-SITE EVALUATION
46
![Page 47: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/47.jpg)
Utility Program Visit Introduction
47
# of ObservationsYes No
Customer expected visit 10 0Explained Program 6 4Reviewed electric usage 4 6Discussed health and safety issues 4 6Discussed comfort issues 3 7Discussed any problems with energy usage
3 7
![Page 48: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/48.jpg)
Utility Program Baseload Visit Summary
48
Length of Visit
Minimum Maximum Average
Visit length (minutes)
70 180 119
RatingExcellent Good Fair Not Acceptable
Overall rating (#of jobs)
4 2 2 2
![Page 49: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/49.jpg)
Utility ProgramBaseload Recommendations
• Review requirements and expectations with contractors.
• Reinforce importance of the walkthrough for baseload jobs with all contractors.
• Provide additional training to contractors on the importance of 2-for-1 swaps in refrigerator replacement, and train contractors to work with customers to obtain their acceptance of this measure.
49
![Page 50: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/50.jpg)
Utility ProgramBaseload Recommendations
• Review CFL replacement procedures with contractors.
• Review customer education requirements with contractors.
• Observe baseload service delivery to ensure that contractors meet program standards for service delivery.
50
![Page 51: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/51.jpg)
Utility Program Inspections Overview
51
# of Observations
Very Good Good FairData collection accuracy 1 1 2Measure selection 1 3Measure appropriateness 2 2
Exceptional Good SatisfactoryEffort 3 1Quality 1 2 1Appropriateness 3 1Overall Rating 1 3
![Page 52: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/52.jpg)
Utility ProgramFull Cost Recommendations
• Develop one set of required forms.
• Provide instructions on the back of each form.
• Require all applicable diagnostic tests at the audit visit.
52
![Page 53: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/53.jpg)
SUMMARY
53
![Page 54: Quantitative On-Site Evaluation of Energy Efficiency Programs Jackie Berger August 16, 2010.](https://reader035.fdocuments.net/reader035/viewer/2022070306/55166117550346a2698b5198/html5/thumbnails/54.jpg)
Recommendations• On-site work is an important component of
technical process evaluations.
• Select sample that accurately represents program.
• Design tools to capture what is observed in the field.
• Train staff to systematically record data and information.
• Analyze findings in conjunction with other research.
54