SWE-9-23 Kick-Off Mtg Presentation.ppt · 9. Verification and due diligence procedures for site...

Part 1Introduction Review of

Part 1Introduction Review ofIntroduction, Review of

Agenda, Agenda RevisionsIntroduction, Review of

Agenda, Agenda Revisions

2

Part 2Review of ACT 129 and Other

Part 2Review of ACT 129 and OtherReview of ACT 129 and Other

Historical DocumentsReview of ACT 129 and Other

Historical Documents

3

Part 3Part 3Part 3Review of Objectives for the

Part 3Review of Objectives for the

Statewide Evaluation and Contents of the Audit PlanStatewide Evaluation and Contents of the Audit PlanContents of the Audit PlanContents of the Audit Plan

4

Act 129 MandateAct 129 MandateAct 129 MandateAct 129 Mandate

The Energy Efficiency and Conservation Program shall include: The Energy Efficiency and Conservation Program shall include:

(2) An evaluation process, including a process to monitor and verify data collection quality assurance and

(2) An evaluation process, including a process to monitor and verify data collection quality assurance andand verify data collection, quality assurance and results of each plan and the program. and verify data collection, quality assurance and results of each plan and the program.

(3) An analysis of the cost and benefit of each plan submitted under subsection (B) in accordance with a total resource cost test approved by the Commission

(3) An analysis of the cost and benefit of each plan submitted under subsection (B) in accordance with a total resource cost test approved by the Commissiontotal resource cost test approved by the Commission.total resource cost test approved by the Commission.

5Source: House Bill No. 2200, page 51

Objectives of Statewide l i

Objectives of Statewide l iEvaluationEvaluation

• Identify the cost-effectiveness of each plan in accordance with • Identify the cost-effectiveness of each plan in accordance with the Commission adopted Total Resource Cost Test Manual.

• Provide reasonable assurance that the claimed measures are being properly installed and utilized.

the Commission adopted Total Resource Cost Test Manual.• Provide reasonable assurance that the claimed measures are

being properly installed and utilized.• Provide reasonable assurance that the installed measures are

obtaining the claimed energy savings or reductions in accordance with the Commission adopted Technical Reference Manual or

• Provide reasonable assurance that the installed measures are obtaining the claimed energy savings or reductions in accordance with the Commission adopted Technical Reference Manual or metered savings.

• Identify areas for improvement.• Identify best practices that may be implemented on a

metered savings.• Identify areas for improvement.• Identify best practices that may be implemented on a y p y p

going-forward basis.y p y p

going-forward basis.

6

Requirements for Contents of the A dit Pl

Requirements for Contents of the A dit PlAudit PlanAudit Plan

1. High-level summary of audit plan. 2 EE&C measures to be audited and the plan measure logic/theory1. High-level summary of audit plan. 2 EE&C measures to be audited and the plan measure logic/theory2. EE&C measures to be audited and the plan measure logic/theory. 3. The type and format of documentation identifying EDC plan expenditures. 4. A description of the audit objectives, audit questions and audit rigor level for each EDC

plan. 5 A description of the methodologies procedures and data tracking systems to be used by

2. EE&C measures to be audited and the plan measure logic/theory. 3. The type and format of documentation identifying EDC plan expenditures. 4. A description of the audit objectives, audit questions and audit rigor level for each EDC

plan. 5 A description of the methodologies procedures and data tracking systems to be used by5. A description of the methodologies, procedures and data tracking systems to be used by

the Contractor to verify the impact evaluations and project verifications for each plan, including data gathering, sampling and analysis methods.

6. Audit approaches to be used for each plan in accordance with the TRM or other approved EM&V plans

5. A description of the methodologies, procedures and data tracking systems to be used by the Contractor to verify the impact evaluations and project verifications for each plan, including data gathering, sampling and analysis methods.

6. Audit approaches to be used for each plan in accordance with the TRM or other approved EM&V plansEM&V plans.

7. Audit approaches to be used for each plan in accordance with approved custom measures. 8. Strategies to conduct audit studies coordinated on a statewide basis, such that results can

be extrapolated back to individual EDC EE programs. 9 Verification and due diligence procedures for site inspections and validating information

EM&V plans. 7. Audit approaches to be used for each plan in accordance with approved custom measures. 8. Strategies to conduct audit studies coordinated on a statewide basis, such that results can

be extrapolated back to individual EDC EE programs. 9 Verification and due diligence procedures for site inspections and validating information9. Verification and due diligence procedures for site inspections and validating information

provided by EDCs. 10. Data and information needed from EDCs and other sources. 11. Budget, schedule and manpower plan for Contractor.

9. Verification and due diligence procedures for site inspections and validating information provided by EDCs.

10. Data and information needed from EDCs and other sources. 11. Budget, schedule and manpower plan for Contractor.

7Source: PA PUC RFP for Statewide Evaluator , page 31

Value of Information (VOI)Value of Information (VOI)• What is the value of

information and data?• What is the value of

information and data?• What is the cost? What is it

worth? What is the return?• VOI balance cost and rigor

• What is the cost? What is it worth? What is the return?

• VOI balance cost and rigor• Deterministic sampling• Reduction of uncertainty• Adjust sample size according to

the cost of changes compared to

• Deterministic sampling• Reduction of uncertainty• Adjust sample size according to

the cost of changes compared tothe cost of changes compared to changes in value

• Sampling plans guided by expectations

the cost of changes compared to changes in value

• Sampling plans guided by expectationse pectat o s

• Audits structured to test the validity of reported results –reasonable scientific checks

e pectat o s• Audits structured to test the

validity of reported results –reasonable scientific checks

8

Value of Information (VOI)Value of Information (VOI)( )( )

• Actual reported impacts may • Actual reported impacts may vary by program and EDC –VOI considerations employed when evaluation efforts may b l d i k

vary by program and EDC –VOI considerations employed when evaluation efforts may b l d i kbe large compared to risk, funding and impacts

• Frequent stratification

be large compared to risk, funding and impacts

• Frequent stratificationFrequent stratification scheme – weight the sample towards significant attributes

Frequent stratification scheme – weight the sample towards significant attributes

• Greater focus on impacts with greater uncertainty –sensitivity analysis

• Greater focus on impacts with greater uncertainty –sensitivity analysis

9

Timetable for Development of A dit Pl

Timetable for Development of A dit PlAudit PlanAudit Plan

• Draft – November 1, 2009• Draft – November 1, 2009,

• Final – December 2, 2009

,

• Final – December 2, 2009

10

Table of Contents of Audit PlanTable of Contents of Audit PlanTable of Contents of Audit PlanTable of Contents of Audit PlanSection 1 Introduction and Purpose of the Audit PlanSection 1 Introduction and Purpose of the Audit Plan

Section 2 Requirements for EDC Evaluation

Section 3 Audit Plan

Section 2 Requirements for EDC Evaluation

Section 3 Audit Plan

Section 4 EDC Reporting Guidelines

Section 5 EDC Data Requirements

Section 4 EDC Reporting Guidelines

Section 5 EDC Data Requirements

Section 6 Statewide Evaluator Reporting Activities

Section 7 Expected Deadlines

Section 6 Statewide Evaluator Reporting Activities

Section 7 Expected Deadlines pp

11

Part 4 Development of Overall

Part 4 Development of OverallDevelopment of Overall

Statewide Evaluation PlanDevelopment of Overall

Statewide Evaluation Plan

12

Topics to be DiscussedTopics to be DiscussedTopics to be DiscussedTopics to be Discussed

• Procedures for measurement and verification• Procedures for measurement and verification• Procedures for measurement and verification

• Procedures for impact evaluation

• Procedures for measurement and verification

• Procedures for impact evaluation

• Procedures for process evaluation• Procedures for process evaluation

• Data reporting requirements• Data reporting requirements

13

Overview of Evaluation Plans S b itt d b EDC

Overview of Evaluation Plans S b itt d b EDCSubmitted by EDCsSubmitted by EDCs

• The EDC submitted plans on July 15, 2009• The EDC submitted plans on July 15, 2009The EDC submitted plans on July 15, 2009• The PUC is in process of reviewing the plans• The EDC plans provide highlights of the EDC

The EDC submitted plans on July 15, 2009• The PUC is in process of reviewing the plans• The EDC plans provide highlights of the EDC• The EDC plans provide highlights of the EDC

evaluation approach• The EDC plans provide highlights of the EDC

evaluation approach

14

Important Aspects of l i l d i i i

Important Aspects of l i l d i i iEvaluation Plans and ActivitiesEvaluation Plans and Activities

• Verification• Verification• M&V Option Selection• Source of Stipulated Data• M&V Option Selection• Source of Stipulated Data• Baseline Definition• Monitoring Duration

d h

• Baseline Definition• Monitoring Duration

d h• Adjustments – Weather, Economic, etc.• Calibration Criteria• Additional Provisions

• Adjustments – Weather, Economic, etc.• Calibration Criteria• Additional Provisions• Additional Provisions• Additional Provisions

15

Review of Best PracticesReview of Best PracticesReview of Best PracticesReview of Best Practices

• National Action Plan for Energy Efficiency• National Action Plan for Energy EfficiencyNational Action Plan for Energy Efficiency • Statewide Impact Evaluations

– New York

National Action Plan for Energy Efficiency • Statewide Impact Evaluations

– New YorkNew York– Wisconsin– California

New York– Wisconsin– CaliforniaCalifornia

• Other Program Evaluations Metrics– Cost-effectiveness: Connecticut

California• Other Program Evaluations Metrics

– Cost-effectiveness: ConnecticutCost effectiveness: ConnecticutCost effectiveness: Connecticut

16

National Action Plan for Energy Effi i G l E l ti P

National Action Plan for Energy Effi i G l E l ti PEfficiency – General Evaluation ProcessEfficiency – General Evaluation Process

• Set evaluation objectives in the context of the program • Set evaluation objectives in the context of the program j p gpolicy objectives

• Select an approach, define baseline scenarios and prepare a plan that takes into account the critical issues

j p gpolicy objectives

• Select an approach, define baseline scenarios and prepare a plan that takes into account the critical issuesa plan that takes into account the critical issues

• Compare energy use and demand before and after the program is implemented to determine energy savings and calc lated a oided emissions

a plan that takes into account the critical issues• Compare energy use and demand before and after the

program is implemented to determine energy savings and calc lated a oided emissionscalculated avoided emissions

• Report the evaluation results and, as appropriate, work with program administrators to implement

calculated avoided emissions• Report the evaluation results and, as appropriate, work

with program administrators to implement recommendations for current or future program improvementsrecommendations for current or future program improvements

17

Source: Model Energy Efficiency Program Impact Evaluation Guideline, a resource of the National Action Plan for Energy Efficiency, November 2007

National Action Plan for Energy Effi i T i l I t R t d

National Action Plan for Energy Effi i T i l I t R t dEfficiency – Typical Impact ReportedEfficiency – Typical Impact Reported• Estimates of Gross Savings• Estimates of Gross Savings

– M&V methods used to determine savings from sample of projects to be applied to all projects in the program

– Deemed Savings based on historical and verified data, applied to conventional measures

– M&V methods used to determine savings from sample of projects to be applied to all projects in the program

– Deemed Savings based on historical and verified data, applied to conventional measures

– Statistical analyses of large volumes of metered energy usage data• Estimates of Net Savings

– Self-reporting surveys

– Statistical analyses of large volumes of metered energy usage data• Estimates of Net Savings

– Self-reporting surveys– Statistical models (i.e., compare usage and knowledge of participants and

non-participants)– Stipulated net-to-gross ratios

• Estimate of Co-Benefits (e.g., avoided air emissions)

– Statistical models (i.e., compare usage and knowledge of participants and non-participants)

– Stipulated net-to-gross ratios• Estimate of Co-Benefits (e.g., avoided air emissions)Estimate of Co Benefits (e.g., avoided air emissions)

– Emissions factors– Emissions scenario analyses (e.g., computer models)

Estimate of Co Benefits (e.g., avoided air emissions)– Emissions factors– Emissions scenario analyses (e.g., computer models)

18

Source: Model Energy Efficiency Program Impact Evaluation Guideline, a resource of the National Action Plan for Energy Efficiency, November 2007

New York:E l ti P d t

New York:E l ti P d tEvaluation ProductsEvaluation Products

• Impact evaluation plans specific to each• Impact evaluation plans specific to eachImpact evaluation plans specific to each program type

• Annual evaluation reports provide gross and

Impact evaluation plans specific to each program type

• Annual evaluation reports provide gross andAnnual evaluation reports provide gross and net energy savings– At the portfolio level,

Annual evaluation reports provide gross and net energy savings– At the portfolio level,– For the largest energy savors,– By sector, and– For the largest energy savors,– By sector, and– By specific program.– By specific program.

19

Source: New York Energy $mart Program Evaluation and Status Report: Year Ending December 31, 2008. Report to the Public Service Commission; Final Report – May 2009.

New York:I t E l ti P t l

New York:I t E l ti P t lImpact Evaluation ProtocolsImpact Evaluation Protocols

Objectives & ActivitiesObjectives & Activities Savings EstimatesSavings EstimatesObjectives & ActivitiesObjectives & Activities• Research Objectives – Determine:

– Realization rates for gross savings

• Research Objectives – Determine:– Realization rates for gross

savings

• Billing Analysis• On-site measurements

d

• Billing Analysis• On-site measurements

d

Savings EstimatesSavings Estimates

savings– Attribution– Precision and bias

Activities

savings– Attribution– Precision and bias

Activities

• Deemed savings• Engineering models• Energy simulations

• Deemed savings• Engineering models• Energy simulations

• Activities – Gross impacts– Demand savings

N i t

• Activities – Gross impacts– Demand savings

N i t– Non-energy impacts– Attribution– Non-energy impacts– Attribution

20


New York:Examples of Data Requirements

New York:Examples of Data RequirementsExamples of Data RequirementsExamples of Data Requirements

• Project Level Information:– Address & contact information for site-owner and/or contractor

Measure Level Information:

• Project Level Information:– Address & contact information for site-owner and/or contractor

Measure Level Information:• Measure Level Information:– Description, quantity installed, energy savings, demand savings,

measure life, installation costs• Participant Level Information:

• Measure Level Information:– Description, quantity installed, energy savings, demand savings,

measure life, installation costs• Participant Level Information:• Participant Level Information:

– Base measure characteristics and/or statistics• Other Information:

– Utility consumption data for participant

• Participant Level Information:– Base measure characteristics and/or statistics

• Other Information:– Utility consumption data for participant– Utility consumption data for participant– Utility consumption data for comparable non-participant– Outside influences – weather, economic conditions, etc.

• Sample Size:

– Utility consumption data for participant– Utility consumption data for comparable non-participant– Outside influences – weather, economic conditions, etc.

• Sample Size:• Sample Size:– Statistically significant samples as appropriate

• Sample Size:– Statistically significant samples as appropriate

21


New York:Summary of Evaluation Results

New York:Summary of Evaluation Resultsy o oy o o

BenefitsThrough Year-

End 2004Through Year-




End 2008

Electricity Savings from Energy Efficiency and On-Site Generation (Annual GWh) 1,400 1,950 2,350 3,070 3,220

P k D d R d ti (MW) 860 1 040 1 113 1 200 1 275Peak Demand Reduction (MW) 860 1,040 1,113 1,200 1,275

Permanent Measures (MW 325 445 495 650 700b

Curtailable 1 535 595 618 550 575

Net Fuel Savings (Annual MMBTu) 2 600 000 4 000 000 4 049 000 4 460 000 5 400 000Net Fuel Savings (Annual MMBTu) 2,600,000 4,000,000 4,049,000 4,460,000 5,400,000

Annual Energy Bill Savings to Participating Customers ($ Million) $195 $275 $330 $570 $590

Renewable Energy Generation (Annual GWh) 102 103 105 106 106

Net Additional Jobs2 2,500 3,100 3,600 4,500 4,900

Nox Emissions Reductions (Annual Tons)3 1,280 1,750 2,060 2,570 2,800

SO2 Emissions Reductions (Annual Tons)3 2,320 3,170 3,800 4,720 5,120

CO2 Emissions Reductions (Annual Tons)3 1,000,000 1,400,000 1,600,000 2,000,000 2,200,000

Equivalent number of cars removed from NY roadways 200,000 275,000 320,000 400,000 435,000

22


Wisconsin:Impact Evaluation Guidelines

Wisconsin:Impact Evaluation GuidelinesImpact Evaluation GuidelinesImpact Evaluation Guidelines

Business ProgramsBusiness Programs Residential ProgramsResidential Programs

I t kW d kWh iI t kW d kWh i • Impacts – kW and kWh savings• Impacts – kW and kWh savings• Impact – kW and kWh savings• Determine verified gross and net

impacts for measures implemented• Gross savings – tracked by measure

• Impact – kW and kWh savings• Determine verified gross and net

impacts for measures implemented• Gross savings – tracked by measure

• Impacts kW and kWh savings • Direct and indirect impact analysis • Gross savings – determined by

deemed savings and engineering i

• Impacts kW and kWh savings • Direct and indirect impact analysis • Gross savings – determined by

deemed savings and engineering i

g y• Gross savings adjustment factor &

attribution factor determined at sector level

• Estimate first year net savings (Y1NS)

g y• Gross savings adjustment factor &

attribution factor determined at sector level

• Estimate first year net savings (Y1NS)

review• Program specific impacts

review• Program specific impacts

Energy ImpactsReported By:

Energy ImpactsReported By:• Estimate first year net savings (Y1NS)

and life cycle net savings (LCNS)• Estimate first year net savings (Y1NS)

and life cycle net savings (LCNS)Reported By:Reported By:

• Program area,• Specific Program,

E d t h l

• Program area,• Specific Program,

E d t h l• End-use technology,• Geographic Region – County,

Service Territory, Assembly, Senate District

• End-use technology,• Geographic Region – County,

Service Territory, Assembly, Senate District

23Source: State of Wisconsin, Public Service Commission of Wisconsin – Focus on Energy Evaluation, Evaluation Calendar Year 2009, Detailed Evaluation Plans. Final: April 16, 2009. Prepared by: The Focus on Energy Evaluation Team.

Wisconsin: Sampling Steps i

Wisconsin: Sampling Steps i– Business Programs –– Business Programs –

• Sample stratified by: Sector and end use;

• Sample stratified by: Sector and end use;– Sector and end-use;

– State facility;– High-level technology type: Custom incentive, CFL, non-CFL deemed,

prescriptive incentive non-deemed;

– Sector and end-use;– State facility;– High-level technology type: Custom incentive, CFL, non-CFL deemed,

prescriptive incentive non-deemed; – Analysis cost category (all measures/projects assigned to cost category based

on size, type and measure complexity); and– Fuel source

• Each sample cell assigned to:

– Analysis cost category (all measures/projects assigned to cost category based on size, type and measure complexity); and

– Fuel source• Each sample cell assigned to:• Each sample cell assigned to:

– Engineering Review: gross savings verification and net-to-gross analysis– Automated Survey: net-to-gross analysis and installation confirmation

• Field resources allocated across all sectors

• Each sample cell assigned to:– Engineering Review: gross savings verification and net-to-gross analysis– Automated Survey: net-to-gross analysis and installation confirmation

• Field resources allocated across all sectorsd ou a o a d a o a o• Sample size by sector based on contribution to Business Programs gross

savings and likely variance of the gross savings adjustment factor

d ou a o a d a o a o• Sample size by sector based on contribution to Business Programs gross

savings and likely variance of the gross savings adjustment factor


Wisconsin: Data Requirementsi

Wisconsin: Data Requirementsi– Business Programs –– Business Programs –

• For deemed measures:• For deemed measures:– Application documentation – no. units installed, deemed algorithm

parameters, measure qualification info.– Invoices that verify no. of units installed and measure qualification

i f

– Application documentation – no. units installed, deemed algorithm parameters, measure qualification info.

– Invoices that verify no. of units installed and measure qualification i finfo.

– Energy Advisor impact statements• For non-deemed measures:

A li ti d t ti d i i id tif d d ib

info.– Energy Advisor impact statements

• For non-deemed measures:A li ti d t ti d i i id tif d d ib– Application documentation and invoices – identify and describe project and equipment installed

– Documentation of savings calculations – supporting docs. For assumptions and/or calculation methodology

– Application documentation and invoices – identify and describe project and equipment installed

– Documentation of savings calculations – supporting docs. For assumptions and/or calculation methodologyassumptions and/or calculation methodology

– Energy Advisor impact statementsassumptions and/or calculation methodology

– Energy Advisor impact statements


Wisconsin: Savings Estimates– Residential Programs –

Wisconsin: Savings Estimates– Residential Programs –Residential Programs Residential Programs

• Energy Star Lighting– Deemed savings, no. of program CFLs sold, installation rates

A li d Pl L d

• Energy Star Lighting– Deemed savings, no. of program CFLs sold, installation rates

A li d Pl L d• Appliance and Plug Loads– Deemed savings from WI water heater research , DOE, CA, etc.

• Apartment and Condo Efficiency Services– M&V – measure installation and use characteristics (e g quantities equipment

• Appliance and Plug Loads– Deemed savings from WI water heater research , DOE, CA, etc.

• Apartment and Condo Efficiency Services– M&V – measure installation and use characteristics (e g quantities equipmentM&V measure installation and use characteristics (e.g., quantities, equipment

efficiencies, operating hours) – Sample Sizes

• Existing buildings in-unit direct install – randomly selected ensuring projects w/ greatest savings are sampled with certainty

M&V measure installation and use characteristics (e.g., quantities, equipment efficiencies, operating hours)

– Sample Sizes• Existing buildings in-unit direct install – randomly selected ensuring projects w/ greatest savings

are sampled with certainty• Existing buildings contracted energy advisors – dependent on no. of projects completed• New constructions – likely to be a census (all measures are sampled, although counts for

individual measures are low)

• Neighborhood Efficiency Pilot ProgramS i t b d t i d th h ti i t l

• Existing buildings contracted energy advisors – dependent on no. of projects completed• New constructions – likely to be a census (all measures are sampled, although counts for

individual measures are low)

• Neighborhood Efficiency Pilot ProgramS i t b d t i d th h ti i t l– Savings to be determined through participant sample

• No energy impact estimates proposed for current evaluation period– Energy Star Homes, Home Performance w/ Energy Star, Efficient Heating and Cooling.

Targeting Home Performance w/ Energy Star

– Savings to be determined through participant sample

• No energy impact estimates proposed for current evaluation period– Energy Star Homes, Home Performance w/ Energy Star, Efficient Heating and Cooling.

Targeting Home Performance w/ Energy Star


Wisconsin:Summary of Evaluation Results

Wisconsin:Summary of Evaluation Resultsyy

Table 2-1d. All Programs: Tracked Energy Impacts -Annual First Year Program to Date (July 1, 2001–December 31, 2008)

Annual kWh Saved kW Reduction Annual Therms Saved

Gross Verified Gross Verified Net Gross Verified Gross Verified Net Gross Verified Gross Verified Net The 18-month Contract Period (July 1, 2007–December 31, 2008) Total Saved 625,335,422 591,021,366 393,477,991 109,615 103,184 64,867 28,698,506 26,623,540 14,758,518 Business 433,653,604 403,836,023 240,728,194 91,286 85,387 50,340 22,145,786 20,248,021 10,796,685

Residential 179,864,369 180,197,195 148,799,826 16,583 16,606 13,824 4,143,595 4,146,014 3,278,792

Renewables 11,817,448 6,988,149 3,949,971 1,746 1,190 703 2,409,124 2,229,506 683,041

FY07 (July 1, 2006–June 30, 2007)

Total Saved 260,254,357 238,189,767 162,763,097 43,911 41,008 27,979 15,914,328 13,694,194 8,520,818 Business 157,130,223 151,014,314 98,462,584 33,460 32,282 21,776 13,658,924 11,513,743 6,809,760

Residential 94,250,649 78,656,907 63,883,459 8,913 6,855 5,852 1,566,780 1,506,977 1,394,290

Renewables 8,873,486 8,518,546 417,054 1,538 1,871 351 688,625 673,475 316,767 FY06 (July 1, 2005–June 30, 2006)


Residential 90,185,421 73,961,454 72,078,061 14,066 11,286 11,066 1,858,471 1,573,064 1,489,822

Renewables 12,765,642 13,059,252 2,348,878 1,916 1,874 385 2,073,725 2,050,914 456,220 FY05 (July 1, 2004–June 30, 2005)

Total Saved 276,013,670 214,892,987 152,928,433 41,103 35,913 22,850 9,615,956 9,175,217 5,232,659

Business 143,707,711 110,718,465 55,861,214 27,083 20,906 9,972 7,045,122 7,106,161 3,463,969

Residential 112,935,136 82,266,121 92,805,638 11,027 11,745 11,893 2,123,643 1,725,612 1,680,593

Renewables 19,370,823 21,908,401 4,261,581 2,993 3,262 985 447,192 343,443 88,097 FY04 (July 1, 2003–June 30, 2004)


Residential 125,867,708 90,474,549 88,589,694 15,890 13,932 13,604 1,824,949 1,640,105 1,622,408

Renewables 497,895 483,954 449,101 199 220 195 91,460 213,833 199,749 FY03 (July 1, 2002–June 30, 2003)


Residential 97,358,904 89,638,260 80,130,947 14,066 13,788 12,184 1,677,293 1,890,084 1,881,419

Renewables 3,672,115 3,719,852 3,003,790 725 604 478 0 0 0 FY02 (July 1, 2001–June 30, 2002)

Total Saved 54,507,968 56,484,523 39,048,723 10,935 11,705 8,163 3,673,296 2,652,791 1,616,168 Business 30,559,935 30,532,158 18,764,493 6,966 7,036 4,397 2,760,541 1,740,729 785,725

Residential 23,948,033 25,952,365 20,284,229 3,969 4,668 3,766 912,755 912,062 830,443

Note: The numbers in this table are annual first year (not lifecycle).


California:Gross Energy Evaluation Rigor Levels

California:Gross Energy Evaluation Rigor Levelsgy ggy g

Rigor Level Minimum Allowable Methods for Gross Energy Evaluation

1. Simple Engineering Model (SEM) with M&V equal to IPMVP Option A and meeting all requirements in the M&V Protocol forthis method. Sampling according to the Sampling and Uncertainty Protocol.

2 Normalized Annual Consumption (NAC) using pre- and post-program participation consumption from utility bills from theBasic

2. Normalized Annual Consumption (NAC) using pre- and post-program participation consumption from utility bills from theappropriate meters related to the measures undertaken, normalized for weather, using identified weather data to normalize forheating and/or cooling as is appropriate to measures included. Twelve (12) months pre-retrofit and twelve (12) months post-retrofit consumption data is required. Sampling must be according to the Sampling and Uncertainty Protocol.

1. A fully specified regression analysis of consumption information from utility bills with inclusion/adjustment for changes andy p g y p y j gbackground variables over the time period of analysis that could potentially be correlated with the gross energy savings beingmeasured. Twelve (12) months post-retrofit consumption data are required. Twelve (12) months pre-retrofit consumption dataare required, unless program design does not allow pre-retrofit billing data, such as in new construction. In these cases, well-matched control groups and post-retrofit consumption analysis is allowable.1 Sampling must be according to the Sampling andUncertainty Protocol utilizing power analysis as an input to determining required sample size(s).

2 Building energy simulation models that are calibrated as described in IPMVP Option D requirements in the M&VEnhanced

2. Building energy simulation models that are calibrated as described in IPMVP Option D requirements in the M&VProtocols. If appropriate, may alternatively use a process-engineering model (e.g., AirMaster+) with calibration as described inthe M&V Protocols. Sampling according to the Sampling and Uncertainty Protocol.

3. Retrofit isolation engineering models as described in IPMVP Option B requirements in the M&V Protocols. Samplingaccording to the Sampling and Uncertainty Protocol.

4. Experimental design established within the program implementation process, designed to obtain reliable net energysavings based upon differences between energy consumption between treatment and non-treatment groups from consumptiondata.2 Sampling must be according to the Sampling and Uncertainty Protocol.

1 Post-retrofit only billing collapses the analysis from cross-sectional time-series to cross-sectional. Given this, even more care and examination is expected with regard to controlling for cross-sectional issues that could potentially bias the savings estimate.2 The overall goal of the Direct Impact Protocols is to obtain reliable net energy and demand savings estimates. If the methodology directly estimates net savings at the same or better rigor than the required level of rigor, then a gross savings and participant net impact analysis is not required to be shown separately.

28Source: State of California PUC, California Energy Efficiency Evaluation, April 2006

California: M&V OptionsCalifornia: M&V OptionsM&V Option How Savings are Calculated Typical ApplicationsOption A: Partially Measured Retrofit IsolationSavings are determined by partial field measurement of the energy Engineering calculations using short Lighting retrofit where power draw isSavings are determined by partial field measurement of the energyuse of the system(s) to which an ECM was applied; separate fromthe energy use of the rest of the facility. Measurements may beeither short-term or continuous.

Partial measurement means that some but not all parameter(s)may be stipulated, if the total impact of possible stipulationerror(s) is not significant to the resultant savings Careful review of

Engineering calculations using shortterm or continuous post-retrofitmeasurements and stipulations.

Lighting retrofit where power draw ismeasured periodically. Operating hours of thelights are assumed to be one half hour perday longer than store open hours

error(s) is not significant to the resultant savings. Careful review ofECM design and installation will ensure that stipulated values fairlyrepresent the probable actual value. Stipulations should be shownin the M&V Plan along with analysis of the significance of the errorthey may introduce.Option B: Retrofit IsolationSavings are determined by field measurement of the energy use ofthe systems to which the ECM was applied; separate from the

Engineering calculations using shortterm or continuous measurements

Application of controls to vary the load on aconstant speed pump using a variable speedthe systems to which the ECM was applied; separate from the

energy use of the rest of the facility. Short-term or continuousmeasurements are taken throughout the post-retrofit period.

constant speed pump using a variable speeddrive. Electricity use is measured by a kWhmeter installed on the electrical supply to thepump motor. In the base-year this meter is inplace for a week to verify constant loading.The meter is in place throughout the post-retrofit period to track variations in energyuse.

Option C: Whole BuildingSavings are determined by measuring energy use at the wholefacility level. Short-term or continuous measurements are takenthroughout the post-retrofit period.

Analysis of whole facility utility meter orsub-meter data using techniques fromsimple comparison to regressionanalysis.

Multifaceted energy management programaffecting many systems in a building. Energyuse is measured by the gas and electric utilitymeters for a twelve month base-year periodand throughout the post-retrofit period.

Option D: Calibrated SimulationSavings are determined through simulation of the energy use ofcomponents or the whole facility. Simulation routines must bedemonstrated to adequately model actual energy performancemeasured in the facility. This option usually requires considerableskill in calibrated simulation.

Energy use simulation, calibrated withhourly or monthly utility billing dataand/or end-uses metering.

Multifaceted energy management programaffecting many systems in a building butwhere no base-year data are available. Post-retrofit period energy use is measured by thegas and electric utility meters. Base-yearenergy use is determined by simulation usinga model calibrated by the post-retrofit periodtilit d t

29

utility data.

Source: State of California PUC, California Energy Efficiency Evaluation, April 2006

California:M t d V ifi ti O ti

California:M t d V ifi ti O tiMeasurement and Verification OptionsMeasurement and Verification Options

Measure Type Basic Rigor Level Enhanced Rigor Level

Appliances Option A: Partially Measured Retrofit Isolation Option B: Retrofit IsolationAppliances Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Commissioning and O&M programs Option D: Calibrated Simulation Option D: Calibrated Simulation

Comprehensive Option D: Calibrated Simulation Option D: Calibrated Simulation

Envelope Option D: Calibrated Simulation Option D: Calibrated Simulation

Food Service Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

HVAC Controls Option D: Calibrated Simulation Option D: Calibrated Simulation

HVAC Equipment Efficiency Option A: Partially Measured Retrofit Isolation Option D: Calibrated Simulation

Lighting Controls Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Lighting Efficiency Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

New Construction Option D: Calibrated Simulation Option D: Calibrated Simulation

Non-HVAC Motor Controls Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Non-HVAC Motor Efficiency Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Process Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Refrigeration Option A: Partially Measured Retrofit Isolation Option D: Calibrated Simulation

Water Heating Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation

Water Pumping/Treatment Option A: Partially Measured Retrofit Isolation Option B: Retrofit Isolation


California: Summary of M&V Protocol for Basic Level of Rigor

California: Summary of M&V Protocol for Basic Level of Rigorgg

• E.g., Appliance Programs• E.g., Appliance ProgramsProvision Requirement

Physical inspection of installation to verify correct measure installation and installation qualityVerification

IPMVP Option Option A

Source of Sti l t d D t

[Technical Reference Manual] assumptions, program work papers, engineering references, manufacturers catalog data, on-site survey data

Stipulated Datag , y

Baseline Definition

Consistent with program baseline definition. May include federal or [state] appliance standards effective at date of equipment manufacture, [state] building standards in effect at time of building permit; existing equipment conditions or common replacement or design practices as defined by the program

Monitoring Strategy and Duration

Spot or short-term measurements depending on measure type

Weather dependent measures: normalize to long-term average weather data as directed by the Impact Weather Adjustments

p g g y pEvaluation Protocol

Calibration Criteria Not applicable

Additional Provisions None


California: Summary of M&V Protocol of Enhanced Level of Rigor

California: Summary of M&V Protocol of Enhanced Level of Rigorgg

Provision RequirementPhysical inspection of installation to verify correct measure installation and installation

• E.g., Commercial lighting Controls, New Construction

Verification quality. Review of commissioning reports or functional performance testing to verify correct operation

IPMVP Option Option B or Option D

Source of Sti l t d D t

[Technical Reference Manual] assumptions, program work papers, engineering references, f t t l d t it d tStipulated Data manufacturers catalog data, on-site survey data

Baseline Definition

Consistent with program baseline definition. May include federal or [state] appliance standards effective at date of equipment manufacture, [state] building standards in effect at time of building permit; existing equipment conditions or common replacement or design practices as defined by the programp y p g

Monitoring DurationSufficient to capture all operational modes and seasons

Weather AdjustmentsWeather dependent measures: normalize to long-term average weather data as directed by the Impact Evaluation Protocolby the Impact Evaluation Protocol

Calibration CriteriaOption D building energy simulation models calibrated to monthly billing or interval demand data. Optional calibration to end-use metered data

Additional ProvisionsHourly building energy simulation program compliant with ASHRAE Standard 140-2001


Cost-Effectiveness Evaluation:C i l

Cost-Effectiveness Evaluation:C i lConnecticut ExampleConnecticut Example

$0.0250

$0.0300

2006 Program Adminstrator Cost per Lifetime kWh Saved (Costs Include Direct Costs and Performance Incentives)

$0.0250

$0.0300

2006 Program Adminstrator Cost per Lifetime kWh Saved (Costs Include Direct Costs and Performance Incentives) • Calculated program

administrator cost per lifetime kWh saved

• Calculated program administrator cost per lifetime kWh saved

$0 0050

$0.0100

$0.0150

$0.0200

$0 0050

$0.0100

$0.0150

$0.0200lifetime kWh saved

• Comparatively assess to other similar programs to

lifetime kWh saved• Comparatively assess to

other similar programs to $0.0000

$0.0050

$0.0000

$0.0050determine relative cost-effectiveness

• Data Requirements:

determine relative cost-effectiveness

• Data Requirements:• Data Requirements:– Lifetime kWh savings– Administrator costs

• Data Requirements:– Lifetime kWh savings– Administrator costs

33

Data Collection and ReportingData Collection and ReportingData Collection and ReportingData Collection and Reporting

• Collect raw data from EDCs- including project, • Collect raw data from EDCs- including project, g p j ,participant, program and aggregate data as well as associated data dictionaries and relationship diagrams.EDC ill l d d t t GDS S FTP (SFTP) it

g p j ,participant, program and aggregate data as well as associated data dictionaries and relationship diagrams.EDC ill l d d t t GDS S FTP (SFTP) it• EDCs will upload data to GDS Secure FTP (SFTP) site to protect participant and other confidential data

• Read only access to EDC data tracking systems (where

• EDCs will upload data to GDS Secure FTP (SFTP) site to protect participant and other confidential data

• Read only access to EDC data tracking systems (whereRead only access to EDC data tracking systems (where possible)

• Raw data allows for verification of aggregate data as ll t h ki f i di id l j t

Read only access to EDC data tracking systems (where possible)

• Raw data allows for verification of aggregate data as ll t h ki f i di id l j twell as spot checking of individual projectswell as spot checking of individual projects

34

Reporting WebsiteReporting WebsiteReporting WebsiteReporting Website

• Can be a link from the PUC website or embedded in • Can be a link from the PUC website or embedded in the site itself.

• Public reports section and capability for password t t d ti f EDC d PUC

the site itself.• Public reports section and capability for password

t t d ti f EDC d PUCprotected section for EDC and PUC use.• Capability to export data to Microsoft Excel (.xls),

Adobe Acrobat (.pdf), .csv, .tiff, etc.

protected section for EDC and PUC use.• Capability to export data to Microsoft Excel (.xls),

Adobe Acrobat (.pdf), .csv, .tiff, etc.Adobe Acrobat (.pdf), .csv, .tiff, etc.• Reports examples include Program Year to date

savings and costs, savings by program, by EDC, etc.

Adobe Acrobat (.pdf), .csv, .tiff, etc.• Reports examples include Program Year to date

savings and costs, savings by program, by EDC, etc.

35

Timetable for Development of h di l

Timetable for Development of h di lthe Audit Planthe Audit Plan

• Kick-off meeting: September 23rd• Kick-off meeting: September 23rdg p

• Draft “straw man” proposal for M&V – October 7th

• Draft “straw man” proposal – impact evaluation –

g p

• Draft “straw man” proposal for M&V – October 7th

• Draft “straw man” proposal – impact evaluation –• Draft straw man proposal impact evaluation October 7th

• Draft “straw man” proposal – process evaluation –

• Draft straw man proposal impact evaluation October 7th

• Draft “straw man” proposal – process evaluation –October 7th

• First draft of Statewide Evaluation Audit Plan –November 1st

October 7th

• First draft of Statewide Evaluation Audit Plan –November 1stNovember 1s

• Final Statewide Evaluation Audit Plan – December 2nd

November 1s

• Final Statewide Evaluation Audit Plan – December 2nd

36

Part 5Development of Sector and

Part 5Development of Sector andDevelopment of Sector and Program Specific Evaluation Development of Sector and Program Specific Evaluation

PlansPlans

37

Impact Evaluation ProcessImpact Evaluation ProcessImpact Evaluation ProcessImpact Evaluation Process

38

ProgramsProgramsResidential Residential Commercial/IndustrialCommercial/Industrial//

• Energy Efficiency Behavior and Education• Energy Star and High Efficiency Appliance• Appliance Turn-In/Recycling • CFL Rewards

• Energy Efficiency Behavior and Education• Energy Star and High Efficiency Appliance• Appliance Turn-In/Recycling • CFL Rewards

• Energy Audits and Assessments• HVAC Efficiency/Tune-up• Lighting Efficiency• Equipment Rebates/Prescriptive

C t T h l A li ti

• Energy Audits and Assessments• HVAC Efficiency/Tune-up• Lighting Efficiency• Equipment Rebates/Prescriptive

C t T h l A li ti• Home Performance/Home Audits/Whole-Building/Weatherization

• HVAC Efficiency• Solar/Renewable Rebates• New Construction

• Home Performance/Home Audits/Whole-Building/Weatherization

• HVAC Efficiency• Solar/Renewable Rebates• New Construction

• Custom Technology Applications• New Construction• “Performance Contracting”• Government (State and Local)/Institutional (Schools)• Government/Non-Profit Lighting Efficiency

• Custom Technology Applications• New Construction• “Performance Contracting”• Government (State and Local)/Institutional (Schools)• Government/Non-Profit Lighting Efficiency

• New Construction• Low-Income Home Performance Check-up

Audit/Appliance Replacement/Weatherization• Low-Income Joint Utility Usage Management• Low-Income Room AC Replacement

• New Construction• Low-Income Home Performance Check-up

Audit/Appliance Replacement/Weatherization• Low-Income Joint Utility Usage Management• Low-Income Room AC Replacement

• Federal Facilities• Street and Traffic Lighting Efficiency• Federal Facilities • Street and Traffic Lighting Conversion• Programmable Controllable Thermostat

• Federal Facilities• Street and Traffic Lighting Efficiency• Federal Facilities • Street and Traffic Lighting Conversion• Programmable Controllable Thermostat

• Efficiency Rewards Rate• Super Peak Time of Use (TOU)• Home Energy Incentives• Multifamily Housing• Direct Load Control

• Efficiency Rewards Rate• Super Peak Time of Use (TOU)• Home Energy Incentives• Multifamily Housing• Direct Load Control

• Pay Ahead (Smart) Service Rate• Motors and Drives• Custom Application• Customer Load Response• Distributed Generation

• Pay Ahead (Smart) Service Rate• Motors and Drives• Custom Application• Customer Load Response• Distributed Generation• Direct Load Control• Direct Load Control• Direct Load Control• Contracted Demand Response• Critical Peak Rebate (CPR) Rate• Time of Use (TOU) w/ Critical Peak Pricing Rate• Hourly Pricing Option (HPO) Rate

• Direct Load Control• Contracted Demand Response• Critical Peak Rebate (CPR) Rate• Time of Use (TOU) w/ Critical Peak Pricing Rate• Hourly Pricing Option (HPO) Rate

39

Hourly Pricing Option (HPO) Rate• Renewable Resources• Permanent Load Reduction• Conservation Voltage Reduction

Hourly Pricing Option (HPO) Rate• Renewable Resources• Permanent Load Reduction• Conservation Voltage Reduction

TRM Matrix – Residential TRM Matrix – Residential TRM Measures with Deemed Savings

TRM Measures with Measurement and/or

TRM Guidelines TRM Notes:

Simulation Required

Energy Efficiency Behavior and Education

Energy Star and High Efficiency Appliance

• ES Refrigerators• ES Clothes Washers• ES Dishwashers• Dehumidifiers• ES Room AC• ES Freezer

Appliance Turn-In/Recycling • Refrigerator/Freezer • Based on number of units and fixed deemed savings per unit

CFL Rewards/Lighting • ES CFL Bulbs• ES Torchieres• ES Indoor Fixtures• ES Outdoor Fixture• ES Ceiling Fan w/ Lights

Home Performance/Home Audits/Whole- • ES Windows • HomeCheck Software Home Types – • No algorithms or Building/Weatherization • Home Performance w/ ES Windows:

• Heat Pump• Electric Heat/CAC• Electric Heat Only

information provided for ES/Generic Home Audits

HVAC Efficiency • Central ACAi S HP• Air Source HP

• Ground Source HP• DSHP Desuperheater• Furnace – High E Fan

Solar/Renewable Rebates

40

Residential Continued Residential Continued TRM Measures with Deemed Savings

TRM Measures with Measurement and/or

TRM Guidelines

TRM Notes: Deemed Savings Measurement and/or

Simulation RequiredGuidelines

New Construction • Insulation Up-Grades• High E Windows• Air Sealing• High E HVAC Equipment

• Specify REM/Rate User Defined Reference Home

• Duct Sealing • Lighting & Appliances• Ventilation Equipment

Low-Income Home Performance Check-p A dit/Applianceup Audit/Appliance

Replacement/Weatherization

Low-Income Joint Utility Usage Management

Low-Income Room AC Replacement p

Efficiency Rewards Rate

Super Peak Time of Use (TOU)

Home Energy Incentives

Multifamily Housing

Direct Load Control • Air Conditioning Cycling • Pool Pump Load Control

41

TRM Matrix – Commercial/Industrial TRM Matrix – Commercial/Industrial TRM Measures with Deemed Savings

TRM Measures with Measurement and/or Simulation

TRM Guidelines

TRM Notes: Savings Measurement and/or Simulation

RequiredGuidelines

Energy Audits and Assessment

HVAC Efficiency/Tune-up • Central Air Conditioners• Room AC• Air Cooled DX• Split Systems• Packaged Terminal SystemsPackaged Terminal Systems• Air Source Heat Pumps• Water Source Heat Pumps• Ground Water/Source Heat Pumps• Electric Chillers

Lighting Efficiency • CFLs• Low Taby T-5/T-8• High Bay T-5/T-8• High E Fluorescent• Pulse Start Metal Halide• Low Bay LED• Fluorescent Lighting Fixture• LED Exit Signs• Lighting Controls – Occupancy Sensor/Controls/Daylight Dimmer• 20% Lighting Power Density (LPD) Reduction

Eq ipment Rebates/P esc ipti eEquipment Rebates/Prescriptive

Custom Technology Applications

New Construction

“Performance Contracting”

Government (State and Local)/Institutional(Schools)

Government/Non-Profit Lighting Efficiency

Federal Facilities

Street and Traffic Lighting Conversion • LED Round Traffic Signals• LED Turn Arrows• LED Pedestrian Signs

42

Commercial/Industrial ContinuedCommercial/Industrial ContinuedTRM Measures with Deemed Savings

TRM Measures with Measurement and/or Simulation Required

TRM Guidelines

TRM Notes:

Programmable Controllable Thermostats

Pay Ahead (Smart) Service Rate

Motors and Drives • Variable Frequency Drives• Air Compressor with VFD

• Motors: No qualitative description or applicability guidelines. Baseline Motor Efficiencies are specified.

Custom Application

Customer Load Response • All Demand Response treated independently

Distributed Generation

Direct Load Control • All Demand Response treated independently

Contracted Demand Response • All Demand Response treated d d lindependently

Critical Peak Rebate (CPR) Rate

Time of Use (TOU) w/ Critical PeakPricing Rate

Hourly Pricing Option (HPO) Rate

Renewable Resources

Permanent Load Reduction

Conservation Voltage Reduction

43

SWE-9-23 Kick-Off Mtg Presentation.ppt · 9. Verification and due diligence procedures for site...

Documents

Transcript of SWE-9-23 Kick-Off Mtg Presentation.ppt · 9. Verification and due diligence procedures for site...