DHP for Houses with Electric FAFResearch Plan: Revisions

Adam Hadley, Ben Hannas, Bob Davis, My TonR&E SubcommitteeFebruary 25, 2015

Agenda

• Overview of comments• Review of revisions made• Select detailed comments & responses• Other comments & responses• Discussions• Wrap up

2 - Outline

3

Summary of Comments

1. Reviewers generally agreed with overall study approach.

2. Comments received were in regard to data collection and analysis details.

3. Comment received from reviewers on the following areas:– Proposed Approach– Field Research– Phone Survey– Analysis Plan

4

Revision of Research Plan in Response to Comments

The Research Plan was restructured, and the following sections of the were revised:• Overview of Proposed Approach: added details regarding

analysis of data from 2012 Study, data collection and analysis methods

• Field Research: added details to address sample design and data collection

• Phone Survey: added details regarding data collection & use• Analysis Plan:

– added details regarding development of DHP/FAF Curve for SEEM model– added details regarding determination of supplemental fuel adjustment,

determination of occupancy adjustment, and determination of control strategy

5

Revision of Research Plan in Response to Comments

Added details in Overview of Proposed Approach regarding analysis of data from 2012 Study:

• Construct a DHP/FAF curve for each home• Review the curves for relative importance of key variables (UA, Uo,

floor area, thermostat behavior, etc.).• Re-analyze the 2012 Study data to give an initial estimate of the SEEM

calibration factor for DHP/FAF homes based on SEEM runs and the metering results.

• Add modeled SEEM runs with the DHP zonal setting and duct effects in a post-processing step.

• This analysis is a proof-of-concept that the methods from the DHP zonal analysis will also work for the DHP/FAF analysis, possibly with some tweaks (e.g., control strategy weights).

6

Revision of Research Plan in Response to Comments

Added other details in Overview of Proposed Approach regarding data collection and analysis methods:1. Use the phone survey to collect key characteristics for a

larger set of homes analyzed in the billing analysis (thermostat behavior, supplemental fuel use, occupancy, size of home, etc.)

2. Conduct a billing analysis to obtain a direct savings estimate for the measure and provide data for calibration of SEEM output based on supplemental fuels and control types.

3. Develop separate DHP/FAF UES workbooks for SF and MH, after collecting and analyzing the field data, phone survey data, billing data, and after updating SEEM.

7

Revision of Research Plan in Response to Comments

Added details in Field Research to address:1. Sample design: Added a table to clearly show how the sites are used in

the various analysis types of the proposed research.2. Data collection: Added specific details about how cooling will be dealt

with.

Added details in Phone Survey regarding data collection and use:• Gather data about thermostat behavior, supplemental fuel use,

occupancy patterns (vacations, partial year residence, etc.), and size of home.

• Integrate with program information (primarily make/model of DHP) to develop a data set comparable to the DHP pilot data set used in the previous evaluations.

• The data set will be helpful in the billing analysis summaries.

8

Revision of Research Plan in Response to Comments

Added details in Analysis Plan regarding Development of DHP FAF Curve for SEEM Model: Use results from Field Research to calculate the ratio of DHP

and FAF output by a number of possible explanatory variables: outdoor temperature, control strategy, UA, Uo, floor area.

Choose the best of these explanatory variables to generate a model that will be inserted into SEEM.

Estimates for the ratio of DHP and FAF for each operating strategy come from analyzing total kWh usage, run-time of each system, and DHP capacity curves from the NEEA Studies.

9

Revision of Research Plan in Response to Comments

Added details in Analysis Plan regarding Determination of Supplemental Fuel Adjustment In the metered dataset, homes were specifically chosen to have little or no

wood heat so that the metered DHP FAF population is different from the total DHP FAF population with regard to supplemental fuel use and an adjustment is needed.

The supplemental fuel adjustment is calculated using both the survey and billing data. Survey data provide whether a home uses supplemental fuels and a rough estimate of how much supplemental fuel. In the dataset, these are then flagged as a Boolean yes/no to whether a home has supplemental fuel.

Billing data are analyzed based on cohorts developed in the survey supplemental fuel analysis to develop a ratio of electric heating use to total heating use (electric + supplemental) on average for the population (possibly by climate zone).

10

Revision of Research Plan in Response to Comments

Added details in Analysis Plan regarding Determination of Occupancy Adjustment The survey data will be used to calculate a ratio of occupancy, and the fraction

will be multiplied by the DHP/FAF savings, which will reduce the savings.

Added details in Analysis Plan regarding Determination of Control Strategy Responses from the survey will be used to categorize homes into control

strategy bins. The metered homes will also receive the survey, and the responses of the

metered group on the survey will be compared to the observations of field staff and analysis of metered heating energy use.

The correlations of survey responses to metered energy use and field staff observations may show a bias in the survey responses. This bias will then be adjusted for in the total survey population. The adjusted population control strategies will be summarized and may be used either inside of SEEM or in the UES analysis.

11

Select Detailed Comments & Responses

12

Comments & Response

Comments regarding Primary Field Research:1. Need to group metered homes into control strategies based

on the same data that is used to group phone survey homes.2. Use other information available from the metered homes to

classify by strategy type. The comparison of these two methods for classifying the metered group into control strategies should be very informative. Would give a sense of how wrong the classification of the billing analysis population is.

Response:• Added language in analysis plan for Primary Field Research to

describe calibration process in more detail

13

Comments & Response

Comment Regarding: UES Development, SEEM Analysis1. It is not clear that current Phase I adjustments would be

reasonable for DHPs in an FAF home.

Response:• Incorporated a new SEEM model capable of FAF/DHP

simulation and separate calibration of FAF/DHP.

14

Comments & Response

Comments Regarding: UES Development, SEEM Analysis 1. The ratios of the SEEM runs for the combinations of the two

equipment types are are critical. A description of how they are computed is needed.

2. Shouldn’t the savings for each control type be computed from SEEM models that are calibrated to the relevant portions of the metered sample?

3. The results from the metered sample provide the fraction of DHP use, and could later be re-weighted to reflect the distribution of the control types found in the full population.

Response:• Added details on supplemental fuel and occupancy adjustments,

and how they will be used in the SEEM simulation runs

15

Comments & Response

Comment Regarding: Energy Savings Calculations 1. The field work from 2012 plus this new round give us 60

homes where the heating use is directly metered. This seems like a good opportunity to see if the Phase 1 and 2 calibrations yield heating use that is close to directly metered heating use. Especially as the DHP fractions are coming primarily in this research from the metered sample.

Response:• Added details on SEEM calibration adjustments, and for

comparison of simulation results with billing analysis.

16

Comments & ResponseComments Regarding: SEEM Calibration 1. The distribution of DHP/FAF use within each control strategy should

be focus for calibrating the SEEM and post-SEEM analyses. They are needed to estimate baseline and efficient case energy use.

2. Subsequent calibration work should be done to account for the characteristics of billing population, but only after the heating use is correct for models that match the characteristics of the 60 metered sites.

Response:• Added details on SEEM calibration adjustments, and for comparison

of simulation results with billing analysis.

17

Other Comments

18

Other Comments

19

Other Comments

20

Other Comments

21

Other Comments

22

Discussion of Current Plan

Are the revisions/modifications sufficient regarding:• Field Research/data collection• SEEM calibration factors• Energy Savings Calculations• Analysis details• Other comments

23

Discussion of Current Plan

Are there other revisions/modifications needed?

24

Discussion of Current Plan

Is the R&E Subcommittee comfortable with how the contract analysts have addressed the comments received?

25

Discussion of Current Plan

Can the R&E subcommittee recommend the revised plan for RTF adoption?

26

Thank you!

Research Plan Review: Data Collection

1. Expand on 2012 Study with field metering: Develop relationship of DHP/FAF heat output fraction to other

factors & embed this relationship within SEEM model Understand interactions of DHP/FAF operations in colder climates Provide dataset to calibrate DHP/FAF SEEM model

2. Conduct Billing Analysis: Estimate savings at the meter for the population of installations Estimate impact of non-electric fuels Calibrate SEEM by control type, or other factors, to population

3. Conduct Phone Survey Collect data on wood/supplemental fuel use, control strategy, house

size, etc.

27 – Research Plan

Research Plan Review: Analysis

1. Use Field Research data to develop and calibrate SEEM simulation (for houses with a solid heat signature)

2. Billing Analysis: Pre/post VBDD for each house Use Billing Analysis and Phone Survey results to estimate

impact of supplemental fuels (similar to method used in zonal study: energy consumption pre and post of those with supplemental fuels vs those without)

Also used as a QC check on resulting UES values after many manipulations through SEEM

3. Use Phone Survey results to establish a population-weighted control strategy mix (and same for other critical variables)

4. Using newly calibrated version of SEEM, estimate UES.

28 – Research Plan