Can High Bill Alerts Help Utility Customers to Save Energy? · 2019-08-02 · • Reduce customer...
Transcript of Can High Bill Alerts Help Utility Customers to Save Energy? · 2019-08-02 · • Reduce customer...
-
Ryan Fulleman
Can High Bill Alerts Help Utility Customers to Save Energy?
IEPEC Denver, COAugust 20, 2019
-
IEPEC, 2019 Denver
Agenda
HBA Pilot Evaluation ApproachFindingsTakeaways and Future Research
2
-
IEPEC, 2019 Denver
HBA Pilot Overview
• Xcel Energy sought to reduce call center call volumes• Many residential customers call about higher-than-expected bills• Reduce customer service costs
• High Bill Alert Pilot Program• 50,000 MN residential dual-fuel customers automatically enrolled• Email alert sent when customer’s monthly electricity or gas consumption
was on track to exceed normal levels• Alerts sent June 2015 - June 2016 • HER customers excluded• Implemented by Opower (Oracle)
3
-
IEPEC, 2019 Denver
High Bill Alert Email
4
Bill alert when spending on track to be 30% higher than normal
Recipient directed to Xcel Energy’s on-line Home Energy Assessment website
Customers can opt out
Features
Xcel Energy branding
-
IEPEC, 2019 Denver
High Bill Alert Delivery
5
Electric HBAs Gas HBAs
• 59% of HBA emails were opened by recipient• 6.4% of HBA emails resulted in clicking the Xcel Energy on-line energy audit link
FuelNumber of
Customers Sent HBAs
Number of HBAs Sent
Average Number of HBAs per Treated
CustomerElectricity 31,591 100,320 3.2Natural Gas 6,163 13,355 2.2
Sheet1
Electricity Savings
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (kWh)0.0770.0770.1330.134% Savings0.4%0.6%
(0.043)(0.043)(0.057)(0.057)kWh Savings0.0770.134
Customer Fixed EffectsYesYesYesYesBaseline21.388888888923.1034482759
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.33%0.41%
Number of Customers75,26775,26775,26775,267kWh Savings Error Margin0.06991250.0939295
Number of Observations2,689,1922,689,1922,689,1922,689,192
Notes: The dependent variable is customer electricity ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
Table 7. Natural Gas Regression Estimates
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (therms)0.00910.0090.00980.0097% Savings0.5%0.5%
(0.004)(0.004)(0.004)(0.004)kWh Savings0.0090.0097
Customer Fixed EffectsYesYesYesYesBaseline1.66666666672.0208333333
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.4%0.3%
Number of Customers75,25675,25675,25675,256kWh Savings Error Margin0.0059220.0067445
Number of Observations2,684,7072,684,7072,684,7072,684,707
Notes: The dependent variable is customer natural gas ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
% Savings
3.2686363636363638E-34.0656052238805963E-33.2686363636363638E-34.0656052238805963E-3Phase 1Phase 23.5999999999999999E-35.7999999999999996E-3
kWh Savings
[VALUE]
6.9912500000000002E-29.3929499999999999E-26.9912500000000002E-29.3929499999999999E-2Phase 1Phase 27.6999999999999999E-20.13400000000000001
Avg. Daily Savings per Customer
% Savings
3.5532000000000003E-33.3374845360824744E-33.5532000000000003E-33.3374845360824744E-3Phase 1Phase 25.4000000000000003E-34.7999999999999996E-3
kWh Savings
3.5532000000000003E-33.3374845360824744E-33.5532000000000003E-33.3374845360824744E-3Phase 1Phase 28.9999999999999993E-39.7000000000000003E-3
Avg. Daily Savings per Customer
Sheet2
FuelNumber of Customers Sent HBAsNumber of HBAs SentAverage Number of HBAs per Treated Customer
Electricity31,591100,3203.2
Natural Gas6,16313,3552.2
Fuel
Number of Customers Sent High Bill Alerts
Total Number of High Bill Alerts Sent
Average Number of High Bill Alerts per Treatment Group Customer
Electricity
31,591
100,320
3.2
Natural Gas
6,163
13,355
2.2
Fuel
Number of Customers
Sent High Bill Alerts
Total Number of High Bill
Alerts Sent
Average Number of High Bill Alerts
per Treatment Group Customer
Electricity 31,591 100,320 3.2
Natural Gas 6,163 13,355 2.2
-
IEPEC, 2019 Denver
Research Objectives
• No statistically significant effect on customer call center call volumes (Source: Opower/Oracle analysis)
• Cadmus research objective • Estimate customer gas and electricity savings during and after the pilot
6
-
IEPEC, 2019 Denver
HBA Pilot Evaluation ApproachFindingsTakeaways and Future Research
7
Agenda
-
IEPEC, 2019 Denver
Evaluation Approach
• Pilot implemented as an RCT• 50,000 treatment group customers and 25,000 control group customers• Only treatment group customers eligible to receive HBAs
• Monthly billing consumption data• Validation of randomized experiment• Fixed-effects D-in-D panel regression • Estimated savings for two phases:
• Phase 1: HBAs delivered (June 2015 - June 2016) • Phase 2: HBA delivery suspended (July 2016 – April 2018)
8
-
IEPEC, 2019 Denver
HBA Pilot Evaluation ApproachFindingsTakeaways and Future Research
9
Agenda
-
IEPEC, 2019 Denver
Electricity Savings Estimates
10
Percentage SavingskWh Savings
Note: Error bars indicated 90% confidence intervals based on standard errors clustered on customers.
• Electricity savings persisted after Xcel Energy stopped sending HBAs• In Phase 2, electricity savings per customer who received an HBA
equaled 0.16 kWh (0.7%)
0.4%
0.6%
0.0%
0.2%
0.4%
0.6%
0.8%
1.0%
1.2%
Phase 1 Phase 2
0.08
0.13
0.00
0.05
0.10
0.15
0.20
0.25
Phase 1 Phase 2
Avg.
Dai
ly Sa
ving
s per
Cus
tom
er
Chart9
% Savings
3.6318181818181818E-34.2057985074626858E-33.6318181818181818E-34.2057985074626858E-3Phase 1Phase 24.0000000000000001E-36.0000000000000001E-3
Sheet1
Electricity Savings
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (kWh)0.0770.0770.1330.134% Savings0.4%0.6%
(0.043)(0.043)(0.057)(0.057)kWh Savings0.0770.134
Customer Fixed EffectsYesYesYesYesBaseline19.2522.3333333333
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.36%0.42%
Number of Customers75,26775,26775,26775,267kWh Savings Error Margin0.06991250.0939295
Number of Observations2,689,1922,689,1922,689,1922,689,192
Notes: The dependent variable is customer electricity ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
Table 7. Natural Gas Regression Estimates
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (therms)0.00910.0090.00980.0097% Savings0.5%0.5%
(0.004)(0.004)(0.004)(0.004)kWh Savings0.0090.0097
Customer Fixed EffectsYesYesYesYesBaseline1.81.94
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.3%0.3%
Number of Customers75,25675,25675,25675,256kWh Savings Error Margin0.0059220.0067445
Number of Observations2,684,7072,684,7072,684,7072,684,707
Notes: The dependent variable is customer natural gas ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
% Savings
3.6318181818181818E-34.2057985074626858E-33.6318181818181818E-34.2057985074626858E-3Phase 1Phase 24.0000000000000001E-36.0000000000000001E-3
kWh Savings
[VALUE]
6.9912500000000002E-29.3929499999999999E-26.9912500000000002E-29.3929499999999999E-2Phase 1Phase 27.6999999999999999E-20.13400000000000001
Avg. Daily Savings per Customer
% Savings
3.2900000000000004E-33.4765463917525775E-33.2900000000000004E-33.4765463917525775E-3Phase 1Phase 25.0000000000000001E-35.0000000000000001E-3
kWh Savings
3.2900000000000004E-33.4765463917525775E-33.2900000000000004E-33.4765463917525775E-3Phase 1Phase 28.9999999999999993E-39.7000000000000003E-3
Avg. Daily Savings per Customer
Chart10
kWh Savings
[VALUE]
6.9912500000000002E-29.3929499999999999E-26.9912500000000002E-29.3929499999999999E-2Phase 1Phase 27.6999999999999999E-20.13400000000000001
Avg. Daily Savings per Customer
Sheet1
Electricity Savings
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (kWh)0.0770.0770.1330.134% Savings0.4%0.6%
(0.043)(0.043)(0.057)(0.057)kWh Savings0.0770.134
Customer Fixed EffectsYesYesYesYesBaseline19.2522.3333333333
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.36%0.42%
Number of Customers75,26775,26775,26775,267kWh Savings Error Margin0.06991250.0939295
Number of Observations2,689,1922,689,1922,689,1922,689,192
Notes: The dependent variable is customer electricity ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
Table 7. Natural Gas Regression Estimates
Phase 1 (June 2015 – May 2016)Phase 2 (June 2016 – April 2018)
Model 1Model 2Model 1Model 2Phase 1Phase 2
Average Daily Savings per Customer (therms)0.00910.0090.00980.0097% Savings0.5%0.5%
(0.004)(0.004)(0.004)(0.004)kWh Savings0.0090.0097
Customer Fixed EffectsYesYesYesYesBaseline1.81.94
Weather VariablesNoYesNoYes
Month-Year Fixed EffectsYesYesYesYes% Savings Error Margin0.3%0.3%
Number of Customers75,25675,25675,25675,256kWh Savings Error Margin0.0059220.0067445
Number of Observations2,684,7072,684,7072,684,7072,684,707
Notes: The dependent variable is customer natural gas ADC for a month-year. All models are linear regression models. Standard errors in parentheses were clustered by customer. Cadmus estimated all models with pre-treatment and post-treatment monthly consumption data.
% Savings
3.6318181818181818E-34.2057985074626858E-33.6318181818181818E-34.2057985074626858E-3Phase 1Phase 24.0000000000000001E-36.0000000000000001E-3
kWh Savings
[VALUE]
6.9912500000000002E-29.3929499999999999E-26.9912500000000002E-29.3929499999999999E-2Phase 1Phase 27.6999999999999999E-20.13400000000000001
Avg. Daily Savings per Customer
% Savings
3.2900000000000004E-33.4765463917525775E-33.2900000000000004E-33.4765463917525775E-3Phase 1Phase 25.0000000000000001E-35.0000000000000001E-3
kWh Savings
3.2900000000000004E-33.4765463917525775E-33.2900000000000004E-33.4765463917525775E-3Phase 1Phase 28.9999999999999993E-39.7000000000000003E-3
Avg. Daily Savings per Customer
-
IEPEC, 2019 Denver
Monthly Electricity Savings Estimates
11
Note: Error bars indicate 90% confidence intervals based on standard errors clustered on customers.
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
Avg
Dai
ly S
avin
gs p
er C
usto
mer
(kW
h)
Estimated Savings 90% Lower Savings Estimate 90% Upper Savings Estimate
Phase 1: HBA treatment period
Chart1
Estimated Savings4215642186422174224842278423094233942370424014243042461424914252242552425834261442644426754270542736427674279542826428564288742917429484297943009430404307043101431324316043191-4.9500000000000002E-21.67E-27.1400000000000005E-29.3799999999999994E-28.1000000000000003E-23.7199999999999997E-29.9500000000000005E-20.131000000000000010.115500000000000014.1099999999999998E-25.1400000000000001E-20.168199999999999990.1510.117000000000000010.108299999999999997.3599999999999999E-23.6799999999999999E-29.3799999999999994E-20.250099999999999990.231399999999999990.14097.2099999999999997E-28.6E-30.043.9600000000000003E-20.123799999999999990.143499999999999990.174499999999999998.0799999999999997E-20.16730.266500000000000010.295599999999999970.260699999999999990.123799999999999997.9200000000000007E-290% Lower Savings Estimate4215642186422174224842278423094233942370424014243042461424914252242552425834261442644426754270542736427674279542826428564288742917429484297943009430404307043101431324316043191-0.15494450000000001-0.11884800000000001-5.5429500000000007E-2-9.8350000000000104E-3-6.8430000000000019E-3-5.1959000000000005E-2-1.0715000000000002E-2-1.916000000000001E-3-5.5719999999999936E-3-6.0232000000000008E-2-4.4503500000000001E-26.6045499999999979E-21.2491000000000002E-2-5.029649999999998E-2-4.8632999999999996E-2-5.0268500000000008E-2-7.4073E-2-1.7566499999999999E-29.8430999999999991E-27.3644500000000002E-23.8714999999999999E-3-5.4893999999999998E-2-0.10753699999999999-7.7453000000000022E-2-0.10993049999999999-5.1886000000000015E-2-1.2610499999999997E-23.22075E-2-4.2904000000000012E-22.7968500000000007E-28.7853000000000014E-29.3593999999999955E-26.8234999999999962E-2-3.9548500000000014E-2-6.358599999999999E-290% Upper Savings Estimate42156421864221742248422784230942339423704240142430424614249142522425524258342614426444267542705427364276742795428264285642887429174294842979430094304043070431014313243160431915.5944500000000008E-20.152247999999999990.19822950.1974350.168843000000000020.1263590.209715000000000010.263916000000000040.2365720.1424320.14730350.27035450.289509000000000020.284296500000000010.2652330.197468499999999990.1476730.205166499999999970.401768999999999990.389155499999999990.277928500000000020.199093999999999990.124736999999999990.157453000000000010.189130499999999980.299486000000000030.29961050.316792499999999980.204504000000000020.30663150.445147000000000010.497605999999999990.453165000000000040.287148500000000030.22198600000000002
Avg Daily Savings per Customer (kWh)
Treatment Estimates
util_typePeriodmodel_typemodelMOdel_nParmEstimateStderrProbZ
Epost1DNDE_DND_SIMPLE1treatment_post1_norm-0.44840.04320.0001
Epost1DNDE_DND_OVERALL2treatment_post1_norm-0.04730.04260.2678
Epost1DNDE_DND_OVERALL_WEATHER3treatment_post1_norm-0.04730.04260.2674
Epost1DNDE_DND_OVERALL_HDD4treatment_post1_norm-0.0470.04260.2704
Epost1PO_E_PO_SIMPLE_NO_CTRL1treatment_post1-1.0990.1270.0001
Epost1PO_E_PO_SIMPLE2treatment_post1-0.53950.04720.0001
Epost1PO_E_PO_SIMPLE_PREINTERACTED3treatment_post1-0.31130.0490.0001
Epost1PO_E_PO_AVGPRENOINT4treatment_post1-0.0130.04290.7624
Epost1PO_E_PO_OVERALL5treatment_post1-0.00120.0450.978
Epost1PO_E_PO_OVERALL_WEATHER6treatment_post1-0.0010.0450.9822
Epost1PO_E_PO_OVERALL_HDD7treatment_post1-0.00120.0450.9793
Epost2DNDE_DND_SIMPLE1treatment_post2_norm0.29140.04870.0001
Epost2DNDE_DND_OVERALL2treatment_post2_norm-0.13190.05710.0209
Epost2DNDE_DND_OVERALL_WEATHER3treatment_post2_norm-0.13240.05710.0203
Epost2DNDE_DND_OVERALL_HDD4treatment_post2_norm-0.13170.05710.021
Epost2PO_E_PO_SIMPLE_NO_CTRL1treatment_post20.39520.13030.0024
Epost2PO_E_PO_SIMPLE2treatment_post20.10990.05330.0393
Epost2PO_E_PO_SIMPLE_PREINTERACTED3treatment_post20.14960.05470.0062
Epost2PO_E_PO_AVGPRENOINT4treatment_post2-0.15980.0620.01
Epost2PO_E_PO_OVERALL5treatment_post2-0.14710.06290.0193
Epost2PO_E_PO_OVERALL_WEATHER6treatment_post2-0.14730.06290.0192
Epost2PO_E_PO_OVERALL_HDD7treatment_post2-0.14690.06290.0195
Gpost1DNDG_DND_SIMPLE1treatment_post1_norm-0.16240.00430.0001
Gpost1DNDG_DND_OVERALL2treatment_post1_norm-0.00920.00360.0106
Gpost1DNDG_DND_OVERALL_WEATHER3treatment_post1_norm-0.00910.00360.0119
Gpost1DNDG_DND_OVERALL_HDD4treatment_post1_norm-0.00910.00360.0113
Gpost1PO_G_PO_SIMPLE_NO_CTRL1treatment_post1-0.15220.01110.0001
Gpost1PO_G_PO_SIMPLE2treatment_post1-0.12280.01080.0001
Gpost1PO_G_PO_SIMPLE_PREINTERACTED3treatment_post1-0.18220.0110.0001
Gpost1PO_G_PO_AVGPRENOINT4treatment_post10.00530.00990.5961
Gpost1PO_G_PO_OVERALL5treatment_post10.00430.010.6636
Gpost1PO_G_PO_OVERALL_WEATHER6treatment_post10.00430.010.6641
Gpost1PO_G_PO_OVERALL_HDD7treatment_post10.00440.010.6631
Gpost2DNDG_DND_SIMPLE1treatment_post2_norm0.06170.00440.0001
Gpost2DNDG_DND_OVERALL2treatment_post2_norm-0.00970.00410.0174
Gpost2DNDG_DND_OVERALL_WEATHER3treatment_post2_norm-0.00960.00410.0187
Gpost2DNDG_DND_OVERALL_HDD4treatment_post2_norm-0.00970.00410.0179
Gpost2PO_G_PO_SIMPLE_NO_CTRL1treatment_post20.18410.01160.0001
Gpost2PO_G_PO_SIMPLE2treatment_post20.16880.01130.0001
Gpost2PO_G_PO_SIMPLE_PREINTERACTED3treatment_post20.13480.01150.0001
Gpost2PO_G_PO_AVGPRENOINT4treatment_post20.00530.01210.665
Gpost2PO_G_PO_OVERALL5treatment_post20.00510.01220.6733
Gpost2PO_G_PO_OVERALL_WEATHER6treatment_post20.00510.01220.6743
Gpost2PO_G_PO_OVERALL_HDD7treatment_post20.00520.01220.6714
Elec Means
E Pre ADC ttest both flags
The TTEST Procedure
Variable: adc
groupNMeanStd DevStd ErrMinimumMaximumgroupMethodMean90% CL MeanStd Dev90% CL Std Dev
CONTROL2391120.180214.47630.09360378.3CONTROL20.180220.02620.33414.476314.368314.5861
TREATMENT4793120.165913.66490.06240437.5GroupAverage Daily Electric Usage (kWh)Difference (Phase 1-Pre)Difference (Phase 2-Pre)TREATMENT20.165920.06320.26913.664913.592713.7379
Diff (1-2)0.014313.94020.1104PrePhase 1Phase 2Diff (1-2)Pooled0.0143-0.16720.195913.940213.8814.001
groupMethodMean90% CL MeanStd Dev90% CL Std DevControl20.18021.38823.0401.2072.860Diff (1-2)Satterthwaite0.0143-0.17080.1994
CONTROL20.180220.026220.334214.476314.368314.5861Treatment20.16621.28322.8831.1172.717
TREATMENT20.165920.063320.268613.664913.592713.7379Difference (T-C)-0.014-0.104-0.157-0.090-0.143
Diff (1-2)Pooled0.0143-0.16720.195913.940213.8814.001
Diff (1-2)Satterthwaite0.0143-0.17080.1994
MethodVariancesDFt ValuePr > |t|
PooledEqual718400.130.89680.6458333333
SatterthwaiteUnequal454140.130.8988
Equality of Variances
MethodNum DFDen DFF ValuePr > F0.36%0.58%000
Folded F23910479301.12 |t|
PooledEqual680340.920.3583
SatterthwaiteUnequal424690.90.3686
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F22537454971.14 |t|
PooledEqual537661.190.2336
SatterthwaiteUnequal332731.160.2469
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F17920358461.18 |t|
PooledEqual58706-1.050.2956
SatterthwaiteUnequal38315-1.040.3
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F19606391001.06 |t|Diff (1-2)Satterthwaite-0.0141-0.03290.00478
PooledEqual77623-1.230.2183
SatterthwaiteUnequal51289-1.230.21970.54%0.48%
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F25856517671.020.099
G Post1 ADC ttest both flags
The TTEST Procedure
Variable: adc
groupNMeanStd DevStd ErrMinimumMaximum
CONTROL247761.67741.24290.0079020.6695
TREATMENT496491.67791.2130.00544020.6873
Diff (1-2)-0.000541.2230.00951groupMethodMean90% CL MeanStd Dev90% CL Std Dev
groupMethodMean90% CL MeanStd Dev90% CL Std DevCONTROL1.67741.66441.69041.24291.23381.2522
CONTROL1.67741.66441.69041.24291.23381.2522TREATMENT1.67791.66891.68691.2131.20671.2194
TREATMENT1.67791.66891.68691.2131.20671.2194Diff (1-2)Pooled-0.00054-0.01620.01511.2231.21781.2283
Diff (1-2)Pooled-0.00054-0.01620.01511.2231.21781.2283Diff (1-2)Satterthwaite-0.00054-0.01630.0152
Diff (1-2)Satterthwaite-0.00054-0.01630.0152
MethodVariancesDFt ValuePr > |t|
PooledEqual74423-0.060.9548
SatterthwaiteUnequal48459-0.060.9552
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F24775496481.05 |t|
PooledEqual58706-1.050.2956
SatterthwaiteUnequal38315-1.040.3
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F19606391001.06 |Z|
Error
Intercept0000..
treatment_post1_norm0.07730.0425-0.16070.0061-1.820.06940.00738750.1472125
treatment_post2_norm0.13360.0571-0.2455-0.0218-2.340.01920.03967050.2275295
yr2014_mo6_norm3.06640.10112.86823.264630.32
-
IEPEC, 2019 Denver
Gas Savings
12
Percentage Savings
0.0090.010
0.000
0.002
0.004
0.006
0.008
0.010
0.012
0.014
Phase 1 Phase 2
Avg.
Dai
ly S
avin
gs p
er C
usto
mer
0.5% 0.5%
0.0%
0.2%
0.4%
0.6%
0.8%
1.0%
Phase 1 Phase 2
Therm Savings
Note: Error bars indicated 90% confidence intervals based on standard errors clustered on customers.
• Gas savings persisted after Xcel Energy stopped sending HBAs• In Phase 2, gas savings per customer who received an HBA equaled
0.011 therms (0.6%)
-
IEPEC, 2019 Denver
Monthly Gas Savings Estimates
13
Note: Error bars indicate 90% confidence intervals based on standard errors clustered on customers.
-0.04
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
Aver
age
Dai
ly S
avin
gs p
er C
usto
mer
(the
rms)
Estimated Savings 90% Lower Savings Estimate 90% Upper Savings Estimate
Phase 1: HBA treatment period
Chart2
Estimated Savings42156421864221742248422784230942339423704240142430424614249142522425524258342614426444267542705427364276742795428264285642887429174294842979430094304043070431014313243160431918.6999999999999994E-36.7999999999999996E-37.4999999999999997E-37.6E-31.26E-21.41E-21.06E-24.1000000000000003E-33.7000000000000002E-38.0999999999999996E-31.0999999999999999E-21.21E-27.4000000000000003E-36.0000000000000001E-31.8E-35.1999999999999998E-38.8999999999999999E-39.5999999999999992E-31.37E-21.2E-29.1000000000000004E-31.0800000000000001E-29.5999999999999992E-31.2800000000000001E-27.9000000000000008E-34.0000000000000001E-31.11E-21.52E-21.43E-21.4500000000000001E-21.61E-21.6500000000000001E-22.3E-31.17E-21.01E-290% Lower Savings Estimate4215642186422174224842278423094233942370424014243042461424914252242552425834261442644426754270542736427674279542826428564288742917429484297943009430404307043101431324316043191-6.4340000000000005E-3-8.992E-3-8.6210000000000002E-3-7.6984999999999996E-31.5785E-36.5329999999999997E-3-3.876000000000001E-3-1.90945E-2-1.3407999999999998E-2-4.5399999999999954E-42.7749999999999997E-3-1.0600000000000002E-3-9.049999999999999E-3-1.09435E-2-1.5307999999999999E-2-1.1085500000000002E-2-3.1085000000000002E-35.5249999999999917E-4-9.4944999999999995E-3-1.2017E-2-7.8434999999999998E-3-1.5374999999999989E-3-2.7000000000000114E-4-1.0179999999999981E-3-9.3724999999999989E-3-1.4095E-2-7.4884999999999969E-3-2.2370000000000011E-33.4429999999999999E-3-4.6949999999999943E-4-1.1207000000000002E-2-1.7715999999999996E-2-2.6322999999999999E-2-5.4079999999999979E-3-7.5700000000000073E-490% Upper Savings Estimate42156421864221742248422784230942339423704240142430424614249142522425524258342614426444267542705427364276742795428264285642887429174294842979430094304043070431014313243160431912.3834000000000001E-22.2592000000000001E-22.3621E-22.2898499999999999E-22.36215E-22.1666999999999999E-22.5076000000000001E-22.7294499999999999E-22.0808E-21.6653999999999999E-21.9224999999999999E-22.5259999999999998E-22.385E-22.2943499999999999E-21.8907999999999998E-22.1485500000000001E-22.09085E-21.8647499999999997E-23.6894499999999997E-23.6017E-22.6043500000000001E-22.3137499999999998E-21.9470000000000001E-22.6617999999999999E-22.51725E-22.2095E-22.96885E-23.2636999999999999E-22.5156999999999999E-22.9469500000000003E-24.3407000000000001E-25.0715999999999997E-23.0922999999999999E-22.8808E-22.0957E-2
Average Daily Savings per Customer (therms)
Treatment Estimates
util_typePeriodmodel_typemodelMOdel_nParmEstimateStderrProbZ
Epost1DNDE_DND_SIMPLE1treatment_post1_norm-0.44840.04320.0001
Epost1DNDE_DND_OVERALL2treatment_post1_norm-0.04730.04260.2678
Epost1DNDE_DND_OVERALL_WEATHER3treatment_post1_norm-0.04730.04260.2674
Epost1DNDE_DND_OVERALL_HDD4treatment_post1_norm-0.0470.04260.2704
Epost1PO_E_PO_SIMPLE_NO_CTRL1treatment_post1-1.0990.1270.0001
Epost1PO_E_PO_SIMPLE2treatment_post1-0.53950.04720.0001
Epost1PO_E_PO_SIMPLE_PREINTERACTED3treatment_post1-0.31130.0490.0001
Epost1PO_E_PO_AVGPRENOINT4treatment_post1-0.0130.04290.7624
Epost1PO_E_PO_OVERALL5treatment_post1-0.00120.0450.978
Epost1PO_E_PO_OVERALL_WEATHER6treatment_post1-0.0010.0450.9822
Epost1PO_E_PO_OVERALL_HDD7treatment_post1-0.00120.0450.9793
Epost2DNDE_DND_SIMPLE1treatment_post2_norm0.29140.04870.0001
Epost2DNDE_DND_OVERALL2treatment_post2_norm-0.13190.05710.0209
Epost2DNDE_DND_OVERALL_WEATHER3treatment_post2_norm-0.13240.05710.0203
Epost2DNDE_DND_OVERALL_HDD4treatment_post2_norm-0.13170.05710.021
Epost2PO_E_PO_SIMPLE_NO_CTRL1treatment_post20.39520.13030.0024
Epost2PO_E_PO_SIMPLE2treatment_post20.10990.05330.0393
Epost2PO_E_PO_SIMPLE_PREINTERACTED3treatment_post20.14960.05470.0062
Epost2PO_E_PO_AVGPRENOINT4treatment_post2-0.15980.0620.01
Epost2PO_E_PO_OVERALL5treatment_post2-0.14710.06290.0193
Epost2PO_E_PO_OVERALL_WEATHER6treatment_post2-0.14730.06290.0192
Epost2PO_E_PO_OVERALL_HDD7treatment_post2-0.14690.06290.0195
Gpost1DNDG_DND_SIMPLE1treatment_post1_norm-0.16240.00430.0001
Gpost1DNDG_DND_OVERALL2treatment_post1_norm-0.00920.00360.0106
Gpost1DNDG_DND_OVERALL_WEATHER3treatment_post1_norm-0.00910.00360.0119
Gpost1DNDG_DND_OVERALL_HDD4treatment_post1_norm-0.00910.00360.0113
Gpost1PO_G_PO_SIMPLE_NO_CTRL1treatment_post1-0.15220.01110.0001
Gpost1PO_G_PO_SIMPLE2treatment_post1-0.12280.01080.0001
Gpost1PO_G_PO_SIMPLE_PREINTERACTED3treatment_post1-0.18220.0110.0001
Gpost1PO_G_PO_AVGPRENOINT4treatment_post10.00530.00990.5961
Gpost1PO_G_PO_OVERALL5treatment_post10.00430.010.6636
Gpost1PO_G_PO_OVERALL_WEATHER6treatment_post10.00430.010.6641
Gpost1PO_G_PO_OVERALL_HDD7treatment_post10.00440.010.6631
Gpost2DNDG_DND_SIMPLE1treatment_post2_norm0.06170.00440.0001
Gpost2DNDG_DND_OVERALL2treatment_post2_norm-0.00970.00410.0174
Gpost2DNDG_DND_OVERALL_WEATHER3treatment_post2_norm-0.00960.00410.0187
Gpost2DNDG_DND_OVERALL_HDD4treatment_post2_norm-0.00970.00410.0179
Gpost2PO_G_PO_SIMPLE_NO_CTRL1treatment_post20.18410.01160.0001
Gpost2PO_G_PO_SIMPLE2treatment_post20.16880.01130.0001
Gpost2PO_G_PO_SIMPLE_PREINTERACTED3treatment_post20.13480.01150.0001
Gpost2PO_G_PO_AVGPRENOINT4treatment_post20.00530.01210.665
Gpost2PO_G_PO_OVERALL5treatment_post20.00510.01220.6733
Gpost2PO_G_PO_OVERALL_WEATHER6treatment_post20.00510.01220.6743
Gpost2PO_G_PO_OVERALL_HDD7treatment_post20.00520.01220.6714
Elec Means
E Pre ADC ttest both flags
The TTEST Procedure
Variable: adc
groupNMeanStd DevStd ErrMinimumMaximumgroupMethodMean90% CL MeanStd Dev90% CL Std Dev
CONTROL2391120.180214.47630.09360378.3CONTROL20.180220.02620.33414.476314.368314.5861
TREATMENT4793120.165913.66490.06240437.5GroupAverage Daily Electric Usage (kWh)Difference (Phase 1-Pre)Difference (Phase 2-Pre)TREATMENT20.165920.06320.26913.664913.592713.7379
Diff (1-2)0.014313.94020.1104PrePhase 1Phase 2Diff (1-2)Pooled0.0143-0.16720.195913.940213.8814.001
groupMethodMean90% CL MeanStd Dev90% CL Std DevControl20.18021.38823.0401.2072.860Diff (1-2)Satterthwaite0.0143-0.17080.1994
CONTROL20.180220.026220.334214.476314.368314.5861Treatment20.16621.28322.8831.1172.717
TREATMENT20.165920.063320.268613.664913.592713.7379Difference (T-C)-0.014-0.104-0.157-0.090-0.143
Diff (1-2)Pooled0.0143-0.16720.195913.940213.8814.001
Diff (1-2)Satterthwaite0.0143-0.17080.1994
MethodVariancesDFt ValuePr > |t|
PooledEqual718400.130.89680.6458333333
SatterthwaiteUnequal454140.130.8988
Equality of Variances
MethodNum DFDen DFF ValuePr > F0.36%0.58%000
Folded F23910479301.12 |t|
PooledEqual680340.920.3583
SatterthwaiteUnequal424690.90.3686
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F22537454971.14 |t|
PooledEqual537661.190.2336
SatterthwaiteUnequal332731.160.2469
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F17920358461.18 |t|
PooledEqual58706-1.050.2956
SatterthwaiteUnequal38315-1.040.3
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F19606391001.06 |t|Diff (1-2)Satterthwaite-0.0141-0.03290.00478
PooledEqual77623-1.230.2183
SatterthwaiteUnequal51289-1.230.21970.54%0.48%
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F25856517671.020.099
G Post1 ADC ttest both flags
The TTEST Procedure
Variable: adc
groupNMeanStd DevStd ErrMinimumMaximum
CONTROL247761.67741.24290.0079020.6695
TREATMENT496491.67791.2130.00544020.6873
Diff (1-2)-0.000541.2230.00951groupMethodMean90% CL MeanStd Dev90% CL Std Dev
groupMethodMean90% CL MeanStd Dev90% CL Std DevCONTROL1.67741.66441.69041.24291.23381.2522
CONTROL1.67741.66441.69041.24291.23381.2522TREATMENT1.67791.66891.68691.2131.20671.2194
TREATMENT1.67791.66891.68691.2131.20671.2194Diff (1-2)Pooled-0.00054-0.01620.01511.2231.21781.2283
Diff (1-2)Pooled-0.00054-0.01620.01511.2231.21781.2283Diff (1-2)Satterthwaite-0.00054-0.01630.0152
Diff (1-2)Satterthwaite-0.00054-0.01630.0152
MethodVariancesDFt ValuePr > |t|
PooledEqual74423-0.060.9548
SatterthwaiteUnequal48459-0.060.9552
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F24775496481.05 |t|
PooledEqual58706-1.050.2956
SatterthwaiteUnequal38315-1.040.3
Equality of Variances
MethodNum DFDen DFF ValuePr > F
Folded F19606391001.06 |Z|
Error
Intercept0000..
treatment_post1_norm0.07730.0425-0.16070.0061-1.820.06940.00738750.1472125
treatment_post2_norm0.13360.0571-0.2455-0.0218-2.340.01920.03967050.2275295
yr2014_mo6_norm3.06640.10112.86823.264630.32
-
IEPEC, 2019 Denver
HBA Pilot Evaluation ApproachFindingsTakeaways and Future Research
14
Agenda
-
IEPEC, 2019 Denver
Takeaways
15
#1: HBAs did not reduce call center call volumes but generated energy savings- Electricity and natural gas savings of about 0.5% during treatment
#2: Gas and electricity savings persisted after HBA treatment ended- Electricity savings increased after Xcel Energy stopped sending HBAs
#3: HBAs have other potential benefits that this evaluation did not assess- Utility customer welfare
• Customers are inattentive• HBAs may reduce costs of tracking consumption, and help customers avoid
billing surprises, regret, and billing arrearages- Customer satisfaction with the utility
-
IEPEC, 2019 Denver
Outstanding Research Questions
16
#1: What are effects of HBAs on customer engagement, satisfaction, and welfare?- Value from alerts- Behavioral changes- Impact on customer satisfaction with utility
Research approach: Customer surveys, willingness-to-pay analysis
#2: How do HBAs generate energy savings? Why do HBA savings persist?- Visits to Home Energy Assessment website?- Participation in Excel Energy efficiency programs?
Research approach: Channeling/joint savings analysis
-
IEPEC, 2019 Denver
Authors
17
Ryan FullemanSenior AnalystCadmus Energy [email protected]
Julie HermanProduct DeveloperXcel [email protected]
Grant Jacobsen, Ph.D.Senior Economist, Cadmus Energy ServicesAssociate Professor, University of [email protected]
Jim Stewart, Ph.D.Principal EconomistCadmus Energy [email protected]
-
IEPEC, 2019 Denver18
Thank You
Questions?
Slide Number 1AgendaSlide Number 3Slide Number 4Slide Number 5Slide Number 6AgendaSlide Number 8AgendaSlide Number 10Slide Number 11Slide Number 12Slide Number 13AgendaSlide Number 15Slide Number 16Slide Number 17Slide Number 18