Application of SAS Software at NSPIpresented by

Voytek GrusPricing Specialist, Marketing Dept.

for

SAS user group, Halifax March 11, 2005

NSPI in brief

• Vertically integrated Power Company under the allowable rate of return regulation:

– 450,000 customers, $1 billion revenues, 1700 employees– Costs of service are subject to regulatory scrutiny.– Need regulatory approval to set prices

• Drivers behind information needs:– Operational and strategic decision making.– Regulatory requirements.– Industry trends:

• deregulation of the industry• Complex pricing arrangements• Energy conservation

SAS application at NSPI: Purpose and resources

• Purpose– To meet analytical information needs of marketing and

regulatory departments through• supporting to cost of service studies.• testing bill impacts of price changes • To meet marketing data mining needs

• Resources– 2 licenses but only one active user.– Products used: Version 8.0, Base

SAS/ACCESS/ASSIST/FSP/GRAPH/STAT– 1 desktop machine, double pentium processor: 864

MHz each, 1 Gigabyte of RAM, 66 GIGs of hard drive memory.

– Pricing specialist spends about 20% of his time working in SAS to provide information.

Philosophy behind SAS applications

• Meet the objective of satisfying company’s information needs in a timely and cost-efficient manner through the means of

• A right balance between SAS and Excel applications.

• Permanent storage of some operational data records in SAS datasets

• Extract value from a careful database design

• Disciplined approach to the code development.

Philosophy behind SAS applications: tap into a symbiotic relationship between SAS and Excel

• Will not shy from moving back and forth between SAS and Excel if it speeds up the process using these two procedures:

• PROC IMPORT OUT= WORK.index3• DATAFILE= "K:\VJG\Projects\Unmetered\

MeteredTrafficLights.xls"• DBMS=EXCEL2000 REPLACE;• RANGE="MetTrafficL"; • GETNAMES=YES;• RUN;

• PROC EXPORT DATA= WORK.test • OUTFILE= "K:\VJG\Rate Case 2005\Bill Impacts\cmesmall.xls" • DBMS=EXCEL2000 REPLACE;• RUN;

Philosophy behind SAS applications: permanent databases

• Maintain several databases:– Monthly billing records:

• Customer consumption records (17 millions)– Has a unique premise index (for the rate design

studies) maintained by the billing dept.

• Customer information records (1 million)

– Hourly consumption data for a few hundred customers on load research sample (6 million)

– Various hourly costs production data (hundred of thousand records)

Database maintenance: reading-in text file records can be a challenge for beginners

RECORDER ID " DATE" " HOUR" " IN" " UN" " KW" " KVAR" " " " " 010210 001201 0100 60 " WT" 83 -48 0 0 010210 001201 0200 60 " WT" 58 -33 0 0 010210 001201 0300 60 " WT" 72 -42 0 0 010210 001201 0400 60 " WT" 58 -33 0 0 010210 001201 0500 60 " WT" 58 -33 0 0 010210 001201 0600 60 " WT" 65 -37 0 0 010210 001201 0700 60 " WT" 50 -29 0 0 010210 001201 0800 60 " WT" 79 -46 0 0 010210 001201 0900 60 " WT" 50 -29 0 0

Directory of C:\Load Research\

08/03/2005 02:48p <DIR> .08/03/2005 02:48p <DIR> ..31/08/2004 08:46a 7,068,578 010110.PRN31/08/2004 08:47a 7,068,578 010210.PRN31/08/2004 08:47a 7,068,336 010410.PRN31/08/2004 08:48a 7,068,578 010910.PRN31/08/2004 08:49a 7,068,094 011110.PRN31/08/2004 08:49a 7,068,578 011310.PRN31/08/2004 08:50a 7,068,578 011510.PRN31/08/2004 08:50a 7,068,820 011610.PRN31/08/2004 08:51a 7,068,820 011710.PRN31/08/2004 08:52a 7,068,578 011810.PRN31/08/2004 08:52a 7,068,578 020110.PRN31/08/2004 08:53a 7,068,578 020210.PRN08/03/2005 02:48p 0 dir.txt 13 File(s) 84,822,694 bytes 2 Dir(s) 17,380,629,504 bytes free

Text files read in in the SAS data step•filename FileList &source; • infile FileList;

Example of test data

structure read in the data

step

Database maintenance: useful code for reading-in text files :

• %let sourcedir=‘C:\Load Research\';• %let source=‘C:\Load Research\dir.txt';• data example;• filename FileList &source; • infile FileList;• length filename_prn $ 12 PRNname $ 89; • input filename_prn 40-51;• PRNname = &sourcedir || filename_prn;• infile PRNname filevar = PRNname end = done firstobs = 2;• do until ( done );• input lrid $ 2-15 date $ 18-23 hour $ 25-26 min

$ 27-28 kwhs_lr 40-50;• keep filename_prn lrid hourlydate kwhs_lr;• output;• end;• run;

•proc expand data=nsr out=nsr from=hour to=hour;

• convert nsr tsr exports / method=join;

• id hourlydate;

•run;

Find these statements very

useful

Will use expand procedure to deal

with missing observations.

Database maintenance: “update”, “index” and “where” statements

• data mf.static;• update mf.static static;• by rc premise account name; • run;

• proc datasets lib=mf;• modify static;• index create premise;• index create rpan=(rc premise account name);• run;

• data ExtractRecord;• set mf.static;• where premise=100;• run;

Use “where” statement to extract a data records for

premise #100

Creates indices for big databases to access records efficiently

Use “update” statement to update database by

key indices.

Philosophy of SAS application at NSPI: invest time in careful database design.

• Database design is a key for an efficient code writing:

– The fewer variables the the better:

– Example of efficiency gain in aggregation:

Voltage Cat. Class Hour KWhsDistribution Resid. 1 12Distribution Resid. 2 14Distribution Resid. 3 57Distribution Resid. 4 15 …. Resid. …Distribution Resid. 1 35Distribution Resid. 2 32Distribution Resid. 3 14 …. ….Transmission Ind. 7 3

HourKWhs/Res/Distr.

KWhs/Comm/Distr.

KWhs/Comm/Trans.

1 13 13 132 12 12 123 8 8 84 8 3 7

5 13 13 136 12 11 127 8 8 88 8 4 39 7 6 9

This design is more

preferable

… than this

proc means data=input sum noprint;

by voltage class hour;

var KWhs;

output out=output sum=;

run;

data aggregate;

set input;

Kwhs_all=KWhs_res_dist+ KWhs_res_comm_ + ...etc;

run;

This data structure lends itself to more efficient processing

Philosophy of SAS usage at NSPI: modular approach to SAS code.

• Delegates specific tasks to SAS programs invoked by %include statement from the master program:

• /*** SECTION I Bottom-Up Calc. Prepare raw load shapes by rate and voltage classes */

• %include 'C:\PROGRAMMS\LSbyRCL1.1 Prepare Accrued Classes.sas';

• %include 'C:\PROGRAMMS\LSbyRCL1.2 Prepare Large Customers.sas';

• %include 'C:\PROGRAMMS\LSbyRCL1.3 Extract NSR.sas';

• %include 'C:\PROGRAMMS\LSbyRCL1.5 Combine and scale to oracle rc shapes.sas';

• %include 'C:\PROGRAMMS\LSbyRCL1.6 Combine rc into vc shapes.sas';

• /*** SECTION II Top-Down Calc. Calibrate line losses by Voltage Classes */

• %include 'C:\PROGRAMMS\LSbyRCL2.1 Calc Line losses by vc.sas';

• etc....

SAS programming philosophy at NSPI: standard naming convention, use of global parameters.

• Extensive use of global parameters (macros)

• %let Juan1=dhms(mdy(9,29,2003),0,0,0); /* adj for the huricane */• %let Juan2=dhms(mdy(10,5,2003),23,0,0);

• …• ….• data nochange Juan_rpl;• set alldata;• if (hourlydate < &Juan1 or hourlydate > &Juan2) then output

nochange;• if (hourlydate >= &Juan1-(&Juan2-&Juan1)-3600 and hourlydate <

&Juan2-(&Juan2-&Juan1)) then output Juan_rpl;• run;

SAS applications: Cost of Service Studies

• NSPI is mandated to provide cost of service studies to the regulator

• Studies determine cost of service to over a dozen customer groups such as residential, commercial industrial rate classes.

• Need to determine hourly consumption patterns (Load Shapes of 8760 hours) by various groups before costs can be allocated.

• Simplified Diagram:

Hourly Load Research Data for 700 customers

(6 million records in MV90 db)Load Research Dept.

Calculates sample weights usingCustomer monthly billing

records.( 5 million recs in SAS db))

Marketing Dept.Calculate Hourly Sales

patterns for 7 rate classes(in MV90 db)

(Load Research Dept)

Econometric Load forecast Projects monthly sales by rate

class.(Excel)

(Marketing Dept)

Calculate hourly sales, losses, requirements

by rate class to arrive at cost allocation stats

such as coincident and non-coincident peak. (SAS)

Allocation of Costs to rate classes(Excel)

(Regulatory Dept)

SAS application: Contribution to Cost of Service Studies

(1) (2) (5) (6) (7) (8) (9) (10)ENERGY CLASS NON- CLASS NON- SYSTEM SYSTEM DEMAND SYSTEM

MW.h LINE COINCIDENT COINCIDENT COINCIDENT COINCIDENT LINE COIN. PEAKSALES LOSSES DMD. (MW) LOAD FACTORFACTOR DMD. (MW) LOSSES DMD. (MW)

( 1 ) DOMESTIC W/O ETS 4,059.7 11.64% 976.6 47.5% 86.7% 846.8 17.50% 995.0

( 2 ) DOMESTIC ETS 70.4 12.06% 40.3 20.0% 21.3% 8.6 18.55% 10.2

( 3 ) DOMESTIC SUBTOTAL 4,130.1 11.64% 1,005.1 46.9% 85.1% 855.4 17.51% 1,005.2

( 4 ) SMALL GENERAL 232.2 11.03% 65.4 40.6% 79.3% 51.8 15.12% 59.6

( 5 ) GENERAL 2,419.5 6.98% 478.9 57.7% 83.4% 399.5 8.54% 433.7

( 6 ) LARGE GENERAL 444.2 5.97% 77.2 65.7% 76.2% 58.8 7.09% 63.0

( 7 ) SMALL INDUSTRIAL 255.2 6.33% 53.1 54.9% 57.7% 30.6 7.60% 33.0

( 8 ) MEDIUM INDUSTRIAL 595.3 5.94% 104.3 65.2% 70.6% 73.6 7.50% 79.1

( 9 ) LARGE IND. FIRM 142.9 4.80% 21.6 75.5% 86.2% 18.6 3.06% 19.2

( 10 ) INTERRUPTIBLE 873.6 3.79% 132.4 75.3% 77.3% 102.4 3.05% 105.5

( 11 ) LARGE IND. SUBUTOTAL 1,016.5 3.93% 149.4 77.7% 81.0% 121.0 3.05% 124.7

( 12) MUNICIPAL 195.6 4.98% 39.7 56.3% 99.2% 39.4 5.33% 41.5

( 13 ) UNMETERED 109.0 10.98% 24.3 51.2% 71.2% 17.3 18.74% 20.6

SAS Applications: Customer Bill Impacts

• Changes in prices are subject to regulatory approval and need to be communicated to NSPI Management, the UARB and customers of NSPI.

Revenue Deficiency

(Finance Dept. )

Allocation of costsamong customer classes

(Regulatory Dept. )

Allocation of cost responsibilities s

among customer classes(Marketing Dept. )

Rate Design: Allocation of cost responsibilities

among individual customers(Marketing Dept. )

Will test bill impacts of proposed rate structure on all customers using SAS

(Marketing Dept. )

Are bill impacts acceptable to NSPI, UARB, Customers?

No

Yes

Submit new tariffs

Give Customers

access to bill impacts on the

internet

Bill Impacts – an example of final Output for the UARB

MASS-MARKET CLASSES Ave. Use CURRENT $/MWh PROPOSED ANNUAL BILL ANNUAL BILL Tested Unit

Accounts MWhs in KWhs ANNUAL BILL ANNUAL BILL VARIANCE % VARIANCE Revenue

Residential Seasonal Stratum 1 (lt 1MWh) 13,545 5,332 394 $141 $359.21 $145 $3 2.40% $367.83 Stratum 2 (1 MWh to 2.5 MWh) 7,757 12,308 1,587 $246 $154.94 $260 $14 5.56% $163.56 Stratum 3 (2.5 MWh to 5 MWh) 3,192 11,106 3,479 $410 $117.70 $440 $30 7.32% $126.32 Stratum 4 (5 MWh to 8 MWh) 1,385 8,740 6,311 $654 $103.60 $708 $54 8.32% $112.22 Stratum 5 (8 MWh to 14 MWh) 1,060 10,938 10,319 $999 $96.80 $1,088 $89 8.91% $105.42 Stratum 6 (14 MWh to 22 MWh) 390 6,630 16,999 $1,575 $92.65 $1,721 $147 9.30% $101.27 Stratum 7 (22 MWh to 32 MWh) 95 2,411 25,380 $2,297 $90.49 $2,515 $219 9.53% $99.11 Stratum 8 (32 MWh to 50 MWh) 33 1,278 38,722 $3,446 $88.98 $3,779 $334 9.69% $97.60 Stratum 9 (50 MWh to 80 MWh) 5 298 59,536 $5,231 $87.86 $5,744 $513 9.81% $96.48 Stratum 10 (gt 80 MWh) 1 162 162,000 $14,057 $86.77 $15,454 $1,396 9.93% $95.39 Total 27,463 59,203 2,156 $294 $136.48 $313 $19 6.32% $145.10

Residential Non-All Electric Stratum 1 (lt 1MWh) 10,482 3,982 380 $152 $401.11 $156 $3 2.15% $409.73 Stratum 2 (1 MWh to 2.5 MWh) 26,546 46,786 1,762 $277 $157.44 $293 $15 5.48% $166.06 Stratum 3 (2.5 MWh to 5 MWh) 57,725 213,914 3,706 $447 $120.52 $479 $32 7.15% $129.14

Bill Impacts – Final Output for NSPI customers available on the NSPI website in 2002.

B i l l C a l c u l a t o r

U s e o u r o n l i n e c a l c u l a t o r t o f i n d o u t h o w t h e n e w r a t e s w i l l a f f e c t y o u r p o w e r b i l l .

P l e a s e e n t e r y o u r 8 - d i g i t a c c o u n t n u m b e r a n d t h e m e t e r

n u m b e r f r o m y o u r b i l l s t a t e m e n t i n t h e fi e l d s b e l o w , t h e n

p r e s s " C a l c u l a t e " .

A c c o u n t # - M e t e r #

W h e r e t o F i n d Y o u r A c c o u n t & M e t e r N u m b e r s o n Y o u r B i l l

P a p e r B i l l E l e c t r o n i c B i l l

Bill Calculator

As of November 2002, this account is billed monthly on our Equal Billing Plan:

Your Equal Billing amount is:

Your new payment amount will be about:

$59 (Old rates, excluding HST)

$61 (New rates, excluding HST)

This is a change of:

$2

To check another account click here.

The amounts shown are estimates provided for information only and are

not guaranteed to be accurate, and impacts may vary. Changes to

Equal Billing amounts reflect average impact by rate class. Individual

results may vary.

Back to top

If you have any questions, please talk to us over the Internet or you can

call 1-800-428-6230 (428-6230 in Metro Halifax). Hearing impaired

customers can call our TDD number: 1-800-565-6051.

Screen #1: enter personal info. Screen #2:

customized bill impact.

Customer Bill Impacts – programming solutions

• To simulate bills use pointer technique to keep the size of work-in-progress data files at a minimum:

• data M_&method&type&scenario /*(keep=rc premise year month simrev);*/

• (drop= code min_bill ratecode montht yeart pos tc • base_ch demand_on_ch demand_off_ch kwh_blk1_ch);

• set dets_&type&scenario;• %include 'C:\Programms\Bill Simulation 1.2.1 Calc Bill.sas';• pointer=((year-&ftyear)*12+month-1)*32+pos;• set tariffs.M_&scenario point=pointer;

• if index(bf,'T')=0 then tc=0;else tc=&toc;

• if rc in (2 3) and index(bf,'P')> 0 then

select;

when (rc=2) pos=1;

when (rc=3) pos=2;

when (rc=4) pos=3;

when (rc=5) pos=4;

when (rc=6) pos=5;

when (rc=7) pos=6;

when (rc=8) pos=7;

when (rc=9) pos=8;

when (rc=10) pos=9;

Etc …

Customer Bill Impacts – programming solutions

• To calculate accrued monthly records will use arrays and will split datasets into smaller subsets to speed up processing of files which otherwise could reach undesirable size of dozens of millions of records.

• data accrsort1(drop=i) accrsort2(drop=i); • /* Rearrange data in preparation for aggregation */• set billed_partitioned;• array kwhsblk1{6} kwhsblk11-kwhsblk16;• array kwhsblk2{6} kwhsblk21-kwhsblk26;• etc ...• do i=1 to 6;• year=year(datex{i});• month=month(datex{i});• kwhs_blk1=kwhsblk1(i);• kwhs_blk2=kwhsblk2(i);• etc ...• if i <=3 then output accrsort1; • else output accrsort2;• end; • run;

Price Studies: Finding a time pattern for pricing structures.

• Use a “by group” processing feature of graph and regression procedures to efficiently analyze data.

Average Daily Profile (year=2003 season=Summer daytype=Business Day) (Rsqr=0.77, 300 obs)

0

200

400

600

800

1000

1200

1400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Hour-ending

MWs

$0

$10

$20

$30

$40

$50

$60

$70

$80

$90

$100

$/MWh

NSR NSR est. Marginal Cost MC Estimate

Price Studies: Finding a time pattern for pricing structures.

• Example of a “by group” processing feature of a Regression procedure

• ….• %let regress_by=year month daytype;

• proc sort data=input_data out=output_data;• by &regress_by;• run;• proc reg data=output_data outest=regstats tableout;• by &regress_by;• m1: model NSR = mid_peak1 on_peak1;• m2: model NSR = on_peak2;• m3: model mc = on_peak2;• run;• data m1_coeff(keep=&regress_by intercept mid_peak1 on_peak1

rename=(intercept=intercept1 mid_peak1=mp1c on_peak1=op1c))• m2_coeff(keep=&regress_by intercept on_peak2 rename=(intercept=intercept2

on_peak2=op2c)) • m3_coeff(keep=&regress_by intercept on_peak2

rename=(intercept=intercept3 on_peak2=op3c));• set regstats;• if _type_='PARMS' and _model_='M1' then output m1_coeff;• if _type_='PARMS' and _model_='M2' then output m2_coeff;• if _type_='PARMS' and _model_='M3' then output m3_coeff;• run;• …

Employs SAS graph capabilities to assist Large Customers in their planning process.