Mat-2.108 Independent Research Project in Applied Mathematics

Temporal aggregation and smoothing of sales forecasts

May 12, 2007

Ilkka Kujamäki 58574U

Helsinki University of TechnologyDepartment of Engineering Physics and Mathematics

Systems Analysis Laboratory

Contents1 Introduction 2

2 Background 22.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Aggregation of time series 5

4 Measures of forecast accuracy 74.1 Standard error measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5 Smoothing of timing differences 95.1 Pair-wise DTW algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.2 Forecast smoothing using linear programming . . . . . . . . . . . . . . . . 14

6 Results 166.1 Simulated sales and forecasts . . . . . . . . . . . . . . . . . . . . . . . . . . 166.2 Real data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

7 Discussion 22

References 24

A Implementation of the pair-wise smoothing using XPress-MP 25

B Implementation of the LP smoothing using XPress-MP 27

1

1 Introduction

A fundamental requirement for a functioning supply chain is predictability. This pre-dictability is usually gained using sales forecasting, and judgemental forecasts are thepreferred method for sales forecasting in most industries. The accuracy of these fore-casts impacts the entire supply chain, which is critical to profitability in many industries.Therefore, forecast accuracy is a pivotal issue. The accuracy of sales forecasts can beconsidered using two dimensions: the volume and timing of sales.

The effect of timing and volume errors of forecasts can be different depending on theapplication. For example, in capacity planning, the priority is the accuracy of sales volumeestimates, not the exact timing of sales. Therefore, the forecasts are usually establishedfor an aggregated time period (e.g., month). However, despite temporal aggregation,intermittent demand can cause problems for the measurement of forecast accuracy. If thetiming of individual orders impacts the aggregated sales, aggregation does not smooththe forecasts as expected: the forecast accuracy is still sensitive to the timing of sales atthe boundary of aggregation periods.

In addition to the aggregation and smoothing problem, it is challenging to define agenerally applicable forecast error measure. Several measures have been suggested in theliterature, but the consensus is that no single measure is generally applicable. The mainissue is the commensurability of forecasts for time series with different characteristics. Forexample, to compare the forecast accuracy for time series of different scale, the measureshould be scale invariant. However, most scale-invariant measures, such as the ones basedon percentage errors, are not defined for observations of zero value.

In this study, the forecast accuracy of judgemental forecasts is considered from the per-spective of supply chain management and capacity management. For these applications,the most important parameter related to forecasts is the volume of sales for a certain timeperiod. Therefore, instead of simple temporal aggregation, forecast accuracy should bemeasured by comparing the forecast to actual sales using a moving time window of speci-fied length. This approach would also reduce the need for product-based aggregation andthus the forecasts could be analyzed in more detail and critical sources of improvementcould be identified.

2 Background

The motivation for this study arises from the problem of forecast accuracy measurementfor the needs of supply chain management (SCM) of a large process industry company.The company uses sales forecasts inter alia for SCM and capacity planning. To optimisethe production processes and capacity usage, capacity planning is used to allocate the

2

orders to production units by creating allocations based on the forecasted sales. Mainly forpractical purposes, sales are forecasts for aggregated time periods and product categories.Similar products are categorized to planning items, whose demand is forecasted monthly.In other words, the forecasts and actual sales are aggregations of daily and product timeseries. The forecast horizon is two periods; that is, in month t, the forecast is given formonth t+2. The outline of the forecasting and capacity management process is following:

1. Salesmen forecast the monthly demand of their customers.

2. Using these forecasts, a group-level consensus forecast is generated.

3. Capacity is allocated for forecasted and confirmed orders, resulting in a feasiblesupply plan.

The accuracy of sales forecasts impacts significantly the efficiency of the supply chainand capacity usage (Caniato et al., 2005). However, the capacity management is moresensitive to the volume than to the exact timing of sales. In practice, it is not optimalto produce the orders just in time, but to combine them to an economical batch size.Thus, a difference of a few days in the timing of forecast is not critical. The effectsof misestimated timing of sales, such as increased cost of working capital and excessinventory, are less critical than the effects of deviations in the volume of sales. Conversely,wrongly estimated sales volumes can result in suboptimal capacity usage and to lostsales. Therefore, insignificant errors in the timing of sales can make a good forecast lookinaccurate. This inconsistency can demotivate the forecasters and distract the focus fromthe most critical issue: the estimation of sales volume.

Therefore, additional means are needed to properly smooth the forecast and actualsales at the aggregation boundaries. The problem of timing differences in forecastinghas gained little attention in the literature. Nevertheless, Kerkkänen et al. (2006) havesuggested a smoothing algorithm to filter the timing difference from the forecast timeseries. Unfortunately, the suggested algorithm is not robust, as shown in Section 6.

2.1 Problem statement

The objective of this research is to develop a method for comparing the forecast errorfor time series with different characteristics (e.g., mean, distribution and spectrum) thatignores timing errors of forecasts smaller than specified periods. The error measure shouldbe scale-invariant, and defined for intermittent demand (zero values of forecasts or sales).

The main challenge is accounting for the differences in the timing of sales at theaggregation boundary. A common pattern in the data is that occasionally sales forecastedfor period t occur at period t− 1 or t+ 1. Due to the aggregation, this can correspond to

3

timing difference of 1 to T days, where T is the aggregation period. However, it is seenthat forecast accuracy should be evaluated by comparing the forecast and sales volumeby allowing a timing difference smaller than τ days.

The research problem can be stated as how the forecast time series should be smoothedto eliminate the deviations caused by random differences in the timing of sales? In additionto developing a method for smoothing the timing differences, we discuss the suitabilityof commonly used forecast accuracy measures for the evaluation of judgemental salesforecast with smoothing and aggregation.

2.2 Data

The data used in the study consist of time series of monthly forecasts and observed salesfor customers and one planning item (a group of similar products). The data includesalso a breakdown of forecasts and sales to forecasts that did not actualise and sales thatwere not forecasted for each period. In addition, weekly and daily time series of saleswere available for some customers. The sample includes 120 customers for which the totalsales of ranges from 6 to 175174 units. All the records date from October 2005 to March2007.

The distribution of daily and monthly sales varies substantially between customers.For most large customers, the daily sales are small compared to aggregated monthly sales,but for some customers the demand is very intermittent; that is, it consists of few largeorders. The share of observation with zero value is 67%, thus the demand is on averagesignificantly intermittent.

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

forecastactuals

Figure 1. Example of monthly forecast and sales data.

4

3 Aggregation of time series

Forecasting is usually based on temporal aggregation of the time series. The selection ofaggregation level (e.g., time or product) depends on the Purposes use of forecasts andhas an impact on the forecast performance (Zotteri et al., 2004). In sales forecasting, thesales are normally recorded daily, while the forecasts consider a longer time period (e.g,weekly or monthly sales). Some level of aggregation is generally necessary particularly forjudgemental forecasting, because it increases the signal to noise ratio of the time series.For instance, while the daily fluctuations of demand may appear random, the aggregatedmonthly demand may follow a clear pattern.

If the daily demand is very intermittent and individual orders are large compared tothe monthly demand, the forecast error calculated from aggregated sales and forecast mayappear random. For example, if a large order is forecasted to occur at the end of period t,but it actualises in period t+1, comparing forecast and actual orders time series is difficult(Figs. 2 and 1). The uncertainty of the timing of events is usually circumvented usingaggregation, although it renders the evaluation of forecast accuracy difficult for bottom-upforecasts. Temporal aggregation also affects the properties of time series, which should betaken into account when making decisions based on aggregated values (Marcellino, 1999).

Moreover, aggregation does not fully eliminate the fluctuations caused by randomdifferences in the timing of sales. If this is a relevant problem, depend on the effect oferrors of different a type. If a timing error of a few days is not critical, the traditional wayof calculating the forecast error as the Euclidean distance of observation gives a biasedview of forecast accuracy even for aggregated time series.

0 5 10 15 20 25 30 35 40 45 500

500

1000

1500

2000

2500

forecastactuals

Figure 2. Comparing the forecasted and actual order for intermittent demand.

5

Alongside the economical effects of forecast errors of a different type, the significanceof the aggregation problem depends on the distribution of demand. If the distribution ofdaily of demand D is cumulated at 0 (where D is a random variable), the aggregationdoes not smooth the random errors (Fig. 2).

In addition, large values of daily demand compared to aggregated demand {x̃}, amplifythis effect. To illustrate the phenomenon, the monthly aggregated demand correspondingto the different type of daily demand is presented in Figure 3. The aggregated demandcan be very different depending on the distribution of daily demand, and in addition,different distributions can result in similar aggregated demand.

100 300 500 700 900 1100 1300 1500 1700 19000

2

4

6

8

10

12x 10

5 Daily demand

200 600 1000 1400 1800 2200 2600 3000 3400 38000

0.5

1

1.5

2

2.5

3x 10

4 Aggregated demand

(a) Poisson(0.01)

350 1050 1750 2450 3150 3850 4550 5250 5950 66500

1

2

3

4

5

6

7x 10

5 Daily demand

0.36 0.68 1 1.32 1.64 1.96 2.28 2.6 2.92 3.24

x 104

0

2000

4000

6000

8000

10000

12000

14000Aggregated demand

(b) Poisson(0.5)

0 200 400 600 800 1000 12000

1

2

3

4

5x 10

5 Daily demand

1.25 1.3 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75

x 104

0

2000

4000

6000

8000

10000

12000Aggregated demand

(c) N(500, 100)

400 410 420 430 440 450 460 470 480 490 5000

2

4

6

8

10

12x 10

4 Daily demand

1.28 1.3 1.32 1.34 1.36 1.38 1.4 1.42

x 104

0

2000

4000

6000

8000

10000

12000Aggregated demand

(d) Uni(400,500)

Figure 3. Distribution of monthly (aggregated) demand corresponding to different dis-tributions of daily demand.

The sensitivity of aggregated demand to small deviations in the forecasted and actualtiming of sales can be illustrated by simulating the demand and forecasts. Let the dailydemand be a random variable D ∼ Dist(λ) from which the aggregated demand {D̃} iscalculated, and let {D̃s} be a time series in which the timing of sales is randomized bys ∼ N(0, σt) days. The Euclidean distance of {D̃} and {D̃s} represents the sensitivity of

6

aggregated demand to changes in the timing of sales. The sensitivity for three distribu-tions of daily demand is illustrated using histograms of differences aggregated demand inFigure 4. On x-axis is the relative difference of demands ((D − Ds)/D), and on y-axisthe frequency of the deviations. In Figure 4 (d), D is uniformly distributed between400 and 500 units, and in figure (c) D ∼ N(500, 100). In such settings, the differenceare small compared to the average aggregated demand. However, If the daily demand ismore intermittent, for example follows Poisson distribution with parameter λ < 0.5, thedeviations in aggregated demand can be large, as illustrated in figures (a) and (b).

−7 −6 −5 −4 −3 −2 −1 0 1 2 3 4 5 6 70

0.5

1

1.5

2x 10

4 Sensitivity

(a) Poisson(0.01)

−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.80

2000

4000

6000

8000Sensitivity

(b) Poisson(0.5)

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.30

2000

4000

6000Sensitivity

(c) N(500, 100)

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.30

2000

4000

6000

8000Sensitivity

(d) Uni(400,500)

Figure 4. Sensitivity of the aggregated demand to the timing of orders (the relativedifference in aggregated demand caused by the random component of N(0,3) days in thetiming of sales).

The daily or monthly sales in the data do not follow well any standard distribution;however, for most customers they are similar to the one presented in figures (a) and (b).Thus, the sensitivity to the timing of sales can result in random deviations of forecastaccuracy and identifying wrong sources for accuracy improvement. For example, if theinaccuracy of forecasts is due to small errors in the timing of sales, it is not efficient touse resources to improve the timing of forecasts. Similarly, the forecasters can considermeasuring such errors unfair and useless, which is not likely to improve the accuracy ofthe judgemental forecasts.

4 Measures of forecast accuracy

The aggregation and smoothing do not solve all the problems related to the measurementof forecast accuracy. In addition to smoothing, a proper measure for forecast error isneeded. Four measures of forecast accuracy are classified by their mathematical propertiesbelow. To compare forecasting methods and forecasts, an important property is scale

7

invariance. For scale-dependent measures, the measure depends on the values of the timeseries, and thus such measures are useful mostly in comparing forecasts for the same data(or of the same scale).

Other commonly used properties are related to favourable properties of forecasts.First, the forecasts should be unbiased and efficient∗; that is, without systematic errorand more accurate than a naïve forecast Ft+1 = Yt. The bias and efficiency of forecastcan be tested by fitting to forecasts a regression model of form

et = β0 + β1et−1, (1)

where et is the relative error of forecast (et = (Yt − Ft)/Yt) (Lawrence et al., 2000). Forunbiased forecasts, β0 is zero, and for efficient forecasts, β1 is zero. Second, the forecastshould be consistent. In other words, a forecast Fτ given in period t+ 1 should be moreaccurate than the forecast given in period t.

4.1 Standard error measures

Several sophisticated methods for assessing forecast accuracy have been presented in theliterature (see e.g., Hyndman & Koehler (2006)). However, the measures used in practiceare seldom as sophisticated as the ones suggested by academicians. Many commonly usedmeasures, such as mean absolute percentage error (MAPE), have shortcomings that canrender the comparison of forecasts difficult or misleading.

As the objective of this study is to compare the accuracy of forecasters and forecasts,the measure should be suitable for comparing time series with different characteristics,such as one presented in Figure 1. In addition, because the data consist of a largeproportion of zero values, the measure should be defined also for intermittent demand.Founding on a comprehensive review of forecast accuracy by Hyndman & Koehler (2006),four measures presented in Table 1 were selected for evaluation.

qt = Yt − Ft1n−1∑ni=2 |Yi − Yi−1| (2)

MSE and MAE are both scale dependent but still widely used due to their intelligibility,and both are useful comparing the forecasts for the same dataset. MAPE is scale invariant,but it is not defined for Ft = 0 and has a skewed distribution if Ft is small. MAPE is alsoasymmetric as it gives larger values for positive errors (Ft > Yt) than for negative errors.

∗Although judgemental forecasts are often biased and inefficient, statistical forecast models do notnecessarily give better estimates (Lawrence et al., 2000). While statistical methods are good for repro-ducing the historical patterns of time series, the fluctuation of sales are usually caused by events whichcannot be accounted in these models. Instead, the information needed to predict these events is frequentlyavailable for judgemental forecasters.

8

Table 1. Measures of forecast acuracyMeasure Definition

Mean Square Error (MSE) mean((Ft − Yt)2)Mean Absolute Error (MAE) mean(|Ft − Yt|)Mean Absolute Percentage Error (MAPE) mean(|100(Ft − Yt)|/Yt)Mean Absolute Scaled Error (MASE) mean(|qt|), where qt is as in Eq. (2).

MASE, a robust measure suggested by Hyndman & Koehler (2006), is not widely usedbut has some favourable properties: MASE is scale invariant and defined for all values ofFt and Yt.

5 Smoothing of timing differences

In practice, some level of temporal aggregation is needed for forecasting. Aggregationreduces the noise (random errors) of time series and is usually needed due to the practicallimitation of judgemental forecasting process. Moreover, considering the efforts and re-source needed to compose the monthly forecast, it would be neither realistic nor practicalto do the forecasts daily. However, the aggregation is not sufficient to smooth intermittentdemand.

A straightforward approach for smoothing forecasts is comparing them to actual salesof adjacent periods. This approach is feasible only if the forecast horizon is over one period;otherwise even a naïve forecast Ft = Yt−1 would have a zero forecast error. A similarapproach is employed by Kerkkänen et al. (2006), who suggest smoothing the forecasterror E (MAE) by following algorithm (referred hence as SMA) (Kerkkänen et al., 2006,s.9-10):

EA = (Fn−1 + Fn)− (Sn + Sn+1) (3)

EB = (Fn + Fn+1)− (Sn−1 + Sn) (4)

EC = Fn − Sn (5)

E = min {|EA|, |EB|, |EC |} . (6)

Unfortunately, this algorithm is not robust as it gives misleading results for some specialcases of sales and forecast patterns. For example, for a forecast (40, 45, 50, 200, 50, 60)and the actual sales (50, 30, 200, 50, 200, 60, 50), the algorithm gives errors (15 20 0 1010), resulting in total error of 55, while the difference of total forecasted and actual salesis -145.

9

The problem arises from the definition of EA and EB: the same forecast Fn can beused to match the sales of periods n − 1 and n + 1. Due to this, the measure can showa small value although the total forecasted and actual sales differ substantially. Thisinconsistency is problematic because a forecast error of total sales volume can result ininefficient use of capacity, lost sales or excess inventories. Thus, an alternative approachis needed for smoothing for the needs of capacity planning.

Dynamic time warping (DTW) is a commonly used generic method for comparing timeseries which vary in the time alignment (Sakoe & Chiba, 1978). As the name suggests,DTW allows shifting of the time axis of the time series. This can be useful for comparingthe forecast and actual sales time series, because, as already pointed out, they typicallycan differ both in amplitude (volume of sales) and the time alignment (timing of sales), asillustrated in Figure 5. Although the time series appear similar expect for the alignmentin the time axis, their Euclidean distance is large. DTW allows that point ft of timeseries f1, f2, . . . , fn is matched to point(s) at−τ , . . . , at+τ ′ , where τ and τ ′ are determinedby warping constraints.

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5 6 7 8 9 10 11

(a) Euclidean distance

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5 6 7 8 9 10 11

(b) DTW distance

Figure 5. The Euclidean and DTW distance of two time series.

However, the algorithm DTW as such is not suitable for calculating the forecast erroror smoothing the forecast time series, because it allows the same forecast to be matchedto multiple periods, which leads to a measure inconsistent with the total difference offorecasts and actual sales. Moreover, it allows a systematic misalignment of forecasts andsales, which is not acceptable for sales forecasts. Two approaches to solve this problemare presented. The first one, a modification of DTW algorithm, is suitable for situationsin which the demand is highly intermittent, and the aggregated demand consists mainlyof individual orders. The latter approach is more specific for the forecast applicationbecause it smoothes the forecasts by distributing the volume exceeding the actual salesto adjacent periods.

10

5.1 Pair-wise DTW algorithm

To solve the problem of multiple matches in standard DTW-algorithm, the constraintsof the algorithm should be modified to prevent matching of forecasts (or actual sales) tomultiple periods. The forecasts are allowed to be matched to the actual sales of previous,current or the next period pair-wise. That is, if the forecast ft is matched to actual salesat−1 or at+1, the forecast of that period is matched to the actual sales at (Fig. 6). Thepair-wise matching is selected so that total distance of time series is minimised. Thisapproach is justifiable if the sales of each period consist of one order, or the fluctuationsin sales are caused by a single large order. The minimisation of time series distanceimplicitly assumes that the differences in the forecast and actual sales are caused mainlyby the timing of the sales.

ft−1

ft

ft+1

at−1

at

at+1

Figure 6. Illustration of pair-wise matching. Each line type corresponds to possible pairof matches.

The DTW matching is usually calculated using dynamic programming. Conversely, thepair-wise matching is easier to implement using linear programming. First, the distancesdi of the forecast ft and actual sales of periods t− 1, t and t+ 1 are calculated (Fig. 6).

11

Then, the sum of these distances is minimised over the time series using the constraintsdescribed in the paragraph above:

minN∑

i=1xidi (7)

s.t. x1 + x2 = 1 (8)

x3k + x3k+1 + x3k+2 = 1 ∀ k ∈ [1, T − 2] (9)

x3T−1 + x3T−2 = 1 (10)

x1 + x3 = 1 (11)

x3k+2 + x3k+4 + x3k+6 = 1 ∀ k ∈ [0, T − 3] (12)

x3T−2 + x3T = 1 (13)

xi ∈ {0, 1}, (14)

where T is the length of time series, xi is the decision variables representing the matchingand di the distance of sales and forecasts defined as

d1 = |f1 − a1|d1 = |f1 − a2|+ εd3t = |ft+1 − at|+ ε ∀t ≤ Td3t+1 = |ft+1 − at+1|+ ∀t ≤ Td3t+2 = |ft+1 − at+2|+ ε ∀t ≤ T − 1

The variable ε is used to ensure that the LP algorithm selects matches the forecast ofperiod t to sales of period t, if it has the difference as matching to t− 1 or t+ 1.

However, if information about forecasts that did not actualise and sales that were notforecasted is available for each period, the smoothing can be applied to these time series.This approach partly eliminates the interference of multiple orders, because for a timingdifference less than one period, forecasts that did not actualise in period t appear as non-forecasted sales in period t−1 and t+1. The smoothed forecast can be calculated by firstsubtracting from forecasts the non-actualised forecasts (Fna) and adding non-forecastedsales (Snf), and calculating the pair-wise smoothing for Fna and Snf . The smoothedforecast is then obtained by subtracting the pair-wise matched forecast and sales fromthe reduced forecast, as illustrated in Table 2. The difference of forecasts errors (ft − at)for unsmoothed, smoothed as well as for smoothed unactualised forecasts and unpredictedsales (corresponding to the data presented in Figure 7) is illustrated in Figure 8. The lastapproach gives the smallest MAE (56960) because other orders do not interference the

12

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Figure 7. An illustration of matching the smoothed forecast with actual sales.

pair-wise matching. For comparison, the MAE is 68835 for standard pair-wise matchingand 96047for unsmoothed forecasts.

Table 2. Smoothing of forecasts using information of non-actualised forecast (Fna) andnon-forecasted sales (Snf).

PeriodVariable 1 2 3 4 5 6 7 8

F 1200 20000 1200 7200 4000 600 4800 9000A 3200 6800 17200 1700 3100 0 1700 5200Fna 600 13200 0 3300 1400 600 3200 3800Snf 1500 0 16000 600 500 0 0 0

ΔF = (F − Fna) + Snf 2100 6800 17200 4500 3100 0 1600 5200E = Fna − Snf 810 0 2740 -847 -2757 -3150 -600 -3945

Fs = ΔF − E 1290 6800 14460 5347 5857 3150 2200 9145

The pair-wise matching can also be implemented using dynamic programming, whichcan be necessary for practical implementation if a LP solver cannot be integrated intothe existing system. The dynamic programming algorithm can be implemented withany programming language that supports recursion (virtually all do), which makes iteasy to integrate into existing systems. Similarly as in the DTW algorithm, the pair-wisematching that minimized the distance can be solved by considering the alignment problemas a graph. The structure of the graph, however, is more complicated because the allowedsteps depend on from which node the current node is reached. In addition, the problem isnon-serial, because the choice made in node i affects the choices possible in nodes i . . . n.

13

-15000

-10000

-5000

0

5000

10000

15000

20000

25000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

No smoothingPair-wise smoothing of forecasts

Pair-wise smoothing of forecast errors

Figure 8. Comparison of forecast error for different type of pair-wise smoothing.

5.2 Forecast smoothing using linear programming

The pair-wise matching is not suitable if the aggregated demand consists of several smallorders. The constraint of the pair-wise algorithm that each forecast is matched to exactlyone actual sales is too restrictive, because the timing can differ for a subset of the ordersonly. Moreover, the data suggests that in practice an order forecasted for period t mayactually distribute over periods t − 1 and t + 1. An alternative approach for smoothingis to divide the forecasts into three components xt−1,3, xt,2 and xt+1,1 representing thedistribution of the forecast over the periods t− 1, t and t + 1. The smoothed forecast isthen calculated by minimising the forecasts error et = xt,1 + xt,2 + xt,1 − at, where at isthe actual sales for period t.

From the aggregated time series it is not possible to infer how the aggregated valuesare composed from individual orders or how the orders are distributed inside aggregationperiod. Therefore, we cannot know if the opposite deviations in forecasts and sales ofconsecutive periods are actually caused by the timing of orders or differences in fore-casted volume. However, the data suggest that most of these patterns are caused by adifference in the timing of sales, while the volume of sales is estimated accurately. Thus,the minimisation of the forecast error seems plausible.

The minimisation problem is solved using linear programming. That is, the sum ofabsolute values of forecasts error |et| is minimised for t < T . The distribution of forecast

14

ft (xt−1,3 + xt+1,1) is constrained to be smaller by the excess forecast |ft − at|, and thesum of distribution is restricted to equal the forecasts:

minT∑

t=1zt (15)

s.t. (1 + ε)xt,1 + xt,2 + (1 + ε)xt,3 − at ≤ zt ∀ t ∈ [2, T − 1] (16)

−[xt,1 + xt,2 + xt,3 − at] ≤ zt ∀ t ∈ [2, T − 1] (17)

xt−1,3 + xt,2 + xt+1,1 = ft ∀ t ∈ [2, T − 1] (18)

x1,2 + (1 + ε)x1,3 − a1 ≤ z1 (19)

−[x1,2 + (1 + ε)x1,3 − a1] ≤ z1 (20)

(1 + ε)xT,1 + xT,2 − aT ≤ zT (21)

−[(1 + ε)xT,1 + xT,2 − aT ] ≤ zT (22)

x1,1 = 0 (23)

x1,2 + x2,1 = f1 (24)

xi−1,3 + xi,2 + xi+1,1 = ft ∀ t ∈ [2, T − 1] (25)

xT,3 = 0 (26)

xT−1,3 + xT,2 = fT (27)

x2,1 ≤ max{f1 − a1, 0} (28)

xi−1,3 + xi+1,1 ≤ max{fi − ai, 0} ∀ t ∈ [2, T − 1](29)

xT−1,3 ≤ max{fT − aT , 0}, (30)

where at is the actual sales and ft the forecast for period t. The variable zt and theconstraints (18)-(24) are used to model the non-linear objective function |xt,1 + xt,2 +xt,3−at|. The constraints (25)-(29) ensure that the distributed forecast equals the forecastof each period, and the constraints (30)-(32) make sure that only the excess amountof forecast is distributed to consecutive periods (this is necessary in order to limit thesmoothing to one period). The variable ε of an arbitrary small value forces the algorithmto prefer perpendicular matching (i.e., ft is matched to at).

The constraints (30)-(32) also prevent smoothing of patterns in which the timingof sales differs sequentially. For example, in time series f = (2000, 1000, 1000)anda =(100010002000) it is possible that the order (of 1000 units) forecasted for period 1 occursat period 2, and the order forecasted for period 2 occurs at period 2. However, it is aswell possible the order forecasted to period 1 has occured at period 3, which should becounted as an forecast error. I general, it is not possible to know how the forecast iscomposed; therefore these patterns are not smoothed. Moreover, the pattern may also beinterpreted as systematic timing error, which should counted as error.

15

6 Results

The proposed smoothing algorithms were tested using simulated and real data. Theobjective is to illustrate the effect of smoothing, and to ensure that the approaches satisfythe properties presented in Section 2.1. First, smoothed forecast is calculated for simulateddata using the same distributions of daily demand as in Section 3. In addition, forecasterrors for smoothed and non-smoothed forecasts calculated using MSE, MAE, MAPE andMASE are compared to the total forecast error (∑Tt Ft − Yt). Second, the same processis conducted with real data.

6.1 Simulated sales and forecasts

To ensure that the proposed smoothing algorithms behave as expected in various settings,they were verified using simulation. The sales time series are generated as in Section 3.The forecasts were formed from the sales {D̃} using the assumption that the relativeforecast error for sales volume is et ∼ N(0, σ). After the calculation of forecasts, thetiming of sales was randomised by s ∼ N(0, σt) days, resulting in time series {D̃s}. Anexample of the data and corresponding smoothed forecasts are presented in Table 3. Thedata in Table 3 was generated using value of σt = 4 and aggregation of 30 days; thus,the timing of sales and forecast differs at most by one aggregation period. In such asetting, the LP approach smoothes the forecast to correspond the actual sales excludingthe deviations caused by forecasts errors in sales volume.

The error measures for the forecasts are presented in Table 4. As expected, the errorsare smaller for smoothed forecasts. The results also illustrate the drawbacks of measuresdiscussed in Section 4.1. The interpretation of MSE is difficult because the values andtheir variation are of a different magnitude than the time series. Conversely, MAPE isdefined only for a small subset of measures, which makes the comparison of forecastsdifficult.

The smoothing should only apply to errors caused by the differences in the timingof sales. This is tested by generating the forecasts from sales by adding a term et ∼N(0, σv)×Dt representing the error in forecasted sales volume and timing error. A scatterplot of total error of sales and forecasts and MAE corresponding to the unsmoothedsmoothed time series generated with σv = 0.5 is presented in Figure 9. As shown infigures (b) and (c), both of the smoothing algorithms work as desired; that is, they donot smooth any significant amount of the volume error, and a high value of total forecasterror corresponds to a high value of smoothed MAE. However, the smoothing methodby Kerkkänen et al. (2006) shows low values even if the total forecast error is large (thepoints in the lower corners of figure).

16

Table 3. Example of simulated sales and (smoothed) forecasts for daily demand ofPoisson(0.05)×1000,aggregation of 30 period and forecast errors corresponding to σv =0.25 and σs = 4.

Sales Forecast DTW LP

0 0 0 01000 1126.7 1126.7 1126.70 0 0 00 0 0 00 0 0 00 0 0 01000 1123.3 1123.3 1123.30 0 0 01000 0 0 999.90 1921.2 0 01000 0 1921.2 921.330 0 0 00 0 0 00 0 0 01000 942.24 942.24 942.240 0 0 00 0 0 00 0 0 0

To illustrate the smoothing of timing errors, in Figure is presented the same plot asin Figure 9 for forecasts generated with relative forecast error of σv = 0.25 and timingerror of σt = 4. By comparing Figure 9 (a) to figures (b) and (c), it can be seen thatthe smoothing removes the largest errors (corresponding to values > 350) deriving fromthe timing differences, but not the smaller errors caused by differences in volume. Asexpected, the pair-wise matching does not smooth all the timing errors when the salesconsists of several orders.

6.2 Real data

To verify the results obtained with simulated data, both the approaches were tested usingreal data and compared to current practises for accuracy evaluation. The real data is usedto verify that the measure works as expected in situations where types and magnitudes offorecast errors are not as clear as with the simulated data. That is, forecast errors causedby timing differences of one period should be smoothed but random errors of sales volumeestimates are not.

First, we identify the observations for which the smoothed forecast differs most fromthe non-smoothed. The difference can be measured by comparing the MAE of unsmoothedand smoothed forecasts (Fig. 11). In Figure 11, these observations correspond to pointsthat have the largest distance from diagonal; that is, the smoothing has reduced the MAE

17

Table 4. Forecast accuracy measures for non-smoothed (Std), pair-wise smoothed (PW)and LP smoothed forecasts (LP).

MSE MAE MAPE MASEStd PW LP Std PW LP Std PW LP Std PW LP

0.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.0016043 16042 16042 126 126 126 11.24 11.24 11.24 0.22 0.22 0.220.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.0015201 15202 15202 123 123 123 10.98 10.98 10.98 0.21 0.21 0.210.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.001000000 1000000 0.01 1000 1000 0.10 Inf Inf 0.01 1.70 0.00 1.703691113 0.00 0.00 1921 0.00 0.00 100.00 NaN NaN 3.27 0.00 0.001000000 848664 6189 1000 921 78 Inf 47.95 8.54 1.70 0.13 1.570.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.003336 3336 3336 57 57 57 6.13 6.13 6.13 0.10 0.10 0.100.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.000.00 0.00 0.00 0.00 0.00 0.00 NaN NaN NaN 0.00 0.00 0.00

318094.13 104624.80 2265.07 234.94 123.83 21.47 NaN NaN NaN 0.40 0.04 0.21

most. To verify that the errors in these forecasts are caused by mistiming of sales, sixforecasts with the largest difference in standard and smoothed MAE were selected forcloser inspection in Figure 12. In all these forecasts, most of the forecast errors derivefrom the timing of sales, which supports the results obtained from simulation.

18

-3000 -2000 -10000 0 1000 2000 3000

50

100

150

200

250

300

350

(a) MAE-3000 -2000 -10000 0 1000 2000 3000

50

100

150

200

250

300

350

(b) MAE (PW)

-3000 -2000 -10000 0 1000 2000 3000

50

100

150

200

250

300

350

(c) MAE (LP)-3000 -2000 -10000 0 1000 2000 3000

50

100

150

200

250

300

(d) MAE (SMA)

Figure 9. Smoothing of random errors in sales volume.

19

-2000 -10000 0 1000 2000

50

100

150

200

250

300

350

400

450

(a) MAE-2000 -10000 0 1000 2000

50

100

150

200

250

300

350

400

450

(b) MAE (PW)

-2000 -10000 0 1000 2000

50

100

150

200

250

300

350

400

450

(c) MAE (LP)-2000 -10000 0 1000 2000

50

100

150

200

250

300

350

400

450

(d) MAE (SMA)

Figure 10. Smoothing of random timing differences from sales. On x-axis is the totaldifference of forecast and sales, and on the y-axis the MAE for the forecasts.

00

500

1000

1000

1500

2000

2000

2500

3000

3000

3500

4000

4000

4500

5000

5000 6000

Figure 11. Smoothing of random timing differences from sales. On x-axis is the MAEfor non-smoothed forecast, and on y-axis is the MAE for smoothed forecasts.

20

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

0

2000

4000

6000

8000

10000

12000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

0

2000

4000

6000

8000

10000

12000

14000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

22000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

0

2000

4000

6000

8000

10000

12000

14000

16000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0

1000

2000

3000

4000

5000

6000

7000

8000

9000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Figure 12. Forecasts (blue line) for which the smoothing reduces the MAE mosts (thesmoothed forecast is presented by the dashed bars).

21

7 Discussion

The study showed that in a setting of highly intermittent demand, temporal aggregationmay not be sufficient for smoothing random fluctuations caused by the timing of sales. Toovercome this problem, two approaches to smooth the forecasts were presented. Assumingthat the deviations in forecasted and actual sales are caused by differences in the timingof sales, the forecasts are smoothed by comparing the values of sales and forecasts forconsecutive periods.

The presented approaches make it possible to set an allowed timing difference betweenforecasted and actual sales more flexibly than using standard temporal aggregation. If thesales are measured daily, a standard aggregation of T days allows a timing difference ofT −1 days for sales but at the aggregation boundary a timing error of 1 day can cause un-expected deviation to sales and forecast time series. This deviation can be significant andhide the relevant patterns of time series if the daily demand is intermittent, as illustratedin Section 6.1 using simulation.

By smoothing the forecast using a modified DTW algorithm or a linear program-ming approach, the timing differences at aggregation boundaries can be eliminated. Thesmoothed forecast ignores timing difference of T to 2T −1 periods, which eliminates errorrandom deviations caused by timing of sales from the forecast time series. Generally,the smoothing ignores timing difference of less than wT days and counts as errors over(w + 1)T days differences for an aggregation period of T and a smoothing window ofw. However, the presented formulation is not flexible to the selection of the smoothingwindow, and changing the window requires modification of the constraints in the LPproblem.

The presented approach is, however, flexible for the selection of forecast accuracy mea-sures. Because method produces a smoothed time series of forecasts (instead of smoothedforecast error) all the standard measures of forecast accuracy can be used with it. Thiscan be seen as a favourable feature, because the appropriateness of the measure dependson the purpose of use.

The downside of the smoothing is that allowed timing error is double compared tostandard aggregation. Thus, in most cases, the aggregation period should to be reduced.Moreover, the presented approach is feasible only if the forecast horizon is longer thanone period. Otherwise, the forecast could be adjusted to match the actual sales of theprevious period; that is, a naïve forecast Ft+1 = Yt would give zero forecast error.

Both the approaches are computationally feasible. Solving the smoothing problemfor 1000 time series of 18 observation takes with the pair-wise smoothing less than 10seconds and with the other approach less than 5 seconds on a standard desktop computer.Moreover, by optimising the model, the running time could likely be reduced.

22

Similarly as aggregation, the smoothing discards information from the time series. Inthis case, the smoothing hides the fluctuations in forecasts and actual sales for consecutiveperiods. Particularly, the minimisation of forecast error can hide errors that are not causedby mistiming of sales. Therefore, it is important to ensure that the fluctuations arecaused by the timing, not by random errors in the volume of forecasts. If this condition issatisfied, the proposed approach can help in identifying the paramount sources of accuracyimprovement for judgemental forecasts. Nevertheless, the smoothed forecast error is bestused in conjunction with standard error measures.

23

ReferencesCaniato, F., Kalchschmidt, M., Ronchi, S., Verganti, R., Zotteri, G. (2005). Clustering

customers to forecast demand. Production Planning & Control, 16(1): 32–43.

Hyndman, R.J., Koehler, A.B. (2006). Another look at measures of forecast accuracy.International Journal of Forecasting, 22: 679–688.

Kerkkänen, A., Korpela, J., Huiskonen, J. (2006). Impacts of different types of salesforecast errors on production planning and inventory management: A case study. In14th International Symposium on Inventories.

Lawrence, M., O’Connor, M., Edmundson, B. (2000). A field study of sales forecastingaccuracy and processes. European Journal of Operational Research, 122: 151–160.

Marcellino, M. (1999). Some consequences of temporal aggregation in empirical analysis.Journal of Business & Economic Statistics, 17(1): 129–136.

Sakoe, H., Chiba, S. (1978). Dynamic programming algorithm optimization for spokenword recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing,26(1): 43–49.

Zotteri, G., Kalchschmidt, M., Caniato, F. (2004). The impact of aggregation level onforecasting performance. International Journal of Production Economics, 93-94: 479–491.

24

A Implementation of the pair-wise smoothing usingXPress-MP

model ferror

uses "mmxprs"

parametersperiods = 18observations = 1000variables = 3*periods - 2

end-parameters

declarationsforecasts: array(1..observations,1..periods) of realactuals: array(1..observations,1..periods) of realdistance: array(1..observations,1..variables) of realx: array(1..(3*periods)) of mpvarFC_const: array(1..observations,1..(periods+1)) of linctrA_const: array(1..observations,1..periods) of linctrdef_out: integerfid_forecasts: integerfid_actuals: integer

end-declarations

initializations from "datafile_real.dat"forecastsactuals

end-initializations

def_out := getfid(F_OUTPUT)fopen("forecasts-pw.dat", F_OUTPUT)fid_forecasts := getfid(F_OUTPUT)fopen("actuals-pw.dat", F_OUTPUT)fid_actuals := getfid(F_OUTPUT)fselect(def_out)

forall (s in 1..observations) do

distance(s,1) := abs(forecasts(s,1) - actuals(s,1))distance(s,2) := abs(forecasts(s,1) - actuals(s,2))+0.1

forall (t in 1..(periods-1)) dodistance(s,3*t) := abs(forecasts(s,t+1) - actuals(s,t))+0.1distance(s,3*t+1) := abs(forecasts(s,t+1) - actuals(s,t+1))if t < (periods-1) then

distance(s,3*t+2) := abs(forecasts(s,t+1) - actuals(s,t+2))+0.1end-if

end-do

! Objective function:TotalCost := sum(i in 1..variables) x(i)*distance(s,i)

! Constraints for matching the forecasts:FC_const(1,1) := x(1) + x(2) = 1forall(n in 1..(periods-2)) FC_const(1,n+1) := x(3*n) + x(3*n+1) + x(3*n+2) = 1FC_const(1,(periods+1)) := x(variables-1) + x(variables) = 1

! Constraints for matching the actual sales:A_const(1,1) := x(1) + x(3) = 1forall(n in 0..(periods-3)) A_const(1,n+2) := x(3*n+2) + x(3*n+4) + x(3*n+6) = 1

25

A_const(1,periods) := x(variables-2) + x(variables) = 1

! x(i) = 1 repsent mathching and x(i) = 0 no matchingforall(i in 1..variables) x(i) is_binary

! Minimise the total distanceminimize(TotalCost)

writeln("Adjusted forecast error ", s, ": ", getsol(TotalCost))

fselect(fid_forecasts)forall(i in 1..variables) do

if getsol(x(i)) = 1 thenwrite(forecasts(s,floor(i/3)+(i-3*floor(i/3))), " ")

end-ifend-dowriteln("")

fselect(fid_actuals)forall(i in 1..variables) do

if getsol(x(i)) = 1 thenwrite(actuals(s,floor(i/3)+1), " ")

end-ifend-dowriteln("")fselect(def_out)

fopen("forecasts"+s+".dat", F_OUTPUT)forall (t in 1..periods) do

writeln(forecasts(s,t))end-dofclose(F_OUTPUT)

fopen("actuals"+s+".dat", F_OUTPUT)forall (t in 1..periods) do

writeln(actuals(s,t))end-dofclose(F_OUTPUT)

! Generating the plotting commands for gnuplotfopen("plot"+s+".gp", F_OUTPUT)writeln("set terminal postscript")writeln("set output ’fc_error", s, ".ps’")writeln("set nokey")writeln("set arrow 1 from graph 0, graph 0 to graph 1, graph 0")writeln("set arrow 2 from graph 0, graph 0 to graph 0, graph 1")writeln("set border 3")writeln("set boxwidth 0.2")writeln("set xtics nomirror out")writeln("set ytics nomirror out")writeln("set function style lines")writeln("set size ratio 0.5")writeln("set xrange [0:19]")writeln("set xtics 1,1,18")writeln("")forall(i in 1..variables) do

if getsol(x(i)) = 1 thenwriteln("set arrow from ", floor(i/3)+1, ",", forecasts(s,floor(i/3)+1),

" to ", floor(i/3)+(i-3*floor(i/3)), ",",actuals(s,floor(i/3)+(i-3*floor(i/3))), " nohead lt 2 lc 1")

end-ifend-do

26

writeln("plot ’forecasts", s, ".dat’ using ($0+1):($1) with lines lw 2 lt 1, \\")writeln("’actuals", s, ".dat’ using ($0+1):($1) with lines lw 3 lt 1")writeln("")fclose(F_OUTPUT)

end-do

fselect(fid_forecasts)fclose(F_OUTPUT)

end-model

B Implementation of the LP smoothing using XPress-MP

model ferror

uses "mmxprs"

parametersperiods = 18observations = 1000

end-parameters

declarationsforecasts: array(1..observations,1..periods) of realactuals: array(1..observations,1..periods) of realx: array(1..periods,1..3) of mpvarz: array(1..periods) of mpvarABS_const: array(1..2*periods) of linctrFC_const: array(1..periods+2) of linctrT_const: array(1..periods) of linctrdef_out: integerfid_forecasts: integerfid_actuals: integer

end-declarations

initializations from "datafile.dat"forecastsactuals

end-initializations

def_out := getfid(F_OUTPUT)fopen("forecasts-lp.dat", F_OUTPUT)fid_forecasts := getfid(F_OUTPUT)fopen("actuals-lp.dat", F_OUTPUT)fid_actuals := getfid(F_OUTPUT)fopen("forecasts.dat", F_OUTPUT)fcoid_actuals := getfid(F_OUTPUT)fselect(def_out)

! The main simulation loopforall (s in 1..observations) do

initglobal

TotalCost := sum(i in 1..periods) z(i)

! Handling of the absolute value objective functionABS_const(1) := x(1,2) + 1.0001*x(1,3) - actuals(s,1) <= z(1)

27

ABS_const(2) := -(x(1,2) + 1.0001*x(1,3) - actuals(s,1)) <= z(1)forall(i in 2..(periods-1)) ABS_const(i+1) := 1.0001*x(i,1) + x(i,2) +

1.0001*x(i,3) - actuals(s,i) <= z(i)forall(i in 2..(periods-1)) ABS_const(i+17) := -(1.0001*x(i,1) + x(i,2) +

1.0001*x(i,3) - actuals(s,i)) <= z(i)ABS_const(2*periods-1) := 1.0001*x(periods,1) + x(periods,2) -

actuals(s,periods) <= z(periods)ABS_const(2*periods) := -(1.0001*x(periods,1) + x(periods,2) -

actuals(s,periods)) <= z(periods)

! Constraints to ensure the distributions equal the forecasts:FC_const(1) := x(1,1) = 0FC_const(2) := x(1,2) + x(2,1) = forecasts(s,1)forall(i in 2..(periods-1)) FC_const(i+1) := x(i-1,3) + x(i,2) + x(i+1,1) = forecasts(s,i)FC_const(periods+1) := x(periods,3) = 0FC_const(periods+2) := x((periods-1),3) + x(periods,2) = forecasts(s,periods)

! Constrains to allow only distribution of forecasts for one period onlyT_const(1) := x(2,1) <= maxlist(forecasts(s,1) - actuals(s,1), 0)forall(i in 2..(periods-1)) T_const(i) := x(i-1,3) +

x(i+1,1) <= maxlist(forecasts(s,i) - actuals(s,i), 0)T_const(periods) := x((periods-1),3) <= maxlist(forecasts(s,periods) -

actuals(s,periods), 0)

! Minimoidaan kustannuksiaminimize(TotalCost)

writeln("Adjusted forecast error ", s, ": ", getsol(TotalCost))

fselect(fid_forecasts)forall(i in 1..periods) do

write(getsol(x(i,1)) + getsol(x(i,2)) + getsol(x(i,3)), " ")end-dowriteln("")

fselect(fid_actuals)forall(i in 1..periods) do

write(actuals(s,i), " ")end-dowriteln("")

fselect(fcoid_actuals)forall(i in 1..periods) do

write(forecasts(s,i), " ")end-dowriteln("")

fselect(def_out)

fopen("smooth_error"+s+".dat", F_OUTPUT)writeln("")forall(i in 1..periods) do

writeln(getsol(x(i,1)) + getsol(x(i,2)) + getsol(x(i,3)))end-dofclose(F_OUTPUT)

fopen("forecasts"+s+".dat", F_OUTPUT)forall (t in 1..periods) do

writeln(forecasts(s,t))end-dofclose(F_OUTPUT)

28

fopen("actuals"+s+".dat", F_OUTPUT)forall (t in 1..periods) do

writeln(actuals(s,t))end-dofclose(F_OUTPUT)

fopen("fcerror_"+s+".gp", F_OUTPUT)writeln("set terminal postscript")writeln("set output ’fc_error_", s, ".ps’")writeln("set nokey")writeln("set arrow 1 from graph 0, graph 0 to graph 1, graph 0")writeln("set arrow 2 from graph 0, graph 0 to graph 0, graph 1")writeln("set border 3")writeln("set boxwidth 0.2")writeln("set xtics nomirror out")writeln("set ytics nomirror out")writeln("set function style lines")writeln("set size ratio 0.5")writeln("set xrange [0:19]")writeln("set xtics 1,1,18")writeln("")

writeln("plot ’forecasts", s, ".dat","’ using ($0+1):($1) with lines lw 1 lt 1 lc 3, \\")

writeln("’actuals", s, ".dat", "’ using ($0+1):($1) with lines lw 3 lt 1, \\")writeln("’smooth_error", s, ".dat’ using ($0+1):($1) with impulses lc 1 lt 2")fclose(F_OUTPUT)

end-do

end-model

29