Chapter 9 Maintenance and Replacement

The problem of determining the lifetime of an asset or

an activity simultaneously with its management during

that lifetime is an important problem in practice. The

most typical example is the problem of optimal

maintenance and replacement of a machine; see

Rapp(1974) and Pierskalla and Voelker(1976). Other

examples occur in forest management such as in

Naslund(1969), Clark(1976), and Heaps(1984), and in

advertising copy management as in Pekelman and

Sethi(1978).

9.1 A Simple Maintenance and Replacement Model

9.1 A Simple Maintenance and Replacement ModelT = the sale date of the machine to be determined, = the constant discount rate,x(t) = the resale value of machine in dollars at time t ;

let x(0) = x0 ,u(t) = the preventive maintenance rate at time t (maintenance here means money spend over and above the minimum required for necessary repairs),g(t) = the maintenance effectiveness function at time t (measured in dollars added to the resale value per dollar spent on preventive maintenance) ,

d(t) = the obsolescence function at time t (measured

in terms of dollars subtracted from x at time t),

= the constant production rate in dollars per unit

time per unit resale value; assume > or else

it does not pay to produce.

It is assumed that g(t) is a nonincreasing function of

time and d(t) is a nondecreasing function of time, and

that for all t

where U is a positive constant.

The state variable x is affected by the obsolescence

factor, the amount of preventive maintenance, and the

maintenance effectiveness function. Thus,

In the interests of realism we assume that

This implies that the resale value of machine can’t

increase.

9.1.1 Solution by the Maximum Principle

The standard Hamiltonian as formulated in Section 2.2

is

where the adjoint variable satisfies

Since T is unspecified, the additional terminal

condition of (3.14) becomes

which must hold on the optimal path at time T .

The adjoint variable can be easily obtained by

integrating (9.6), i.e.,

The interpretation of (t) is as follows. It gives in

present value terms, the marginal profit per dollar of

gain in resale value at time t. The first term represents

the value of one dollar of additional salvage value at

T brought about by one dollar of additional resale

value at the current time t. The second term represent

the represents the present value of incremental

production from t to T bought about by the extra

productivity of the machine due to the additional one

dollar of resale value at time t .

To find how the optimal control switches, we need toexamine the switching function in (9.9). Rewriting it as

It is clear that the optimal control in (9.9) can now berewritten as

Note that all the above calculations were made on theassumption that T was fixed, i.e., without imposingcondition (9.7). On an optimal path, this condition,which uses (9.5), (9.7), and (9.8), can be restated as

This means that when , we have

and when , we have

Since d(t) is nondecreasing, g(t) is nonincreasing, and

x(t) is nonincreasing, equation (9.13) or equation

(9.14), whichever the case may be, has a solution for

T*.

9.1.2 A Numerical ExampleSuppose U=1, x(0)=100, d(t)=2, =0.1, =0.05, and g(t)= 2/(1+t)1/2. Let the unit of time be one month. First,we write the condition on ts by equating (9.10) to 0,which gives

In doing so, we have assumed that the solution of(9.15) lies in the open interval (0,T). As we shallindicate later, special care needs to be exercised if thisis not the case. Substituting the data in (9.15) we have

which simplifies to

Then, integrating (9.3), we find

and hence

Since we have assumed 0< ts <T, we substitute x(T)into (9.13), and obtain

which simplifies to

We must solve (9.16) and (9.17) simultaneously.Substituting (9.17) into (9.16), we find that ts must bea zero of the function

A simple binary search program was written to solve

this equation, which obtained the value ts =10.6.

Substitution of this into (9.17) yields T=34.8. Since this

satisfies our supposition that 0< ts <T , we can

conclude our computations. Thus, the optimal solution

is to preventive maintenance at the maximum rate

during the first 10.6 months, and thereafter not at all.

The sale date is at 34.8 months after purchase. Figure

9.1 gives the functions x(t) and u(t) for this optimal

maintenance and sale date policy.

Figure 9.1: Optimal Maintenance and Machine Resale Value

If, on the other hand, the solution of (9.16) and (9.17)

did not satisfy our supposition, we would need to

follow the procedure outlined earlier in the section.

This would result ts= 0 or ts= T. If ts= 0, we would

obtain T from (9.17), and conclude u*(t)=0, 0 t T.

Alternatively, if ts= T, we would need to substitute x(T)

into (9.14) to obtain T. In this case the optimal control

would be u*(t)=U, 0 t T.

9.1.3 An Extension

The pure bang-bang result in the model developed

above is a result of the linearity in the problem. The

result can be enriched as in Sethi(1973b) by

generalizing the resale value equation (9.3) as

follows:

where g is nondecreasing and concave in u. For this

section, we will assume the sale date t to be fixed for

simplicity and g to be strictly concave in u,i.e., gu 0

and guu < 0 for all t . Also, gt 0 , gut < 0 ,and g(0,t)=0;

see Exercise 9.5 for an example of function g(u,t).

The standard Hamiltonian is

where is given in (9.8).To maximize the Hamiltonian,

we differentiate it with respect to u and equate the

result to zero. Thus,

If we let u0(t) denote the solution of (9.21), then u0(t)

maximizes the Hamiltonian (9.20) because of the

concavity of g in u. Thus, for a fixed T, the optimal

control is

To determine the direction of change in u*(t), we use

(9.21) and value (t) from (9.8) to obtain

Since > , the denominator on the right-hand side of

(9.23) is monotonically decreasing with time.

Therefore, the the right-hand side of (9.23) is

increasing with time. Taking the time derivative of

(9.23), we have

But and . it is therefore obvious that

.

To sketch the optimal control u*(t) specified in (9.22),

define , such that for t t1 and

for t t2 .Then, we can rewrite the sat function in (9.22) as

In (9.24), it is possible to have t1 =0 and/or t2 =T . In

Figure 9.2 we have sketched a case when t1 >0 and

t2 <T. Note that while u0(t) in Figure 9.2 is decreasing over time, the way it will decrease will depend on thenature of the function g . Indeed, the shape of u0(t) ,while always decreasing, can be quite general.

Figure 9.2: Sat Function Optimal Control

In particular, you will see in Exercise 9.5 that the

shape of u0(t) is concave and, furthermore u0(t) >0,

t 0, so that t2 = T in that case.

9.2 Maintenance and Replacement for a Machine Subject to Failure

T = the sale date of a machine to be determined,

u(t) = the preventive maintenance rate at time t;

0 u(t) 1

R = the constant positive rate of revenue produced by

a functioning machine independent of its age at

any time, net of all costs except preventive

maintenance,

= the constant discount rate,

L = the constant positive junk value of the machine

independent of its age at failure,

B(t) = the (exogenously specified) resale value of the machine at time t, if it is still functioning;h(t) = the natural failure rate (also termed the nature hazard rate in the reliability theory);F(t) = the cumulative probability that the machine has failed by time t ,C(u,h) = the cost function depending on the preventive maintenance u when the natural failure rate is h.To make economic sense, an operable machine mustbe worth at least as much as an inoperable machineand its resale value should not exceed the present value of the potential revenue generated by the

machine if it were to function forever.

Thus,

Also for all t > 0,

Finally, the cost of reducing the natural failure rate is

assumed to be proportional to the natural failure rate.

Specifically, we assume that C(u,h)=C(u)h denotes the

cost of preventive maintenance u when the natural

failure rate is h. In other words, when the natural

failure rate is h and a controlled failure rate of h(1-u)

is sought, the action of achieving this reduction will

cost C(u)h dollars.

It is assumed that

which gives the state equation

Using (9.29), we can rewrite J as follows:

The optimal control problem is to maximize J in (9.30)

subject to (9.29) and (9.26)

9.2.1 Optimal Policy

The problem is similar to Model Type (f) in Table 3.3

subject to the free-end-point condition as in Row 1 of

Table 3.1. Therefore, we follow the steps for solution

by the maximum principle stated in Chapter 3. The

standard Hamiltonian is

and the adjoint variable satisfies

Since T is unspecified, we apply the additional terminal

condition (3.14) to obtain (See Exercise 9.6)

Interpretation of (9.33),the first two terms in (9.33) give the net

cash flow, to which is added the junk value L multiplied by the

probability [1-u*(T)]h(T*) that the machine fails. From this, we

subtract the third term which is the sum of loss of the entire

resale value B(T*), and the loss of the entire resale value

when the machine fails, Thus, the left-hand side of (9.33)

represents the marginal benefit of keeping the machine.

Equation (9.33) determining the optimal sale date is the usual

economic condition equating marginal benefit to marginal cost.

In the trivial case in which the natural failure rate h(t) is zero or

when the machine fails with certainty by time t (i.e., F(t)=1), then

u*(t)=0, Assume therefore h>0 and F <1 . Under these

conditions, we can infer from (9.27) and (9.34) that

Using the terminal condition from

(9.32), we can derive u*(T) satisfying (9.35):

The next question is to determine how u*(t) changes

over time. Kamien and Schwartz(1998) have shown

that ; see Exercise 9.7. That means there

exists such that

Here u0(t) is the solution of (9.35) (iii), and it is easy to

show that . Of course, u*(T) is immediately

known from (9.36). If u*(T) (0,1) , it implies t2 = T ;

and if u*(T)=1, it implies t1= t2=T .

For this model, the sufficiency of the maximum

principle follows from Theorem 2.1; see Exercise 9.8.

9.3 Chain of Machines

We now extend the problem of maintenance and

replacement to a chain of machines. By this we mean

that given the time periods 0,1,2,…,T-1, we begin with

a machine purchase at the beginning of period zero.

Then, we find an optimal number of machines, say l,

and optimal times 0< t1 < t2 <… t l –1 < t l < T of their

replacements such that the existing machine will be

replaced by a new machine at time tj, j =1,2,… l. at the

end of the horizon defined by the beginning of period

T, the last machine purchased will be salvaged.

Moreover, the optimal maintenance policy for each of

the machines in the chain must be found.

Two approaches to this problem have been developed

in the literature. The first attempts to solve for an

infinite horizon (T= ) with a simplifying assumption of

identical machine lives,i.e.,

for all j 1.

Consider buying a machine at the beginning of period

s and salvaging it at the beginning of period t > s. Let

Jst denote the present value of all net earnings

associated with the machine. To calculate Jst we need

the following notation for s k t-1.

= the resale value of the machine at the beginning

of period k ,

= the production quantity (in dollar value) during

period k ,

= the necessary expense of the ordinary

maintenance (in dollars) during period k ,

= the rate of preventive maintenance (in dollars)

during period k ,

= the cost of purchasing machine at the beginning

of period s ,

= the periodic discount rate.

It is required that

We can calculate Jst in terms of the variables andfunctions defined above

We must also have functions that will provide us withthe ways in which states change due to the age of themachine and the amount of preventive maintenance.Also, assuming that at time s, the only machines available are those that are up-to-date with respect tothe technology prevailing at s, we can subscript thesefunctions by s to reflect the effect of the machine’s technology on its state at a later time k .

Let and be such concave functions

so that we can write the following state equations:

where is the fractional depreciation immediately after

purchase of the machine at time s .

To convert the problem into the Mayer form, define

Using equations (9.43) and (9.44), we can write the

optimal control problem as follows:

subject to

and the constraints (9.41), (9.42), and (9.39).

9.3.1 Solution by the Discrete Maximum PrincipleWe associate the adjoint variablesrespectively with the state equations (9.46), (9.47),(9.41), and (9.42). Therefore, the Hamiltonianbecomes

where the adjoint variables 1 , 2 , 3 , and 4 , satisfythe following difference equations and terminalboundary conditions:

The solutions of these equations are:

Note that are constraints for a fixed machine

salvage time t . To apply the maximum principle, we

substitute (9.53)-(9.56) into the Hamiltonian (9.48),

collect terms containing the control variable uk , and

rearrange and decompose H as

where H1 is that part of H which is independent of uk

and

Next we apply the maximum principle to obtain the

necessary condition for the optimal schedule of

preventive maintenance expenditures in dollars. The

condition of optimality is that H should be a maximum

along the optimal path. If uk were unconstrained, this

condition, given the concavity of and , would

be equivalent to setting the partial derivative of H with

respect to u equal to zero, i.e.,

Equation (9.59) is an equation in uk with the exception

of the particular case when and are linear in uk

(which will be treated later in this section). In general,

(9.59) may or may not have a unique solution. For our

case we will assume and to be of the form such

that they give a unique solution for uk . One such

case occurs when and are quadratic in uk . In

this case, (9.59) is linear in uk and can be solved

explicitly for a unique solution for uk . Whenever a

unique solution does exist, let this be

The optimal uk* is given as

9.3.2 Special Case of Bang-Bang ControlWe now start the special case in which the problem,and therefore H, is linear in the control variable uk. Inthis case, H can be maximized simply by having thecontrol at its maximum when the coefficient of uk in His positive, and minimum when it is negative, i.e., theoptimal control is of bang-bang type.In our problem, we obtain the special case if andassume the form

and

respectively, where and are given constants.

Then, the coefficient of uk in H, denoted by Ws(k,t ), is

and the optimal control uk* is given by

9.3.3 Incorporation into the Wagner-Whitin Framework for a Complete Solution

Once uk* has been obtained as in (9.61) or (9.65), we

can substitute it into (9.41) and (9.42) to obtain

and , which in turn can be used in (9.40) to obtain

the optimal value of the objective function denoted by

.This can be done for each pair of machine

purchased time s and sale time t > s .

Let gs denote the present value of the profit

(discounted to period 0) of an optimal replacement and

preventive maintenance policy for periods s, s+1, …,

T -1. Then,

With the boundary condition

The value of g0 will give the required maximum.

9.3.4 A Numerical Example

Machines may be bought at times 0,1, and 2. The cost

of a machine bought at time s is assumed to be

The discount rate, the fractional instantaneous

depreciation at purchase, and the maximum preventive

maintenance per period are assumed to be

respectively.

Let be the net return (net of necessary maintenance)

of a machine purchased in period s and operated in

period s . we assume

In a period k subsequent to the period s of machine

purchase, the returns , k > s, depends on the

preventive maintenance performed on the machine in

periods prior to period k. The incremental return

function is given by , which we assume to be

linear. Specially,

where

This means that the return in period k on a machine

purchased in period s goes down by an amount ds

every period between s and k, including s, in which

there is no preventive maintenance. This decrease can

be offset by an amount proportional to the amount of

preventive maintenance.

Note that the function is assumed to be stationary

over time in order to simplify the example.

Let be the salvage value at time k of a machine

purchased at s. We assume

The incremental salvage value function is given by

where

and

That is, the decrease in salvage value is a constant

percentage of the purchase price if there is no

preventive maintenance. With preventive maintenance,

The salvage value can be enhanced by a proportional

amount.

Let be the optimal value of the objective function

associated with a machine purchased at s and sold at

t s+1. We will now solve for , s=0,1,2, and s<t 3,

where t is an integer.

Before we proceed, we will as in (9.64) denote by

Ws(k,t), the coefficient of uk in the Hamiltonian H, i.e.,

The optimal control is given by (9.65).

It is noted in passing that

so that

This implies that

In this example , which means that if there is

a switching in the preventive maintenance trajectory of

a machine, the switch must be from $100 to $0.

Solution of Subproblems. We now solve the

subproblems for various values of s and t (s < t) by

using the discrete maximum principle.

From (9.65) we have

Now,

Similar calculations can be carried out for other

subproblems. We will list these results.

Wagner-Whitin Solution of the Entire Problem.With reference to the dynamic programming equation in (9.66) and (9.67), we have

Now we can summarize the optimal solution. The optimal number of machines is 2, and their optimalpolicies are:

First Machine Optimal Policy:

Purchase at s =0; sell at t =1; optimal preventive

Maintenance policy u0*= 0.

Second Machine Optimal policy:

Purchase at s =1; sell at t =3; optimal preventive

maintenance policy u1*=100, u2*=0. The value of the

objective function is J*= $1237.4.