Multi-objective Optimization of Water-using

Systems

Carlos E. Mariano-Romero and Vıctor H. Alcocer-YamanakaHydraulics Department,

Mexican Institute of Water Technology,Paseo Cuahunahuac 8532, Progreso, Jiutepec, Mexico

{cmariano,valcocer}@tlaloc.imta.mx,Eduardo F. Morales

INAOE,Luis Enrique Erro No. 1, Santa Marıa Tonantzintla,

Puebla, 72840, Mexico

emorales@inaoep.mx

Abstract

Industrial water systems often allow efficient water uses via waterreuse and/or recirculation. The design of the network layout connect-ing water-using processes is a complex problem which involves severalcriteria to optimize. Most of the time, this design is achieved usingWater Pinch technology, optimizing the freshwater flow rate enter-ing the system. This paper describes an approach that considers twocriteria: (i) the minimization of freshwater consumption and (ii) theminimization of the infrastructure cost required to build the network.The optimization model considers water reuse between operations andwastewater treatment as the main mechanisms to reduce freshwaterconsumption. The model is solved using MDQL (Multi-objective Dis-

tributed Q-Learning), a heuristic approach based on the exploitationof knowledge acquired during the search process. MDQL has beenpreviously tested on several multi-objective optimization benchmarkproblems with promising results [16]. In order to compare the qualityof the results obtained with MDQL, the reduced gradient method was

1

applied to solve a weighted combination of the two objective functionsused in the model. The proposed approach was tested on three cases:(i) a single contaminant four unitary operations problem where fresh-water consumption is reduced via water reuse, (ii) a four contaminantsreal-world case with ten unitary operations, also with water reuse, and(iii) the water distribution network operation of Cuernavaca, Mexico,considering reduction of water leaks, operation of existing treatmentplants at their design capacity, and design and construction of newtreatment infrastructure to treat 100% of the wastewater produced.It is shown that the proposed approach can solved highly constrainedreal-world multi-objective optimization problems.

1 Introduction

Water pinch technology (WPT) evolved out of the broader concept of processintegration of materials and energy and the minimization of emissions andwastes in chemical processes. WPT can be seen as a type of mass-exchangeintegration involving water-using operations, that enables practicing engi-neers to answer important questions when retrofitting existing facilities anddesigning new water-using networks. There are three basic tasks in WPT: a)identification of the minimum freshwater consumption and wastewater gen-eration in water-using operations (analysis), b) water-using network designto comply with the flow rate targets for freshwater and wastewater throughwater reuse, regeneration, and recycle (synthesis), and c) modification of anexisting water-using network to maximize water reuse and minimize wastew-ater generation through effective process changes (retrofit).

Nowadays most WPT problems are formulated as non linear highly re-stricted programming problems [1, 13, 14]. Important efforts have aimedto make the mathematical models more robust and applicable to real worldproblems [2, 7, 10]. Other efforts have aimed to apply WPT technology toother fields such as design and retrofit of urban distribution systems [3].

In general, WPT traditionally minimizes freshwater flow rate entering asystem, using mass balance and the concentrations of contaminants at theinlet and outlet in all water-using operations as restrictions. Because of thediverse types of water-using operations, treatment effectiveness and cost, andtypes of contaminants, the criteria for efficient use of water is inherently nonlinear, multiple and conflicting [2, 10, 13]. Some of the criteria that can easilybe identified are: equipment cost minimization, maximization of reliability

2

(amount of contaminant captured at treatment plants) and minimization ofwastewater production.

This paper describes a mathematical formulation that extends WPT anal-ysis with elements of capital cost of the required pipe work. Consequently,the optimization is based on cost efficient networks and networks featuringfreshwater consumption. The model involves two criteria: (i) the minimiza-tion of freshwater consumption and (ii) the minimization of infrastructurecosts. Two techniques are used to solve this problem: 1) weighted aggrega-tion considering variation in the weight coefficients in order to construct thePareto set and using a reduced gradient method, and 2) MDQL, a heuristicapproach based on the exploitation of the knowledge generated during thesearch process. Results obtained with both approaches are compared withsolutions reported in the literature for the solution of the single-objectiveproblem that minimizes the freshwater flow rate entering the system.

The proposed multi-objective optimization model was applied to threetest cases: a) Four water-using operations and single contaminant, b) tenwater-using operations and four contaminants and c) Cuernavaca’s waterdistribution network operation considering two different strategies: c.1) re-duction of leaks in the network and operation of wastewater treatment plantsat their design capacity, and c.2) reduction of leaks in the network, opera-tion of wastewater treatment plants at their design capacity, and constructionof new treatment infrastructure to reach 100% wastewater treatment. It isshown that the highly constrained subjective optimization real-world prob-lems can be solved with MDQL.

Section 2 presents the mathematical formulation for the bi-objective op-timization problem. In Section 3 the weighted aggregation method and theMDQL heuristic approach are described. Section 4 describes the three casesunder study and discusses the main results. Section 5 gives conclusions andfuture research directions.

2 Mathematical formulation

The mathematical model describing an industrial water demanding processconsiders two main components: a) the available freshwater sources to sat-isfy demands, and b) the water-using operations described by loads of con-taminants and concentration levels. An example of two sources and twooperations is sketched in Figure 1. This figure represents with rectangles

3

i, 1

2,1

X1,2

X2,1

X1,2

O1

O2

W 1

W 2

f1

f2

f1,loss

f2,loss

XR,1

XR,2X2,R

X1,R

Reusable water from treatment plants (X )

Fresh water, f

Wastewater, W i

Water losses, f

Reusable water from O 1

Reusable water from O 2 (X

i, loss

i, R

i, 2 )

)(X

X

Figure 1: Block diagram of a water-using system with two sources and twooperations.

the two unitary operations (Oi), and with solid lines on the left side of theoperations their corresponding freshwater demands (fi). Wastewater fromoperations are represented with dashed lines on the right side of operations.The rest of the connections represent all the potential links between unitaryoperations (water reuse), leaks, and treatment plants. The direction arrowheads at the end of lines indicate the direction of flux.

The design task is to find the network configuration that minimizes theoverall demand for freshwater,

∑

fi, (and consequently reduce the wastew-ater volume

∑

Wi) compatible with minimum investment cost. In order tocomplete the design task, the optimization problem is stated in terms of lowfreshwater consumption, a suitable network topology for water reuse, Xi,j,and a low investment cost.

Unitary operations of demanded water are defined through their con-taminant loads, required flow rates, and allowable minimal and maximalcontaminant concentrations at influxes and discharges.

The objective functions for freshwater consumption minimization and forinfrastructure minimization are represented by Equations 1 and 2.

4

MinZ1 = F1 =∑

j

cstj + TPC, (1)

MinZ2 = F2 =∑

i

fi (2)

Where: F1 is the total cost of the distribution network considering theconnection of freshwater sources to unitary operations receiving water di-rectly, and the connection for reusing water between unitary operations. Thetotal distribution network cost is composed by the sum of the partial costs,cstj, of the pipe segments used for connecting freshwater sources to uni-tary operations and unitary operations to unitary operations, and TPC, thetreatment plant construction cost that applies only for new treatment in-frastructure. In F1 we are not considering maintenance and rehabilitationcosts.

F2, is the total freshwater demanded by the system, obtained by thepartial demands of freshwater from each of the unitary operations in thesystem. Partial demands from unitary operations, say operation Oi, arerepresented as fi. That is fi is the partial freshwater demand of operationOi.

2.1 Infrastructure cost

Evaluation of the first objective function, F1, depends only on the pipe seg-ment costs in the network. These costs are represented as cstj, and dependon three variables (see Equation 3): a) pipe length, Lj; b) cost per unitlength, PCj; which depends on the pipe diameter required to transport thedemanded flow of water, Dj; and c) a cost factor, CFj, related to pipe ma-terials required to resist corrosive effects of contaminants. It is important tonote that the main objective of this work is to demonstrate the benefits ob-tained by the solution of the multi-objective approach, compared with thoseobtained with a single objective approach. For this reason, some considera-tions regarding the hydraulic behavior of the network are not included.

cstj = Lj × PCj × CFj (3)

As previously mentioned, PCj depends on the minimum pipe diameter,Dj = f(Qj), required to transport the water flow through the pipe. Theminimum diameter, Dminj

, is obtained applying Equation 4; deduced from

5

Table 1: Cost per unit length for commercial diameter pipes.Diameter (mm) PC $/m

99 4.8150 5.0200 8.9250 12.9300 17.7350 23.6400 25.6450 34.1500 40.9610 42.6762 45.9838 54.6

1, 016 69.91, 118 831, 219 941, 372 110

the definition of flow (Q = velocity/area) considering maximum velocitiesof water in pipes of 2.5m/s. Dminj

is approximated to the closest uppercommercial diameter. Table 1 shows diameters and cost per unit lengthfor commercial pipes considered in this work. The data in Table 1 is onlydemonstrative and can be substituted with real data from local markets.

Dmin = 0.714√

Q (4)

where: Dmin is the minimal pipe diameter in mm required to transport flowrate Q ; Q ∈ {fi, Xi,j, Wi}∀i, j and is given in m3/s.

In a similar manner, the factor CFj is related to the capacity of the pipesegments to resist corrosive effects due to the presence of contaminants inwater flows. Values for the CFj factor are included in Table 2, calculatedconsidering local prices in Mexico for non corrosive pipes.

Finally the treatment plant construction cost considered in this work is10$/l, that is the construction cost in monetary units per liter of treatmentcapacity for the plant or plants.

6

Table 2: Cost factors for pipes resistant to abrasive effects of contaminants.Contaminant concentration (mg/l) CF

0 ≤ c ≤ 50 1.2550 < c ≤ 100 1.5100 < c ≤ 150 2.0150 < c ≤ 200 3.0200 < c ≤ 500 5.0

500 < c 10.0

i,k,out

C0

Xi,R

Ci,k,outXR,i

TP

floss

O i

Xj,iCi,k,out

W iCi,k,out

XR,iC

Xi,jCj,k,out

fi

Xi,RC0

fi,lossCi,k,out

Figure 2: General structure for mass balance.

2.2 Freshwater demand

To guarantee steady state conditions in the system, it is necessary to restrictthe objective functions by the mass balance between unitary operations, andby the maximum and minimum allowed contaminant concentrations on theinfluxes and discharges of operations [14].

The flow-rate required in each unitary operation is related to the massload of contaminants (∆mi,k,tot) discharged by operations. This is describedin Equation 5.

fi = maxc

∆mi,k,tot

cmaxi,k,out − cmax

i,k,in

(5)

7

where fi is the freshwater flow rate for operation Oi; ∆mi,k,tot is the total masstransfer for each contaminant, k, to the water used at operation Oi (this termis also known as the contaminant mass charge [3] and is expressed in kg/h);cmaxi,k,out and cmax

i,k,in are the maximum allowed concentration of contaminant kon the discharge and influx of operation Oi, in mg/l respectively.

The optimization model depends on the mass balance between all inletsand all outlets of water to the operation Oi. According to Figure 2, theexpression for the mass balance has the form shown in Equation 6.

fi +∑

j 6=i

Xi,j + Xi,R − fi,loss −Wi −∑

j 6=i

Xj,i −XR,j = 0 (6)

where, Xi,j is the reusable water flow rate from other operations, say Oj,in operation Oi; Xi,R is the treated water from the wastewater treatmentplants that can be used in operation Oi; fi,loss is the portion consideredas water loss in the operation or water consumption by the operation; Wi

is the wastewater flow rate from operation Oi; Xj,i is the reusable waterflow rate from operation Oi in operations Oj; and XR,i is the portion of thedischarged water from operation Oi that receives treatment. All flow-ratesare represented in m3/h. TP in Figure 2 represents a treatment plant.

k different contaminants can be considered in the optimization model.This consideration requires the definition of constraints to restrict the con-centration of contaminants at the inlets and outlets of operations, in order toguarantee that water influxes will not affect the operation performance, andto avoid the violation of environmental or operation norms. The satisfactionof these constraints will determine the quantities of fresh and reused waterto supply to operations. The contaminant concentration constraint at theinflux of the ith operation, ci,k,in is defined by Equation 7.

ci,k,in =

∑

j 6=i Xi,jcj,k,out + ck,0Xi,R − fi,losscmaxi,k,in

∑

j 6=i Xi,j + fi + Xi,R − fi,loss

≤ cmaxi,k,in (7)

where, cj,k,out is the concentration of contaminant, k, at the discharge ofoperation Oj, ck,0 is the concentration of contaminant k in the treated water,ci,k,in is the maximum allowable concentration of contaminant k at the influxof operation Oi. Concentrations are expressed in mg/l.

The same way, contaminant concentration constraint at the outlet of j th

operation, cj,k,out is defined by Equation 8.

8

cj,k,out = ci,k,in +∆mi,k,tot

∑

j 6=i Xi,j + fi + Xi,R − fi,loss

≤ cmaxi,k,out (8)

Finally, non negativity constraints are established according to the fol-lowing equations.

Xi,j ≥ 0,

fi ≥ 0,

Lj × PCj × CFj ≥ 0.

3 Solution methods

For multi-objective optimization problems there is not a single solution, buta set of non dominated solutions (Pareto-set), such that the quality of asolution can be improved with respect to a single criterion only by becomingworse with respect to at least one other criterion [5].

In this sense, we propose the use of two techniques especially designedto solve optimization problems with more than one criterion. The first, usesan aggregated function constructed with the use of weight coefficients repre-senting the relative importance of the two objective functions. The resultingoptimization problem is solved by the reduced gradient method for five com-binations of weights to construct the Pareto set. The second technique, calledMDQL, is a heuristic approach based on the solution of Markov decision pro-cesses [15]. MDQL is capable of exploiting the knowledge acquired duringthe solution process, and has been tested on several benchmark problemsshowing good performance (e.g., [15], [16]).

3.1 Aggregated function

This approach is probably the most known and simplest way to solve this typeof problems. Some of the first references on it are [12] and [26]. The mainidea is to construct a weighted combination of the objective functions. Theweighted function is then used on a single objective optimization problem.In general, the weight coefficients, pi, are real values such that pi ≥ 0∀i =1, . . . , k for k objectives. It is also recommended to use normalized weight

9

Table 3: Weight coefficient combinations used by the reduced gradient ap-proach.

Combination p1 p2

1 0.10 0.902 0.25 0.753 0.50 0.504 0.75 0.255 0.90 0.10

coefficients, so∑k

i=1 pi = 1. More precisely, the multi-objective optimizationproblem is transformed to the problem stated in Equation 9, which will becalled from now on the “weighted problem”.

mink

∑

i=1

pi · Fi (9)

where, pi ≥ 0∀i = 1. . . . , k and∑k

i=1 pi = 1.This approach guarantees the optimality of the Pareto set if the weighted

coefficients are positive or the solution is unique [18]. Pareto set construc-tion is made with the variation of the weight coefficients values, solving theweighted problem as many times as the number of variations of the weightcoefficients can be configured. This procedure can be computationally ex-pensive and slow, although it is a simple approach to generate some Paretosolutions.

The weighted problem of the two objective functions presented in sec-tion 2, is shown in Equation 10.

F = p1

∑

i

fi + p2(∑

j

cstj + TPC) (10)

Its solution is obtained through the reduced gradient method with the useof the GAMS/MINOS program [11]. The weight coefficients combinationsused are included in Table 3.

3.2 Multiple Objective Distributed Q-Learning(MDQL)

In order to efficiently solve optimization problems with more than one ob-jective function it is desirable to use population based approaches, that is,

10

approaches with the capability to generate more than one solution concur-rently. Moreover, it is necessary to apply the dominance optimality criterionto evaluate the generated solutions. This is the main hypothesis of much ofthe recently developed approaches designed to efficiently solve multi-objectiveoptimization problems based on evolutionary computation. However, evolu-tionary approaches do not exploit the knowledge generated along the searchprocess [24].

Taking advantage of some of the characteristics of evolutionary approaches,optimization problems can be solved considering the search processes of aMarkov decision problem. Similar ideas have been previously used with theAnt Colony Optimization Meta-heuristic [8, 9].

MDQL considers a group of agents searching a terminal state, st, in anenvironment formed by a set of states, S. The set of states, or environmentis constructed with the division of the parameter space into a fixed numberof parts, considering that all the decision variables can be discretized into afinite number of divisions. Each division is considered as a state, as illustratedin Figure 3. An environment with these characteristics allows the agents topropose values for each one of the decision variables in the problem.

For each state, s ∈ S, a set of actions, As, is established, see Figure 3. Allstate-action pairs have an associated value function, Q(s, a), indicating thegoodness of taking action a in state s, for reaching a terminal state st ∈ S(complete a task).

The search mechanism for an agent in MDQL operates when an agentlocated in a state selects an action based on its value function, Q(s, a). Mostof the times the agent selects the best evaluated action (the action withthe higher estimated value for Q(s, a)), and sometimes a random action isselected with a probability ε ≈ 0. Action value functions are updated de-pending on how useful an action can be for an agent to reach a terminalstate. This behavior is adjusted with the help of a reward value, r ∈ <, andthe value function for the best evaluated action in the future state reachedby the agent after the execution of the selected action, Q(s′, a′). This updaterule is expressed in Equation 11.

Q(s, a)← Q(s, a) + α[

r + γ maxa′∈A′

s

Q(s′, a′)−Q(s, a)]

(11)

where Q(s, a) is the value function for the action, (0 ≤ α ≤ 1) is the learningstep, r is an arbitrary reward value, r ∈ <, γ is a discount factor, s′ and a′

are the next state and the best evaluated action for s′ respectively.

11

sx 2

sx n−1

sk 2

xmin

y z kmin minmin

initialstate

As0

s0

x y z k

variables

xmax

y z kmax maxmax

sk p−1

sz 2

sz o−1sy m−1

sy 2

Asx6

finalstatest

Figure 3: Variable space division for MDQL.

As an agent explores the state space, the Q(s, a) estimates improve gradu-ally, and, eventually, each maxa′∈A′

sQ(s′, a′) approaches: E {

∑∞n=1 γn−1rt+n}

[23]. Here rt is the reward received at time t due the action chosen at timet − 1. Watkins and Dayan [25] have shown that this Q-learning algorithmconverges to an optimal decision policy for a finite Markov decision process.

In MDQL there is a group of agents, instead of a single agent, interactingwith the environment described above, and since the task for the agents is theconstruction of the Pareto set, the original Q-Learning [25] algorithm mustbe adapted. The main adaptations considered in MDQL are listed below.

• Decision variables in the environment have a predefined order, theagents move in the decision variables space obeying this order, so thedefinition of the values for the decision variables is made in the same or-der by all the agents. Each agent assigns a value for a decision variableat a time.

• When all the agents finish (set values for all the decision variables),all solutions are evaluated using the Pareto dominance criterion. En-vironments for non dominated solutions and solutions that violate anyconstraint remain in memory to be used in future episodes.

• Agents are randomly assigned to the environments in memory.

12

• Action values are updated in two stages. The first is made when agentsmake a transition using a ‘map’ of the environment. Maps are con-structed making a copy of the environments in memory, and are usedby agents to show to the rest of the agents the experience acquiredduring the search process [17]. This experience is represented by theactualization of the action value functions in the ‘map’ using the Q-learning rule of Equation 11. At the end of an episode and after theevaluation of solutions, non dominated solutions receive a positive re-ward and solutions violating any constraint receive a negative reward,which is used to update the original value functions in the environmentwhere they were found (second stage). After the update procedure,all ‘maps’ are destroyed and a new episode initiates. More details ofMDQL algorithm can be found in [15] and [6].

4 Test cases

The proposed mathematical model was validated and MDQL was tested onthree cases described in the following sections. The MDQL operation param-eters used for all test cases were: α = 0.1, γ = 0.9, ε = 0.01 and r = 1 for nondominated solutions and r = −1 for solutions violating constraints. Previousvalues for the operation parameters in MDQL are in some sense typical andwere originally suggested in [24]. Some work related with the sensitivityof the algorithm to these parameters is presented in [16] using benchmarkevaluation functions. The conclusion of the previous work indicates that thebest combination of values for the operation parameters is to consider α ≈ 0,γ ≈ 1 and ε ≈ 0.

4.1 Four Water Using Operations

The first test case considers four water using operations (O1, O2, O3, and O4)and a single freshwater source. Table 4 shows the allowable values for fresh-water, fi, the maximum concentration at influxes, cmax

i,k,in, and at discharges,cmaxi,k,out, and the total mass transfer for contaminants, ∆mi,k,tot, for the four

unitary operations. The objective is to find a network configuration connect-ing the four unitary operations to the source, with the lowest cost and thelowest freshwater consumption, considering reuse as the sole mechanism toreduce freshwater demands.

13

Table 4: Operation parameters for test case 1.Operation f (m3/h) cmax

i,k,in (mg/l) cmaxi,k,out (mg/l) ∆mi,k,tot (g/h)

O1 20 0.0 100.0 2, 000O2 100 50.0 100.0 5, 000O3 40 50.0 800.0 30, 000O4 10 400.0 800.0 4, 000

O1

O2

O3

O4

100 m

100 m

100 m

100 m

100 m

20 t/h

50 t/h

37.5 t/h

5 t/h

waste waterFreshwater

source

Figure 4: Non optimized solution for the first test case.

Figure 4 shows a non optimized solution where water demands in all waterusing operations are satisfied with freshwater, resulting in a flow of 112.5 t/hand with a cost of infrastructure of $1, 875.00 monetary units.

MDQL implementation for the solution of this test case considers therange and number of divisions for the variables shown in Table 5. Thesevalues represent increments of 0.2 m3/h for all variables, which is the criterionused for the parameter space partition in the three problems. For exampleconsidering O1, with a maximum freshwater flux equal to 0 ≤ f1 ≤ 20(m3/h),it is divided in 100 states (each state represents a variation of 0.2m3/h forf1). The same number of divisions is considered for the rest of the variables.Each action moves the agent to a state of the next consecutive variable,i.e. assigns a value in the discretized space of the next consecutive variable.Figure 5 shows two traces of two different agents. Each of the two tracesrepresents a solution to the optimization problem, that is a set of values forthe parameters of the problem.

Twenty agents are considered for the solution of all cases. When all agentsconclude an episode (definition of the 24 variables), the 20 solutions obtained

14

are evaluated under the dominance criterion. Non dominated solutions re-ceive a reward value, r, used to update that state-action value function par-ticipating in its construction. The environment settings with non dominatedsolutions and solutions violating constraints are stored in memory and usedin the next episode.

Results obtained by MDQL are shown in Figure 6 represented with ∗, cor-responding to the best solutions found from 20 different algorithm runs withthe same parameters. Figure 6 also includes the five solutions obtained bythe weighted function approach, represented with ◦. As can be appreciated,MDQL generated more solutions over the Pareto front, 13, some of themcoincide with those found by the mathematical programming approach. Thesolutions found by the weighted functions approach are shown in Figure 7.It is also important to notice that for this type of problem the Pareto set issmall because of the restrictions.

CPU time to complete the Pareto front using the aggregated functionapproach and mathematical programming (solution marked with the ◦) wasaround 5 seconds. This time is obtained with the sum of the CPU timerequired to obtain each of the five solutions on the Pareto front, not consid-ering the time required to change the weight coefficients. Besides, the averageCPU time required by MDQL to define the Pareto Front presented in Fig-ure 6 was around 8 seconds. Considering that the Pareto Front obtainedby MDQL contains 13 solutions and if we extrapolate the time taken by themathematical approach, it will require 13 seconds for the same number ofsolutions found by MDQL.

MDQL was also tested with a finer discretization ranges, the double ofstates per variable, obtaining the same solutions. This behavior of MDQLindicates that the reported Pareto set is the optimal solution for the test caseconsidered and for this discretization level adopted.

4.2 Real industrial problem with ten operations and

four contaminants

The second test case was reported by Alva-Argaez [2] and Alcocer [1]. Thiscase was constructed with real data obtained from an industrial process in theUnited Kingdom, considering ten unitary operations, O = O1, O2, . . . , O10,and four types of contaminants, U = A, B, C, D. Operation parameters areincluded in Table 6.

15

f1

f2

f3

f4

x1,1

x1,2

x1,3

x1,4

x2,1

x2,2

x2,3

x2,4

x3,1

x3,2

x3,3

x3,4

x4,1

x4,3

x4,4

x4,2

w1

w2

w3

w4

so

so so

st

Figure 5: An example of a path taken by two agents in the MDQL imple-mentation for the first test case.

Table 5: Variable space configuration for the first case.Variable Range Number of states

f1 0− 20 100f2 0− 100 500f3 0− 40 200f4 0− 10 50

X1,j ; j ∈ 1, 2, 3, 4 0− 20 100X2,j ; j ∈ 1, 2, 3, 4 0− 100 500X3,j ; j ∈ 1, 2, 3, 4 0− 40 200X4,j ; j ∈ 1, 2, 3, 4 0− 10 50

W1 0− 20 100W2 0− 100 500W3 0− 40 200W4 0− 10 50

16

91 92 93 94 95 96 97 981100

1200

1300

1400

1500

1600

1700

1800

1900

2000Pareto front

Freshwater t/hr

Net

wor

k co

st $

Figure 6: Pareto fronts obtained for the first test case: ◦ mathematicalprogramming solutions; ∗ MDQL solutions; � solution obtained for the min-imization of freshwater consumption as a sole objective.

Table 6: Operation parameters for the second test case.Oper. O cmax

i,j,in(mg/l) cmaxi,j,out(mg/l) f(m3/h)

(A-B-C-D) (A-B-C-D)

O1 200-500-100-1500 25000-20000-28500-230000 24.87O2 350-3000-500-400 8000-9000-24080-3000 40.98O3 350-450-150-500 3500-2500-1500-1500 39.20O4 800-650-450-300 15000-5000-700-1500 4.00O5 1300-2000-2000-4000 2000-7000-9000-10000 3.92O6 3000-2000-100-0 12000-10000-8000-200 137.50O7 450-0-250-560 2000-3000-1000-12000 290.96O8 100-250-200-550 3450-4000-700-7000 23.81O9 150-450-3000-100 1000-1000-4000-100 65.44O10 0-0-0-0 100-100-100-100 4.00

17

5.7

39.4

45.8

3.5

5.7

20

50

24.4

1.6

14.97

O1

O3

O4

O2

5.7

39.8

45

0.9

5.7

20

50.2

21.5

0.8

18.3

O1

O3

O4

O2

5.7

39.2

46.8

4.6

5.7

20

50.2

25.7

1.9

13.4

O1

O3

O4

O2

5.7

39

46.4

7.9

5.7

20

50

26.5

2.08

12.5

O1

O3

O4

O2

5.7

39

44.5

7.9

5.7

20

50.2

26.9

0.024

12.02

O1

O3

O4

O2

Figure 7: Solutions found with the aggregated function approach.

18

550 600 650 700 750 800 8501.25

1.3

1.35

1.4

1.45

1.5

1.55

1.6x 10

4 Pareto front

Freshwater t/hr

Net

wor

k co

st $

Figure 8: Solutions found by MDQL on the second test case ∗ and the solutionfound with the a single objective model ◦.

This problem was also solved in [3] by mathematical programming con-sidering the sole objective function of freshwater minimization. Reportedresults for this test case indicate that the optimal value for the objectivefunction is 594.80m3/h, which is identical to the result reported in [2]. Thesimilarity of results indicates that the two models are identical and that theresults reported in this work can be compared with them.

For this second case only MDQL was applied due to the approximation ofthe Pareto fronts obtained with MDQL and the mathematical programmingapproach observed in the first case and other problems previously solved[15, 16]. Figure 8 includes the solutions obtained with MDQL. The solutionreported in [3] is also plotted. As in the first test case, the solution for thesingle objective function is located in the upper left corner of the graph,the region for the lowest flow rates and highest costs. This seems logicalbecause the solution of the optimization problem with a single freshwaterminimization objective function is equivalent to a zero weight coefficient p2 forthe cost objective function Z2 in the mathematical programming approach.

In Figure 8 it can be appreciated that the solution obtained for the singleobjective function (reported in [3]) is non dominated with respect to thoseobtained by MDQL, with the best evaluation for the freshwater minimizationcriterion. This situation is also presented in the solution of the first test case.

19

0 150 300 450 600 750 900 1050 1200 1350 15005

6

7

8

9

10

11

12

Number of Function Evaluations

PA

reto

Cou

ntin

g

Figure 9: Average number of solutions in the Pareto Front vs Average numberof function evaluations in MDQL.

This behavior can be attributed to the precision of float numbers, and to thediscretization levels for the parameter space. The combination of these twofactors causes a truncation of the variables, and consequently in the solutions.Reported solutions are the best from ten executions of the algorithm with thesame operation parameters. In four of of the ten executions MDQL convergeto the same Pareto Front reported in Figure 8, in two executions only nineof the ten solutions reported were found and in four cases only eight solutionswere found. Otherwise, the mean number of solutions in the Pareto Front(Pareto Counting) is presented in Figure 9.

The solutions obtained for the second test case make evident the advan-tages when more than one criteria are considered, as there is more flexibilityto take a good decision. Additionally, and considering the type of problemsexposed in this paper, the computational and analysis effort required to solvea problem with more than one criteria is not much greater than the requiredfor the single objective case, but results are more valuable, from the point ofview of information content.

Average CPU time taken by MDQL to obtain the Pareto Front presentedin Figure 8 was around 20 seconds.

20

4.3 Water distribution system of Cuernavaca

The Cuernavaca city water distribution system, in Mexico, operates as illus-trated in Figure 10. There are three different types of sources of freshwaterin the city, according to the National Water Commission (NWC): 42 watersprings supplying 1, 409 l/s, 328 deep wells with a contribution of 1, 503.58l/s, and water wheels contributing with 751.50 l/s1.

Water users are classified into five categories according to the water worksuser census. A brief description of the kind of exploitation given to waterby each category is given below, accompanied with their freshwater demandtaken from [21]. In order to be consistent with the nomenclature previouslyused, every category is considered as an unitary operation.

Self service: Users that have its own source to satisfy any kind of needsincluding human consumption. Water demand for this type of users is1.36 l/s.

Industrial: Users exploiting water to operate only industrial processes inwhich there are no human needs to satisfy. Water demand for thisunitary operation is 47.58 l/s.

Agriculture: Covers all the users exploiting freshwater only for irrigation.The main crops cultivated in the region of Cuernavaca are rice, corn,grass and rose trees. This is the second most water demanding opera-tion in the system with 593.00 l/s.

Services: Users with high consumption rates, such as hotels, schools, restau-rants, supermarkets, etc. Freshwater demand for this operations as-cends to 16.19 l/s.

Urban & Public: Most of the domestic users in the city, including smallschools, stores, public offices and small workshops. This is the mostdemanding operation with a demand of 3, 003 l/s.

Multiple: Users not classified in any of the previous categories with anactivity that can be classified as a service, but with less consumptionrate. This operation demands 2.24 l/s.

1It is important to note that the net extractions and run offs from the sources reportedare greater because they also supply freshwater to other other towns close to Cuernavaca.

21

Table 7: Inflow and outflow limit concentration for all current operations inthe city of Cuernavaca.

BOD5 TSScmaxi,A,in cmax

i,A,out ∆mi,tot cmaxi,B,in cmax

i,B,out ∆mi,totOperation Omg/l mg/l kg/h mg/l mg/l kg/h

Urban & Public 0.00 220.00 1, 767.74 0.00 220.00 1, 403.07Services 0.00 220.00 9.53 0.00 200.00 7.56

Agriculture 50.00 350.00 449.57 50.00 300.00 449.57Multiple 0.00 220.00 1.32 0.00 220.00 1.05Industrial 0.00 874.00 85.57 0.00 371.00 36.32

Self Service 0.00 220.00 0.60 0.00 240.00 0.73

It is relevant to note that part of the demanded water is consumed bythe operation itself, another part cannot be registered and is considered asa loss caused by leaks occurring along the distribution systems. The restis declared as wastewater and is supposedly discharged with the effluents tothe receiving water bodies. For Cuernavaca city this body is the Apatlacoriver. It is estimated that the water consumption and the flow lost in leaksis about 43.41% of the water demanded by operations [22]. This estimationis illustrated with the label ‘leaks & consumption’ in Figure 10 for everyoperation considered.

Two contaminants indexes are considered, in connection with the con-taminants threw by the operations to the effluents, 5 day biochemical oxygendemand (BOD5) and total suspended solids (TSS). These indexes are usedin the general water quality index, according to the NOM-001-ECOL-1996standard, which is the Mexican official standard for wastewater discharges.Wastewater treatment plants treat 339.15 l/s to BOD5 and TTS mean con-centration of 50 mg/l according to the data reported in the literature [3].

Values for both water quality indexes, cmaxi,k,out, were established using in-

formation from studies that evaluated the degree of contamination in theApatlaco river [20]. For both contaminants, the concentration in the fresh-water supplied to the system is considered to be zero, see Table 7.

There are 15 wastewater treatment plants in Cuernavaca city, ten of thoseplants treat municipal wastewater while five plants are used for treatment ofindustrial wastewater. The total treated wastewater flow-rate is 364.15 l/s[20]. Table 8 shows the design and current operation treatment capacity for

22

Table 8: Cuernavaca city municipal and industrial wastewater treatmentplant capacity[20].

Design l/s Operation l/s

Municipal Plant 1 15.00 13.00Municipal Plant 2 27.00 9.00Municipal Plant 3 38.00 13.00Municipal Plant 4 1.70 1.50Municipal Plant 5 0.16 0.15Municipal Plant 6 400.00 300.00Municipal Plant 7 4.00 3.00Municipal Plant 8 8.00 0.00Municipal Plant 9 10.00 0.00Municipal Plant 10 8.00 0.00

Total 511.86 339.65

Industrial Plant 1 23.00 16.00Industrial Plant 2 0.90 0.80Industrial Plant 3 5.00 3.50Industrial Plant 4 4.00 3.60Industrial Plant 5 1.40 0.60

Total 34.50 24.50

the 15 plants in the city.As can be seen, municipal plant 6 operates at 75% of its capacity, while

the rest of the plants work at an average of 35.44% of their design capacity,treating only 39.65l/s when they could treat 111.86l/s. Since great quanti-ties of wastewater can be treated with the same infrastructure, without theinvestment in new treatment plants, this can be seen as an opportunity areathat can help to improve the system’s performance.

Two strategies were evaluated to improve the performance of the system.The first one considers the operation of the treatment plants at their cur-rent operation capacity, that is 339.65l/s, and the reduction of leaks in thedistribution network from 43% to 25%. The second strategy considers theoperation of the existing treatment plants at their design capacity, 511.86l/s, a leak reduction program to decrease the non accounted water from 43%to 25%, and the construction of new additional water treatment plants toincrease the treatment capacity in the city to 100%. The second strategy

23

was designed to improve the water quality in the receiving body according tothe NOM-001-ECOL-1996 official standard, in this case the Apatlaco river.

4.3.1 Results for the first strategy

This strategy reduces Cuernavaca city water distribution network leaks from43% to 25%. The waste water treatment plants operation maintain theircurrent levels, that is, 339.65 l/s. Urban and public water demand is coveredwith 3, 003 l/s from wells, springs and water wheels, but 1, 291.6 l/s, thatis, 43% of this demand is lost through domiciliary and network leaks (seeFigure 10).

Results are presented in Figure 11. The main change in the operation ofthe water distribution network in Cuernavaca found by MDQL is to supplywater for agriculture from three different sources: the mayor quantity fromthe wastewater treatment plants, followed by the freshwater and a minorquantity of wastewater from the urban and public sector. This operationguarantees acceptable levels of contaminant according to standards. Themain water savings is obtained from the freshwater that is no longer sup-plied to agriculture. The results can be analyzed from two perspectives: a)increment of the irrigated area since there is an increment of water availabil-ity in the region, and b) reduction of freshwater sources exploitation with abenefit to the environment.

Analyzing the left most solution in Figure 11, which is shown in Figure 12,it can be seen that the total demanded freshwater flow-rate by the systemis 2, 752.6 l/s, representing a decrement of approximately 24.87 % (see Fig-ure 10), that is, 911.57 l/s of the amount of water taken from the sources.As previously mentioned, the main change is in agriculture, for which waterdemand flow-rate could be satisfied with 339.65 l/s taken from wastewatertreatment plants, 9, 96 l/s with wastewater from the urban and public sector,and 234.15 l/s with fresh water taken from springs in the city.

This savings in freshwater can be used to increase the irrigated surface inabout 511 ha of corn, 1, 080 ha of beans, 219 ha of sugar cane, or 859 ha ofonion, considering irrigation depths of 74, 35, 172, and 44 cm respectively.

On the other hand, freshwater savings represent approximately 28.74 mil-lions of m3 per year that could increase water availability in the Cuernavacavalley aquifer from eight millions of cubic meters to 36.74 millions of cubicmeters.

Finally, it can be appreciated that the solution of the bi-objective model

24

Apatlaco river

and consumption

and consumption

and consumption

and consumptionleaks

and consumptionleaks

and consumptionleaks

Water Sources

Wells

Water Wheels

Springs1,409 l/s

1,503.58 l/s

3,664.08 l/s

751.5 l/s

leaks

leaks

leaks

Domestic

Industrial

Multiples

Services

1.36 l/s

2.24

47.58 l/s

l/s

16.19 l/s

0.58 l/s

20.46

0.963

6.96

67.37 l/s

1,436.21 l/s

0.78 l/s

BOD5 = 220 mg/l

27.12 l/s

874mg/l

1,276 l/s

220 mg/l

9.23 l/s

220 mg/l

Urban/Public

Treatmentplants

Agriculture

816 l/s

1,409 l/s

593 l/s 177.9 l/s

1,291 .59 l/s 339.65 l/s

1,972.47 l/s

220 mg/l

339.65 l/s50 mg/l

415.1 l/s350 mg/l

Figure 10: Current operation of the distribution system of the city of Cuer-navaca, Mexico.

25

2750 2800 2850 2900 2950 30001.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45x 10

4 Pareto front

Freshwater t/hr

Net

wor

k co

st $

F1= 2,752.6 l/s

F2= $ 14,156

Figure 11: Cuernavaca city distribution system results for the first strategy.

for the case of the distribution network of Cuernavaca city with MDQL ap-proach allows, the construction of a Pareto set with three optimal solutions.Only the left most one was analyzed in detail since it represents the lowestfreshwater demand solution, but the same analysis can be made with theother two.

4.3.2 Results for the second strategy

The second operation strategy considers the operation of existing wastewa-ter treatment plants to their design capacity, that is, treatment capacity isincreased to 511.86 l/s with no investment cost. As in the previous strategy,a leak reduction to 25% is considered.

The Pareto front obtained for this test case is shown in Figure 13. MDQLwas capable of finding four solutions. The left most solution with the lowestfreshwater demand of 2, 581.3 l/s and cost of $1, 796 is shown in Figure 15.The construction of two new wastewater treatment plants is proposed in thissolution. The first proposed wastewater treatment plant capacity is 81.38l/s. This proposed plant can receive 4.6 % of the discharged wastewaterfrom the urban and public sector, and 53 % of the discharged water by themultiple sector.

The second proposed wastewater treatment plant capacity is 1, 130.44 l/s

26

TSS

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

Freshwater

Springs

Waterwheels

Wells

Urban andpublic

Municipaltreatment plants

Agriculture

234.15 l/s

2,463.07 l/s

816 l/s

751.5 l/s

895.57 l/s

2,463 l/s

234.15 l/s

750.92 l/s339.65 l/s286 mg/l227 mg/l

339.65 l/s

9.96 l/s285 mg/l

50 mg/l50 mg/l

592.99 l/s

177.89 l/s

415.09 l/s225.91 mg/l350 mg/l

1362.54 l/s286 mg/l227 mg/l

227 mg/l

Wells

Self service

Multiple

Service

Industrial

9.29 l/s286 mg/l226.92 mg/l

0.78 l/s214.4 mg/l250 mg/l

1.28 l/s286 mg/l227.96 mg/l

27.12 l/s874 mg/l370.97 mg/l

39.02 l/s

13.27 l/s

1.84 l/s

1.12 l/s

0.34 l/s

0.56 l/s

4.04 l/s

11.9 l/s

BOD5

Figure 12: Solution with the lowest freshwater demand in the Pareto setfound for the first strategy.

27

2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 30001.5

1.55

1.6

1.65

1.7

1.75

1.8x 10

4 Pareto front

Freshwater t/hr

Net

wor

k co

st $

F1= 2,518.3 l/s

F2= $ 1,796

Figure 13: Cuernavaca city distribution system analysis results obtained bythe second strategy.

and could receive the rest (95.4 %) of the discarged wastewater by the urbanand public sector.

Similarly to the results found for the first strategy, demanded water byagriculture is satisfied with the total of the municipal treated water, andwith 81.38 l/s coming from the new plant proposed in the design. Industrialwater is treated independently in the existing industrial treatment plants.

Water savings arise since a considerable flow of freshwater is not longersupplied to the agriculture sector. This reduction represents an increase ofthe irrigated surface in the Cuernavaca valley of approximately 642 ha forcorn, 1, 357 ha for beans, 276 ha for sugar cane, or 1, 080 ha for onion,with water depths of 74, 35, 172 and 44 cm, respectively. There are 34.15millions of cubic meters per year of freshwater savings that could incrementthe freshwater availability of the aquifer from eight to 44.13 millions of cubicmeters per year.

As can be seen from Figure 13, the left most solution is the lowest fresh-water flow-rate demand solution of both strategies. It is also the highest costsolution, but at the same time all the discharged water by the Cuernavaca citywater distribution system is treated so it represents the lowest contaminationlevels (see Figure 15 and Figure 12 wastewater discharges to the Apatlacoriver). Qualitative, efficiency is measured in terms of the remaining contam-

28

2500 2550 2600 2650 2700 2750 2800 2850 2900 2950 30001.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8x 10

4 Pareto fronts for the two strategies

Freshwater t/hr

Net

wor

k co

st $

PF for the first strategy PF for the second strategy

Figure 14: Comparison of the Pareto fronts obtained for the two strategiesused for the Cuernavaca city water distribution system.

inant concentration in discharged wastewater to the reception bodies. Thisparameter is not included in the optimization model, but according to theenvironmental standards (included in the model) solutions for both strategiesare feasible and do not violate them.

Figure 14 shows the Pareto fronts obtained with the two strategies andfacilitates the analysis and decision making. It is desirable to constructgraphs with Pareto fronts obtained from different strategies to appreciatepotential benefits. In this case it is possible to evaluate that the solutionswith the mayor benefit in terms of freshwater savings is obtained with thesecond strategy with a high cost (left most second strategy solution). Aspreviously mentioned, solutions for the two strategies are in accordance withenvironmental standards, so, if cost is considered as the main criterion fordecision making, a desirable solution could be the left most solution of thefirst strategy.

5 Conclusions

In this work we presented a multi-objective optimization problem for waterdistribution systems using water pinch technology criteria, we evaluated the

29

TSS

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

leaks andconsumption

Springs

Waterwheels

Wells

Urban andpublic

Municipaltreatment plants

Agriculture

2,463.07 l/s

816 l/s

751.5 l/s

895.57 l/s

2,463 l/s

750.92 l/s502.63 l/s286 mg/l227 mg/l

592.99 l/s

177.89 l/s

415.09 l/s300 mg/l300 mg/l

Wells

Self service

Multiple

Service

Industrial

39.02 l/s

13.27 l/s

1.84 l/s

1.12 l/s

0.34 l/s

0.56 l/s

4.04 l/s

11.9 l/s

1,130.44 l/s286 gm/l227 mg/l

Additionaltreatmentplant

1130.44 l/s75 mg/l75 mg/l

Additionaltreatmentplant

79.08 l/s286 mg/l227 mg/l

50 mg/l50 mg/l

51.86 l/s

79.76 l/s30 mg/l30 mg/l

250 mg/l214.4 mg/l0.78 l/s

1.28 l/s286 mg/l227.35 mg/l

0.68 l/s286 mg/l227.35 mg/l

0.6 l/s

286 mg/l22.35 mg/l

9.29 l/s

286 mg/l

226.92 mg/l

27.12 l/s

874 mg/l370.97 mg/l

Industrial wastewatertreatment plants

BOD5

Figure 15: Solution with the lowest freshwater demand in the Pareto frontfound with the second strategy.

30

multi-objective optimization model, and we verified the capability of MDQLto solve complex real problems with highly restricted non convex spaces.

The water pinch optimization model considers more than one criteria.The model considers the reuse of wastewater from operations, wastewatertreatment, consumption flow-rates and leaks in the system. With the re-duction of freshwater demands it is possible to guarantee that the quality ofthe water served to the different users do not violate ecological and sanitarynorms. The bi-objective optimization model operates considering mass bal-ances between operations, freshwater sources, wastewater treatment plants,and wastewater disposal effluents. Contaminants loads from operations towater flows are restricted by environmental and operational constraints, re-sulting in a highly non linear model.

The proposed model was tested on three cases: (i) a four unitary oper-ations and one contaminant; (ii) a real industrial process in the UK withten unitary operations and four contaminants, and (iii) the Cuernavaca citywater distribution operation system. A heuristic approach based on the so-lution of Markov decision processes, MDQL, was used to solve these cases.Although MDQL performance was previously measured on several bench-mark problems [15, 16, 17] with very promising results, we wanted to verifyits performance in the solution of highly restricted real problems. A weightedaggregated function was also used as a mean of comparison, selected on thebasis of previous results and convergence properties reported in the literature[18].

The application of MDQL to the solution of water pinch problems withhighly restricted non linear spaces makes evident that the algorithm is rel-atively robust. MDQL does not depend on complex codings or operatorsand its reinforcement learning approach allows it to improve its performanceduring the search process. We have already tested some of MDQL’s charac-teristics on a previous work [17]. Our results in a highly restricted real worldapplication reinforces our hypothesis that learning during the search processis relevant to find good solutions for optimization problems.

Solutions to water pinch problems, represent important technical chal-lenges that are only partially solved by the industry. The results presentedhere represent an example of how real applications can be solved with theparticipation of multidisciplinary teams involving researches from differentcommunities, as in this case.

As future work we are considering implementing constraints to select moreefficiently different processes. For example, if wastewater treatment technol-

31

ogy is selected in terms of the type of contaminants, the mass remotion couldbe made more effective and the system more efficient if the proper processis selected and optimized in terms of cost and efficiency. Another importantaspect to implement is the cost function, which needs to be extended in orderto quantify operation costs, reuse costs, and other economic factors affectingthe operation of a system with these characteristics.

5.1 Future Research Work

In relation with the future work related with the optimization model pre-sented in this paper, an extension of the model is being prepared. Thisextension considers the analysis in detail of the mass exchange in unitary op-erations opening the possibility to make dynamic analysis of the phenomenon.With the use of this extension it could be possible to make finer optimizationof the process and to construct operation manuals to make the operation ofsystems optimal.

Acknowledgments

This work has been partially supported by CONACyT under grant ref 33000-A and the Mexican Institute of Water Technology. The authors wish to thankto Dr. Velitchko Tzatchkov and Veronica Soto for their technical commentsand reviews.

References

[1] Alcocer, V. and Arreguın, F., Minimizacion de agua de primer uso enprocesos industriales a traves de tecnicas de optimizacion. Memoriasdel XX Congreso Latinoamericano de Hidraulica, La Habana, Cuba,september 2002, (in spanish).

[2] Alva-Argaez, A An Automated design tool for water and wastewater min-imization, PhD Thesis, Department of Process Integration University ofManchester IST, UK 1999.

[3] Arreguın, C., F., Alcocer, Y. V. Modelacion sistemica del uso eficientedel agua. Revista Ingenierıa Hidraulica en Mexico. Vol. XIX, No. 3, july,2004.

32

[4] Athan, T. and Papalombros P. A note on weighted criteria methodsfor compromise solutions in multi-objective optimization. EngineeringOptimization, 27:155-176, 1996.

[5] Back T. Evolutionary algorithms in theory and practice, Oxford Univer-sity Press, 1996.

[6] Coello, C. and Mariano, C., (2002) Multiple criteria optimization. stateof the art annotated bibliographic surveys, Edited by Matthias Ehrgott,Xavier Gambibelux, Kluwer Academic Publishers, Chapter 6.Evolution-ary Algorithms and Multiobjective Optimization, pp. 277-331.

[7] Coetzer, D., Stanley, C. and Kumana, J. Systemic Reduction of Indus-trial Water Use and Wastewater Generation, In Proc. of AIChE NationalMeeting, Houston, March 1997.

[8] Dorigo, M. (1992) Optimization, Learning and Natural Algorithms.Ph.D.Thesis, Politecnico di Milano, Italy, in Italian.

[9] Dorigo M., V. Maniezzo and A. Colorni (1996). The Ant System: Op-timization by a Colony of Cooperating Agents. IEEE Transactions onSystems, Man, and Cybernetics-Part B, 26(1):29-41

[10] Galan, B., and Grossmann, I.E., Optimal Design of Distributed Wastew-ater Treatment Networks, Ind. Eng. Chem. Res., 37:4036, 1998.

[11] GAMS/MINOS http://www.gams.com General Algebraic Modeling Sys-tem Development Corporation, (2001).

[12] Gass S. and Saaty T., The conputational Algorithm for the ParametricObjective Function, Naval Research Logistics Quarterly, 2 (1995).

[13] Kuo, W.C. and Smith, R. Efluent treatment system design, Chem Eng.Sci., 52:4273, 1997.

[14] Mann, G.J. and Liu, Y.A. Industrial Water reuse and wastewater min-imization McGrawHill Eds., 1999.

[15] Mariano, C. and Morales, E., (2000) A new approach for the solution ofmultiple objective optimization problems based on reinforcement learn-ing, Lecture Notes in Artificial Intelligence, Proceedings of the Mexican

33

International Conference on Artificial Intelligence, pp. 212-223, Aca-pulco, Mex, April, 2000.

[16] Mariano, C., (2001)Reinforcement learning in multiobjective optimiza-tion PhD Thesis in computer Science, Instituto Tecnolgico y de Estu-dios Superiores de Monterrey, Campus Cuernavaca, March, 2002, Cuer-navaca, Mor., Mexico.

[17] Mariano, C. and Morales, E., (2001) A new updating strategy for rein-forcement learning based on Q-learning, In P. Flach and L. de Raeldt(eds.), Springer Verlag, Lecture Notes in Artificial Intelligence,vol 2167:12th European conference on Machine Learning, pages 324-335, 2001.

[18] Miettinen K., Nonlinear Multiobjective Optimization, Kluwer AcademicPublishers, (1999)

[19] National Water Commission, Study of fees over wastewater dischargerights, Annual Report, Disenos Hidraulicos e Ingenierıa Ambiental, 1990(In Spanish).

[20] National Water Commission, Clasification estudy in the Apatlaco river,Sanitation and Water Quality Department from the Morelos state office,1996 (In Spanish).

[21] National Water Commission, Hydro-geological update of the aquifer ofCuernavaca, state of Morelos 1999, CNA 2000, (In Spanish).

[22] Ochoa, L., Bourguett, V. Integral reduction of leaks, Mexican Institutefor Water Technology, National Water Commission, September 1998,(In Spanish)

[23] Putterman, M. Markov Decision Processes Discrete Stochastic DynamicProgramming, Wiley Series in Probability and Mathematical Statistic,(1994)

[24] Sutton R., and Barto G. Reinforcement Learning An Introduction, MITPress, (1998).

[25] Watkins, C., Dayan, P., (1992) Q-Learning, Machine Learning, 3:279-292, 1992.

34

[26] Zadeh L., (1963) Optimality and Non-Scalar-Valued Performance Cri-teria, IEEE Transactions on Automatic Control, 8, (1963), 59-60.

35