NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming...

54
PROF. DR. VEDAT CEYHAN NON-LINEAR PROGRAMMING (NLP)

Transcript of NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming...

Page 1: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

PROF. DR. VEDAT CEYHAN

NON-LINEAR PROGRAMMING (NLP)

Page 2: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Similarities & differences

Characteristics of NLP

NLP models

One Variable NLP

Multi Variable NLP

Unlimited NLP

Limited NLP

Basic concept

Case studies

2

Page 3: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Similarities & differences

• Linear programming(LP)– Linear target function+ linear constraints

– Continuous variable

• Integer programming (IP)– Linear target function+ linear constraints

– Discrete variable

• Non-linear programming (NLP)– The objective is linear but constraints are non-linear

– Non-linear objectives and linear constraints

– Non-linear objectives and constraints

3

Page 4: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Characteristics of NLP

Solution is difficult

Solution may tie initial point.

Initial point is subjective.

4

Page 5: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

• NLP forms:– Non-linear objectives

– Non-linear constraints

– Non-linear objectives and constraints

The main problem is algorithm selection

2min ( -3)

. . x< 3

x

s t

1 2

2 21 2

min

. . 4

x x

s t x x

21 2

2 21 2

min ( 2)

. . 4

x x

s t x x

5

Page 6: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

• Constructing model is required for solving optimizationproblem. Mathematical model is based on determination ofvariables and defining their functional relationship

• 3 components of mathematical model

– Objective function,

– Constraints

– Non-negativity restriction

6

Page 7: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

• Linear programming, integer programming and goalprogramming assume that objective function and constraintsis linear. Not include non-linear expressions such as 1/X2, logX3

• NLP procedures does not produce optimum solution everytime in contrary with linear programming (Render ve ark,2012).

7

Page 8: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Objective function in NLP

• gi(x) ≤ bi constraints

• z = f(x) objective function

We find the vector of x= (x1, x2,…, xn), which produce optimum solution for objective

8

Page 9: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

NLP models

• Associated with the number of variable

–One variable or multivariate,

Associated with the presence of restrictions

– restricted or unrestricted

9

Page 10: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

NLP models

Constraints: equality

Limited Model

One variable models Constraints: inequality

Unrestricted Model

Constraints: equality

Limited Model

Multivariate models Constraints:inequality

Unresticted Model

10

Page 11: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Basic concepts

• Increasing and decreasing function

y=f(x)

x1 and x2 is a random figüre

If the function is f(x1)<f(x2) when inequality is x1<x2,

then function is called increasing.

If the function is f(x1)>f(x2) when inequality is x1<x2,

then function is called decreasing.

11

Page 12: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Increasing and decreasing function

Increasing function Decreasing function

12

Page 13: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

– Local Maximum and Minimum

f(x)’’> f(x)’

f(x)’’< f(x)’ local min. or max

–Gradiant of function

Gradiant function of Z=f(x1,x2,…,xn) is vector offirst derivatives.

13

Page 14: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Hessian Matrix

• Hessian matrix is a nxn matrix of second partial derivative of f(x1,x2,…,xn) function

14

Page 15: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

In LP, solution region is convex set andoptimum solution is one of the corner point

In NLP, it is not necessary that optimumsolution is one of the corner point

15

Page 16: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Concave function Convex function

16

Page 17: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Example:

1) Is f(x)= √x function whether convex or concave at [S= 0, ∞)?

f(x)=√x concave function

Since line between two point take place under the curve, the

function is concave.17

Page 18: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

2) Is f(x)=x3 function whether convex or concave at S = R1=(-∞, ∞)?

f(x)=x³ Neither convex nor concave function

18

Page 19: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

3) Is f(x)=3x-3 function whether convex or concave at S = R1=(-∞, ∞)?

1

Both convex and concave function

-3

19

Page 20: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

One variable unrstricted NLP

• Deal with finding maximum or minimum pointof one variable function with no restriction

20

Page 21: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Multivariate unrestricted NLP

• Deal with finding optimum point(x1*,x2*,…,xn*) that is maximum or minimumpoint of multivariate function of f(x1,x2,…,xn)with no restriction

• Demand function explained by price,preference, income, etc. İs a example formultivariate unrestricted NLP.

21

Page 22: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Restricted NLP

• Examining the colineairty among x1,x2,……,xn that is variablesof multivariate function of f(x1,x2,……,xn

• f(x1,x2,……,xn) function,

• g1(x1,x2,……,xn) = b1

• g2(x1,x2,……,xn) =b2 restrictions

• gm(x1,x2,……,xn) =bm

We are looking for point that is maximum or minimum underupper restrictions.

22

Page 23: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Solution

• Lagrange function

L(x1,x2,…………….xn,λ1,λ2,……….λm)= f(x1,x2,……..xn) + i[ bi- gi(x1,x2,………xn)]

23

Page 24: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Example

1 1

2

2 2

1 2

1 2

1 2

1 2

2 2

1 2 1 1 2 2 1 2

1 1 2 1

2

Minimize ( 4) ( 4)

subject to 2 3 6

3 2 12

and , 0

The Lagrangian is:

( 4) ( 4) (6 2 3 ) ( 12 3 2 )

Kuhn Tucker Conditions:

2( 4) 2 3 0 0 and 0

2( 4)

x x

x

C x x

x x

x x

x x

Z x x x x x x

Z x x x Z

Z x

2

1 1

2 2

1 2 2

1 2 1 1

1 2 2 2

28 36 161 2 1 213 13 13

3 2 0 0 and 0

6 2 3 0 0 and 0

12 3 2 0 0 and 0

Solution: , , 0,

xy x Z

Z x x Z

Z x x Z

x x

24

Page 25: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Non-linear objective function and linear constraints

25

Page 26: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Case of non-linear objective function and linear constraints

Great Western Appliance firm sell toast machine ofMikrotoaster (X1) and Self-Clean Toaster Oven (X2). Firm gainnet profit by $28 per toaster. Profit function is 21X2+0,25

Objective function is non-linear:

Maximum profit =28X1+21X2+0,25

There are two linear constraints

X1+X2 ≤ 1,000 (production capacity)

0.5X1+0.4X2 ≤ 500 (term of sale)

X1, X2 ≥ 0

(Source: Render ve ark., 2012)

26

Page 27: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

2. Quadratic programming

Objective function includes terms such as 0,25 and when the restrictions is linear

Quadratic programming problems can be solved by using adjusted simplex algorithms.

(Source: Render and ark., 2012)

27

Page 28: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

EXAMPLES

28

Page 29: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

WIN QSB for NLP

29

Page 30: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

WİN QSB interface

30

Page 31: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

WİN QSB EKRANINI TANIYALIM

31

Page 32: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Unrestricted NLP example

• minimum x(sin(3.14159x))

• 0 <= x <= 6

• We have one non-linear expression such as sin(3.14159x) for minimizing objective function

32

Page 33: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

33

Forunrestrictedcase, enter

“0”

Page 34: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Unrestricted NLP example

34

Page 35: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

35

Attention!

Interval of X1

Graphicalsolution

Page 36: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

x=3.5287 Objective function= -3.5144.

36

Page 37: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Example 2:

Objective function:

maximum 2x1 + x2 - 5loge(x1)sin(x2)

Constraintsx1x2 <= 10 | x1 - x2 | <= 20.1 <= x1 <= 5 0.1 <= x2 <= 3

37

Page 38: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

38

Page 39: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

39

Page 40: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

x1=3.3340 ve x2=2.9997 Objective function=8.8166

40

Page 41: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Non-linear function and constraints

41

Page 42: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

• In medium size hospital having 200-400 patient bed,Hospicare Corporation, annual profit depend onnumber of patient(X1) ve number of patientsurgeon(X2) bağlıdır.

• Non-linear objective function for Hospicare :$13X1 + $6X1X2 + $5X2 + $1/X2

Constraints;

• 2X12+4X2≤90 (nursery capacity)

• X1+X23≤75 (X-ray capacity)

• 8X1-2X2≤61 (marketing budget)

42

Page 43: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

43

Page 44: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

44

Page 45: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

45

Page 46: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Example:Pickens Memorial Hospital

Patient demand exceeds capacity of hospital

Objective: Maximum profit

46

Page 47: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Decision variables

M =number of served patient

S = number of served patient for surgery

P = number of served child patient

Profit functionWhen increasing patient, profit increasing non-

linearly.

47

Page 48: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Constraints

• Hospital capacity: Total 200 patient

• X-ray capacity: 560 x-rays per week

• Marketing budget: $1000 per week

• Laboratory capacity : 140 hours per week

48

Page 49: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

Objective function

Max 45M + 2M2 + 70S + 3S2 + 2MS + 60P + 3P2

Constraints:

M + S + P < 200 (patient capacity)

M + 3S + P < 560 (x-ray capacity)

3M + 5S + 3.5P < 1000 (budget $)

(0.2+0.001M)x(3M+3S+3P) < 140 (lab hour)

M, S, P > 0

49

Page 50: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

50

Page 51: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

51

Page 52: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

52

Page 53: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

53

Page 54: NON-LINEAR PROGRAMMING (NLP) · • Linear programming, integer programming and goal programming assume that objective function and constraints is linear. Not include non-linear expressions

54

Objective functionMax 45M + 2M2 + 70S + 3S2 + 2MS + 60P + 3P2

Constraints:

M + S + P < 200 (patient capacity)

M + 3S + P < 560 (x-ray capacity)

3M + 5S + 3.5P < 1000 (budget $)

(0.2+0.001M)x(3M+3S+3P) < 140 (lab hour)

M, S, P > 0