Download - Research Article A Conjugate Gradient Type Method for the ...A Conjugate Gradient Type Method for the Nonnegative Constraints Optimization Problems CanLi ... is a steplength obtained

Hindawi Publishing CorporationJournal of Applied MathematicsVolume 2013 Article ID 986317 6 pageshttpdxdoiorg1011552013986317

Research ArticleA Conjugate Gradient Type Method for the NonnegativeConstraints Optimization Problems

Can Li

College of Mathematics Honghe University Mengzi 661199 China

Correspondence should be addressed to Can Li canlymathe163com

Received 16 December 2012 Accepted 20 March 2013

Academic Editor Theodore E Simos

Copyright copy 2013 Can Li This is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

We are concernedwith the nonnegative constraints optimization problems It is well known that the conjugate gradientmethods areefficient methods for solving large-scale unconstrained optimization problems due to their simplicity and low storage Combiningthe modified Polak-Ribiere-Polyak method proposed by Zhang Zhou and Li with the Zoutendijk feasible direction method weproposed a conjugate gradient type method for solving the nonnegative constraints optimization problems If the current iterationis a feasible point the direction generated by the proposed method is always a feasible descent direction at the current iterationUnder appropriate conditions we show that the proposed method is globally convergent We also present some numerical resultsto show the efficiency of the proposed method

1 Introduction

Due to their simplicity and their low memory requirementthe conjugate gradient methods play a very important rolefor solving unconstrained optimization problems especiallyfor the large-scale optimization problems Over the yearsmany variants of the conjugate gradient method have beenproposed and some are widely used in practice The key fea-tures of the conjugate gradient methods are that they requireno matrix storage and are faster than the steepest descentmethod

The linear conjugate gradient method was proposed byHestenes and Stiefel [1] in the 1950s as an iterative method forsolving linear systems

119860119909 = 119887 119909 isin 119877119899

(1)

where 119860 is an 119899 times 119899 symmetric positive definite matrix Pro-blem (1) can be stated equivalently as the followingminimiza-tion problem

min 1

2119909119879

119860119909 minus 119887119879

119909 119909 isin 119877119899

(2)

This equivalence allows us to interpret the linear conjugategradient method either as an algorithm for solving linear

systems or as a technique for minimizing convex quadraticfunctions For any 119909 isin 119877

119899 the sequence 119909119896 generated by the

linear conjugate gradient method converges to the solution119909lowast of the linear systems (1) in at most 119899 stepsThe first nonlinear conjugate gradient method was intro-

duced by Fletcher and Reeves [2] in the 1960s It is one of theearliest known techniques for solving large-scale nonlinearoptimization problems

min119891 (119909) 119909 isin 119877119899

(3)

where119891 119877119899

rarr 119877 is continuously differentiableThe nonlin-ear conjugate gradient methods for solving (3) have the fol-lowing form

119909119896+1

= 119909119896+ 120572119896119889119896

119889119896=

minusnabla119891 (119909119896) 119896 = 0

minusnabla119891 (119909119896) + 120573119896119889119896minus1

119896 ge 1

(4)

where 120572119896is a steplength obtained by a line search and 120573

119896is

a scalar which deternimes the different conjugate gradientmethods If we choose119891 to be a strongly convex quadratic and120572119896to be the exact minimizer the nonliear conjugate gradient

method reduces to the linear conjugate gradient method

2 Journal of Applied Mathematics

Several famous formulae for 120573119896are the Hestenes-Stiefel (HS)

[1] Fletcher-Reeves (FR) [2] Polak-Ribiere-Polyak (PRP) [34] Conjugate-Descent (CD) [5] Liu-Storey (LS) [6] andDai-Yuan (DY) [7] formulae which are given by

120573HS119896

=nabla119891(119909119896)⊤

119910119896minus1

119889⊤

119896minus1119910119896minus1

120573FR119896

=

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

1003817100381710038171003817nabla119891 (119909119896minus1

)1003817100381710038171003817

2 (5)

120573PRP119896

=nabla119891(119909119896)⊤

119910119896minus1

1003817100381710038171003817nabla119891 (119909119896minus1

)1003817100381710038171003817

2 120573

CD119896

= minus

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

119889⊤

119896minus1nabla119891 (119909119896minus1

) (6)

120573LS119896

= minusnabla119891(119909119896)⊤

119910119896minus1

119889⊤


) 120573

DY119896

=

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

119889⊤

119896minus1119910119896minus1

(7)

where 119910119896minus1

= nabla119891(119909119896)minusnabla119891(119909

119896minus1) and sdot stands for the Euclid-

ean norm of vectors In this paper we focus our attentionon the Polak-Ribiere-Polyak (PRP) method The study ofthe PRP method has received much attention and has madegood progress The global convergence of the PRP methodwith exact line search has been proved in [3] under strongconvexity assumption on 119891 However for general nonlinearfunction an example given by Powell [8] shows that the PRPmethod may fail to be globally convergent even if the exactline search is used Inspired by Powellrsquos work Gilbert andNocedal [9] conducted an elegant analysis and showed thatthe PRPmethod is globally convergent if 120573PRP

119896is restricted to

be nonnegative and 120572119896is determined by a line search satisfy-

ing the sufficient descent condition 119892⊤

119896119889119896

le minus119888 1198921198962 in

addition to the standard Wolfe conditions Other conjugategradient methods and their global convergence can be foundin [10ndash15] and so forth

Recently Li andWang [16] extended themodified Fletch-er-Reeves (MFR) method proposed by Zhang et al [17] forsolving unconstrained optimization to the nonlinear equa-tions

119865 (119909) = 0 (8)

where 119865 119877119899

rarr 119877119899 is continuously differentiable and pro-

posed a descent derivative-freemethod for solving symmetricnonlinear equations The direction generated by the methodis descent for the residual function Under appropriateconditions the method is globally convergent by the use ofsome backtracking line search technique

In this paper we further study the conjugate gradientmethod We focus our attention on the modified Polak-Ribiere-Polyak (MPRP) method proposed by Zhang et al[18] The direction generated by MPRP method is given by

119889119896=

minus119892 (119909119896) 119896 = 0

minus119892 (119909119896) + 120573

PRP119896

119889119896minus1

minus 120579119896119910119896minus1

119896 gt 0

(9)

where 119892(119909119896) = nabla119891(119909

119896) 120573PRP119896

= 119892(119909119896)119879

119910119896minus1

119892(119909119896minus1

)2 120579119896=

119892(119909119896)119879

119889119896minus1

119892(119909119896minus1

)2 and 119910

119896minus1= 119892(119909

119896) minus 119892(119909

119896minus1) The

MPRP method not only reserves good properties of thePRP method but also possesses another nice property thatit is always generates descent directions for the objective

function This property is independent of the line searchused Under suitable conditions the MPRP method withthe Armoji-type line search is also globally convergent Thepurpose of this paper is to develop an MPRP type methodfor the nonnegative constraints optimization problems Com-bining the Zoutendijk feasible direction method with MPRPmethod we propose a conjugate gradient type method forsolving the nonnegative constraints optimization problemsIf the initial point is feasible the method generates a feasiblepoint sequence We also do numerical experiments to testthe proposed method and compare the performance of themethod with the Zoutendijk feasible direction method Thenumerical results show that the method that we proposeoutperforms the Zoutendijk feasible direction method

2 Algorithm

Consider the following nonnegative constraints optimizationproblems

min 119891 (119909)

st 119909 ge 0

(10)

where 119891 119877119899

rarr 119877 is continuously differentiable Let 119909119896ge 0

be the current iteration Define the index set119868119896= 119868 (119909

119896) = 119894 | 119909

119896(119894) = 0 119869

119896= 1 2 119899 119868

119896

(11)

where 119909119896(119894) is the 119894th component of 119909

119896 In fact the index set

119868119896is the active set of problem (10) at 119909

119896

The purpose of this paper is to develop a conjugategradient type method for problem (10) Since the iterativesequence is a feasible point sequence the search directionsshould be feasible descent directions Let 119909

119896ge 0 be the

current iteration By the definition of feasible direction wehave that [19] 119889 isin 119877

119899 is a feasible direction of (10) at 119909119896if and

only if 119889119868119896

ge 0 Similar to the Zoutendijk feasible directionmethod we consider the following problem

min nabla119891(119909119896)119879

119889

st 119889119868119896ge 0 119889 le 1

(12)

Next we show that if 119909119896is not a KKT point of (10) the solu-

tion of problem (12) is a feasible descent direction of 119891 at 119909119896

Lemma 1 Let 119909119896ge 0 and let 119889 be a solution of problem (12)

then nabla119891(119909119896)119879

119889 le 0 Moreover nabla119891(119909119896)119879

119889 = 0 if and only if 119909119896

is a KKT point of problem (10)

Proof Since 119889 = 0 is a feasible point of problem (12) theremust be nabla119891(119909

119896)119879

119889 le 0 Consequently if nabla119891(119909119896)119879

119889 = 0 theremust be nabla119891(119909

119896)119879

119889 lt 0 This implies that the direction 119889 is afeasible descent direction of 119891 at 119909

119896

We suppose thatnabla119891(119909119896)119879

119889 = 0 Problem (12) is equivalentto the following problem

min nabla119891(119909119896)119879

119889

st 119889119868119896ge 0 119889

2

le 1

(13)

Journal of Applied Mathematics 3

Then there exist 120582119868119896and 120583 such that the following KKT con-

dition holds

nabla119891 (119909119896) minus (

120582119868119896

0) + 2120583119889 = 0

120582119868119896ge 0 119889

119868119896ge 0 120582

119879

119868119896

119889119868119896= 0

120583 ge 01003817100381710038171003817100381711988910038171003817100381710038171003817le 1 120583 (

1003817100381710038171003817100381711988910038171003817100381710038171003817

2

minus 1) = 0

(14)

Multiplying the first of these expressions by 119889 we obtain

nabla119891(119909119896)119879

119889 minus 120582119879

119889 + 21205831003817100381710038171003817100381711988910038171003817100381710038171003817

2

= 0 (15)

where 120582 = (120582119868119896

0

) By combining the assumption nabla119891(119909119896)119879

119889 =

0 with the second and the third expressions of (14) we findthat 120583 = 0 Substituting it into the first expressions of (14) weobtain that

nabla119891119868119896(119909119896) minus 120582119868119896= 0 nabla119891

119869119896(119909119896) = 0 (16)

Let 120582119894= 0 119894 isin 119869

119896 then 120582

119894ge 0 119894 isin 119868

119896cup 119869119896 Moreover we have

nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(17)

This implies that 119909119896is a KKT point of problem (10)

On the other hand we suppose that 119909119896is a KKT point of

problem (10) Then there exist 120582119894 119894 isin 119868

119896cup 119869119896 such that the

following KKT condition holds

nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(18)

From the second of these expressions we get 120582119869119896

= 0 Substi-tuting it into the first of these expressions we have nabla119891

119868119896(119909119896) =

120582119868119896

ge 0 and nabla119891119869119896(119909119896) = 0 so that nabla119891(119909

119896)119879

119889 = nabla119891119868119896(119909119896)119879

119889119868119896

=

120582119879

119868119896

119889119868119896

ge 0 However we had shown that nabla119891(119909119896)119879

119889 le 0 sonabla119891(119909119896)119879

119889 = 0By the proof of Lemma 1 we find that nabla119891

119868119896(119909119896) ge 0 and

nabla119891119869119896(119909119896) = 0 are necessary conditions of the fact that 119909

119896is a

KKT point of problem (10) We summarize these observationresults as the following result

Lemma 2 Let 119909119896ge 0 then 119909

119896is a KKT point of problem (10)

if and only if nabla119891119868119896(119909119896) ge 0 and nabla119891

119869119896(119909119896) = 0

Proof Firstly we suppose that 119909119896is a KKT point of problem

(10) Similar to the proof of Lemma 1 it is easy to get thatnabla119891119868119896(119909119896) ge 0 and nabla119891

119869119896(119909119896) = 0

Secondly we suppose that nabla119891119868119896(119909119896) ge 0 and nabla119891

119869119896(119909119896) = 0

Let 120582119868119896

= nabla119891119868119896(119909119896) ge 0 120582

119869119896= 0 then the KKT condition (18)

holds so that 119909119896is a KKT point of problem (10)

Based on the above discussion we propose a conjugategradient type method for solving problem (10) as follows Let

feasible point 119909119896be current iteration For the boundary of the

feasible region 119909119896119868119896

= 0 we take

119889119896119894=

0 119892119894(119909119896) gt 0

minus119892119894(119909119896) 119892

119894(119909119896) le 0

forall119894 isin 119868119896 (19)

where 119892119894(119909119896) = nabla119891

119894(119909119896) For the interior of the feasible region

119909119896119869119896

gt 0 similar to the direction 119889119896in the MPRP method we

define 119889119896119869119896

by the following formula

119889MPRP119896119869119896

=

minus119892119869119896(119909119896) 119896 = 0

minus119892119869119896(119909119896) + 120573

PRP119896

119889119896minus1119869119896

minus 120579MPRP119896

119910119896minus1

119896 gt 0

(20)

where 119892119869119896(119909119896) = nabla119891

119869119896(119909119896) 120573PRP119896

= 119892119869119896(119909119896)119879

119910119896minus1

119892(119909119896minus1

)2

120579MPRP119896

= 119892119869119896(119909119896)119879

119889119896minus1119869119896

119892(119909119896minus1

)2 and 119910

119896minus1= 119892119869119896(119909119896) minus

119892119869119896(119909119896minus1

)It is easy to see from (19) and (20) that

minus10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

le 119892119868119896(119909119896)119879

119889119896119868119896

le 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(21)

The above relations indicate that

119892(119909119896)119879

119889119896= 119892119868119896(119909119896)119879

119889119896119868119896

+ 119892119869119896(119909119896)119879

119889119896119869119896

le minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(22)

119892(119909119896)119879

119889119896ge minus

10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

=1003817100381710038171003817119892(119909119896)

1003817100381710038171003817

2

(23)

where 119892(119909119896) = nabla119891(119909

119896)

Theorem 3 Let 119909119896ge 0 119889

119896be defined by (19) and (20) then

119892(119909119896)119879

119889119896le 0 (24)

Moreover 119909119896is a KKT point of problem (10) if and only if

119892(119909119896)119879

119889119896= 0

Proof Clearly inequality (22) implies that

119892(119909119896)119879

119889119896le 0 (25)

If 119909119896is a KKT point of problem (10) similar to the proof

of Lemma 1 we also get that 119892(119909119896)119879

119889119896= 0

If 119892(119909119896)119879

119889119896= 0 by (22) we can get that

119892119868119896(119909119896)119879

119889119896119868119896

= 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

= 0

(26)

The equality 119892119868119896(119909119896)119879

119889119896119868119896

= 0 and the definition of 119889119896119868119896

(19)imply that

119892119868119896(119909119896) ge 0 (27)


Let 120582119868119896

= 119892119868119896(119909119896) ge 0 120582


also holds so that 119909119896is a KKT point of problem (10)

By combining (22) with Theorem 3 we conclude that 119889119896

defined by (19) and (20) provides a feasible descent directionof 119891 at 119909

119896 if 119909119896is not a KKT point of problem (10)

Based on the above process we propose an MPRP typemethod for solving (10) as follows

Algorithm 4 (MPRP type method)

Step 0 Given constants 120588 isin (0 1) 120575 gt 0 120598 gt 0 Choose theinitial point 119909

0ge 0 Let 119896 = 0

Step 1 Compute 119889119896

= (119889119896119868119896

119889119896119869119896

) by (19) and (20) If|119892(119909119896)119879

119889119896| le 120598 then stop Otherwise go to the next step

Step 2 Determine 120572119896= max120588119895 119895 = 0 1 2 satisfying 119909

119896+

120572119896119889119896ge 0 and

119891 (119909119896+ 120572119896119889119896) le 119891 (119909

119896) minus 120575120572

2

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(28)

Step 3 Let the next iteration be 119909119896+1

= 119909119896+ 120572119896119889119896

Step 4 Let 119896 = 119896 + 1 and go to Step 1

It is easy to see that the sequence 119909119896 generated by

Algorithm 4 is a feasible point sequence Moreover it followsfrom (28) that the function value sequence 119891(119909

119896) is decreas-

ing In addition if 119891(119909) is bounded from below we have from(28) that

infin

sum

119896=0

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

lt infin (29)

In particular we have

lim119896rarrinfin

120572119896

10038171003817100381710038171198891198961003817100381710038171003817 = 0 (30)

Next we prove the global convergence of Algorithm 4under the following assumptions

Assumption A (1)The level set 120596 = 119909 isin 119877119899

| 119891(119909) le 119891(1199090)

is bound(2) In some neighborhood 119873 of 120596 119891 is continuously

differentiable and its gradient is the Lipschitz continuousnamely there exists a constant 119871 gt 0 such that

1003817100381710038171003817nabla119891 (119909) minus nabla119891 (119910)1003817100381710038171003817 le 119871

1003817100381710038171003817119909 minus 1199101003817100381710038171003817 forall119909 119910 isin 119873 (31)

Clearly Assumption A implies that there exists a constant1205741such that

1003817100381710038171003817nabla119891 (119909)1003817100381710038171003817 le 1205741 forall119909 isin 119873 (32)

Lemma 5 Suppose that the conditions in Assumption A hold119909119896 and 119889

119896 are the iterative sequence and the direction

sequence generated by Algorithm 4 If there exists a constant120598 gt 0 such that

1003817100381710038171003817119892 (119909119896)1003817100381710038171003817 ge 120598 forall119896 (33)

then there exists a constant119872 gt 0 such that1003817100381710038171003817119889119896

1003817100381710038171003817 le 119872 forall119896 (34)

Proof By combining (19) (20) and (33) with Assumption Awe deduce that

10038171003817100381710038171198891198961003817100381710038171003817 le

100381710038171003817100381710038171003817119889119896119868119896

100381710038171003817100381710038171003817+100381710038171003817100381710038171003817119889MPRP119896119869119896

100381710038171003817100381710038171003817

le 1205741+10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817+ 2

10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

1003817100381710038171003817119910119896minus11003817100381710038171003817

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1003817100381710038171003817119892 (119909119896minus1

)1003817100381710038171003817

2

le 21205741+

21205741119871120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1205982

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

(35)

By (30) there exists a constant 120574 isin (0 1) and an iteger 1198960such

that the following inequality holds for all 119896 ge 1198960

21198711205741

1205982120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817le 120574 (36)

Hence we have for any 119896 ge 1198960

10038171003817100381710038171198891198961003817100381710038171003817 le 2120574

1+ 120574

1003817100381710038171003817119889119896minus11003817100381710038171003817

le 21205741(1 + 120574 + 120574

2

+ sdot sdot sdot + 120574119896minus1198960minus1)

+ 120574119896minus1198960

100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

le21205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

(37)

Let

119872 = max100381710038171003817100381711988911003817100381710038171003817

100381710038171003817100381711988921003817100381710038171003817

100381710038171003817100381710038171198891198960

1003817100381710038171003817100381721205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817 (38)

Then1003817100381710038171003817119889119896

1003817100381710038171003817 le 119872 forall119896 (39)

Theorem 6 Suppose that the conditions in Assumption Ahold Let 119909

119896 and 119889

119896 be the iterative sequence and the direc-

tion sequence generated by Algorithm 4 Then

lim inf119896rarrinfin

100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816= 0 (40)

Proof We prove the result of this theorem by contradictionAssume that the theorem is not true then there exists aconstant 120576 gt 0 such that

100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816ge 120598 forall119896 (41)

So by combining (41) with (23) it is easy to see that (33) holds

(1) If lim inf119896rarrinfin

120572119896

gt 0 we get from (30) that 119889119896

rarr

0 so that lim119896rarrinfin

|119892(119909119896)119879

119889119896| = 0 This contradicts

assumption (41)



120572119896= 0 there is an infinite index set 119870

such that

lim119896isin119870 119896rarrinfin

120572119896= 0 (42)

It follows from Step 2 of Algorithm 4 that when 119896 isin 119870 issufficiently large 120588minus1120572

119896does not satify119891(119909

119896+120572119896119889119896) le 119891(119909

119896)minus

1205751205722

119896 1198891198962 that is

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896) gt minus120575120588

minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(43)

By the mean-value theorem Lemma 1 and Assumption Athere is ℎ

119896isin (0 1) such that

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896)

= 120588minus1

120572119896119892(119909119896+ ℎ119896120588minus1

120572119896119889119896)119879

119889119896

= 120588minus1

120572119896119892(119909119896)119879

119889119896

+120588minus1

120572119896(119892 (119909119896+ ℎ119896120588minus1

120572119896119889119896) minus 119892 (119909

119896))119879

119889119896

le 120588minus1

120572119896119892(119909119896)119879

119889119896+ 119871120588minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(44)

Substituting the last inequality into (43) we get for all 119896 isin 119870

sufficiently large

0 le minus119892(119909119896)119879

119889119896le 120588minus1

(119871 + 120575) 120572119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(45)

Taking the limit on both sides of the equation then bycombining 119889

119896le 119872 and recalling lim

119896isin119870 119896rarrinfin120572119896= 0 we

obtain that lim119896isin119870 119896rarrinfin

|119892(119909119896)119879

119889119896| = 0 This also yields a

contradiction

3 Numerical Experiments

In this section we report some numerical experiments Wetest the performance of Algorithm 4 and compare it with theZoutendijk method

The code was written in Matlab and the program wasrun on a PC with 220GHz CPU and 100GB memory Theparameters in the method are specified as follows We set 120588 =

12 120575 = 110 We stop the iteration if |nabla119891(119909119896)119879

119889119896| le 00001

or the iteration number exceeds 10000We first test Algorithm 4 on small and medium size

problems and compared them with the Zoutendijk methodin the total number of iterations and the CPU time used Thetest problems are from the CUTE library [20]The numericalresults of Algorithm 4 and the Zoutendijk method are listedin Table 1 The columns have the following meanings

119875(119894) is the number of the test problemDim is thedimension of the test problem Iter is the number of iterationsandTime is CPU time in seconds

We can see fromTable 1 that Algorithm 4 has successfullysolved 12 test problems and the Zoutendijk method hassuccessfully solved 8 test problems From the number of iter-ations Algorithm 4 has 12 test results better than Zoutendijkmethod From the computation time Algorithm 4 performs

Table 1 The numerical results

119875(119894) Dim Algorithm 4 Zoutendijk methodIter Time Iter Time

3 2 1973 15710 mdash mdash4 2 201 02290 mdash mdash

6

2 30 00160 mdash mdash3 35 00160 mdash mdash4 39 00470 mdash mdash10 124 01210 mdash mdash50 220 05370 mdash mdash

8 3 44 00150 40 0218811 3 3 00000 4 0109415 4 10 00160 20 0156318 6 322 00690 1936 12093819 11 438 05440 8338 72421923 50 12 00300 4 0500024 100 142 03750 mdash mdash25 100 38 00810 6 03438

26 100 8 00470 6 012501000 4 479060 4 1901406

Table 2 Test results for VARDIM with various dimensions

Problem Dim Algorithm 4 Zoutendijk methodIter Time Iter Time

VARDIM


Table 3 Test results for Problem 1 with various dimensions


Problem 1

1000 17 01400 8 11025782000 26 168604 8 26326603000 39 396561 11 55403104000 51 681729 30 91010905000 55 1105660 mdash mdash

much better than the Zoutendijk method did We then testAlgorithm 4 and the Zoutendijk method on two problemswith a larger dimension The problem of VARDIM comesfrom [20] and the following problem comes from [16] Theresults are listed in Tables 2 and 3

Problem 1 The nonnegative constraints optimization prob-lem

min 119891 (119909)

st 119909 ge 0

(46)


with Engval function 119891 119877119899

rarr 119877 is defined by

119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2

minus 4119909119894minus1

+ 3 (47)

We can see fromTable 2 that Algorithm 4 has successfullysolved the problem of VARDIMwhose scale varies from 1000dimensions to 5000 dimensions However the Zoutendijkmethod fails to solve the problem of VARDIM with largerdimension From Table 3 although the number of iterationsof Algorithm 4 is more than the Zoutendijk method thecomputation time of Algorithm 4 is less than the Zoutendijkmethod and this feature becomesmore evident as increase ofthe dimension of the test problem

In summary the results from Tables 1ndash3 show thatAlgorithm 4 is more efficient than the Zoutendijk methodand provides an efficient method for solving nonnegativeconstraints optimization problems

Acknowledgment

This research is supported by the NSF (11161020) of China

References

[1] M R Hestenes and E Stiefel ldquoMethods of conjugate gradientsfor solving linear systemsrdquo Journal of Research of the NationalBureau of Standards vol 49 pp 409ndash436 1952

[2] R Fletcher and C M Reeves ldquoFunction minimization by con-jugate gradientsrdquo The Computer Journal vol 7 pp 149ndash1541964

[3] B Polak and G Ribire ldquoNote sur la convergence de directionsconjugeesrdquo Revue Francaise drsquoInformatique et de RechercheOperationnelle vol 16 pp 35ndash43 1969

[4] B T Polyak ldquoThe conjugate gradient method in extremal prob-lemsrdquo USSR Computational Mathematics and MathematicalPhysics vol 9 no 4 pp 94ndash112 1969

[5] R Fletcher Practical Methods of Optimization John Wiley ampSons Ltd Chichester UK 2nd edition 1987

[6] Y Liu and C Storey ldquoEfficient generalized conjugate gradientalgorithms I Theoryrdquo Journal of Optimization Theory and Ap-plications vol 69 no 1 pp 129ndash137 1991

[7] Y H Dai and Y Yuan ldquoA nonlinear conjugate gradient methodwith a strong global convergence propertyrdquo SIAM Journal onOptimization vol 10 no 1 pp 177ndash182 1999

[8] M J D Powell ldquoConvergence properties of algorithms for non-linear optimizationrdquo SIAM Review vol 28 no 4 pp 487ndash5001986

[9] J C Gilbert and J Nocedal ldquoGlobal convergence properties ofconjugate gradient methods for optimizationrdquo SIAM Journal onOptimization vol 2 no 1 pp 21ndash42 1992

[10] R Pytlak ldquoOn the convergence of conjugate gradient algo-rithmsrdquo IMA Journal of Numerical Analysis vol 14 no 3 pp443ndash460 1994

[11] G Li C Tang and ZWei ldquoNew conjugacy condition and relat-ed new conjugate gradient methods for unconstrained opti-mizationrdquo Journal of Computational and Applied Mathematicsvol 202 no 2 pp 523ndash539 2007

[12] X Li and X Zhao ldquoA hybrid conjugate gradient method foroptimization problemsrdquoNatural Science vol 3 no 1 pp 85ndash902011

[13] Y H Dai and Y Yuan ldquoAn efficient hybrid conjugate gradientmethod for unconstrained optimizationrdquo Annals of OperationsResearch vol 103 pp 33ndash47 2001

[14] W W Hager and H Zhang ldquoA new conjugate gradient methodwith guaranteed descent and an efficient line searchrdquo SIAMJournal on Optimization vol 16 no 1 pp 170ndash192 2005

[15] D-H Li Y-Y Nie J-P Zeng andQ-N Li ldquoConjugate gradientmethod for the linear complementarity problemwith 119878-matrixrdquoMathematical and ComputerModelling vol 48 no 5-6 pp 918ndash928 2008

[16] D-H Li and X-L Wang ldquoA modified Fletcher-Reeves-typederivative-free method for symmetric nonlinear equationsrdquoNumerical Algebra Control and Optimization vol 1 no 1 pp71ndash82 2011

[17] L Zhang W Zhou and D Li ldquoGlobal convergence of a mod-ified Fletcher-Reeves conjugate gradient method with Armijo-type line searchrdquo Numerische Mathematik vol 104 no 4 pp561ndash572 2006

[18] L Zhang W Zhou and D-H Li ldquoA descent modified Polak-Ribiere-Polyak conjugate gradient method and its global con-vergencerdquo IMA Journal of Numerical Analysis vol 26 no 4 pp629ndash640 2006

[19] D H Li and X J Tong Numerical Optimization Science PressBeijing China 2005

[20] J J More B S Garbow and K E Hillstrom ldquoTesting uncon-strained optimization softwarerdquo ACM Transactions on Mathe-matical Software vol 7 no 1 pp 17ndash41 1981

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of


Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of


Several famous formulae for 120573119896are the Hestenes-Stiefel (HS)

[1] Fletcher-Reeves (FR) [2] Polak-Ribiere-Polyak (PRP) [34] Conjugate-Descent (CD) [5] Liu-Storey (LS) [6] andDai-Yuan (DY) [7] formulae which are given by

120573HS119896

=nabla119891(119909119896)⊤

119910119896minus1

119889⊤

119896minus1119910119896minus1

120573FR119896

=

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

1003817100381710038171003817nabla119891 (119909119896minus1

)1003817100381710038171003817

2 (5)

120573PRP119896

=nabla119891(119909119896)⊤

119910119896minus1

1003817100381710038171003817nabla119891 (119909119896minus1

)1003817100381710038171003817

2 120573

CD119896

= minus

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

119889⊤


) (6)

120573LS119896

= minusnabla119891(119909119896)⊤

119910119896minus1

119889⊤


) 120573

DY119896

=

1003817100381710038171003817nabla119891 (119909119896)1003817100381710038171003817

2

119889⊤

119896minus1119910119896minus1

(7)

where 119910119896minus1

= nabla119891(119909119896)minusnabla119891(119909

119896minus1) and sdot stands for the Euclid-

ean norm of vectors In this paper we focus our attentionon the Polak-Ribiere-Polyak (PRP) method The study ofthe PRP method has received much attention and has madegood progress The global convergence of the PRP methodwith exact line search has been proved in [3] under strongconvexity assumption on 119891 However for general nonlinearfunction an example given by Powell [8] shows that the PRPmethod may fail to be globally convergent even if the exactline search is used Inspired by Powellrsquos work Gilbert andNocedal [9] conducted an elegant analysis and showed thatthe PRPmethod is globally convergent if 120573PRP

119896is restricted to

be nonnegative and 120572119896is determined by a line search satisfy-

ing the sufficient descent condition 119892⊤

119896119889119896

le minus119888 1198921198962 in

addition to the standard Wolfe conditions Other conjugategradient methods and their global convergence can be foundin [10ndash15] and so forth

Recently Li andWang [16] extended themodified Fletch-er-Reeves (MFR) method proposed by Zhang et al [17] forsolving unconstrained optimization to the nonlinear equa-tions

119865 (119909) = 0 (8)

where 119865 119877119899

rarr 119877119899 is continuously differentiable and pro-

posed a descent derivative-freemethod for solving symmetricnonlinear equations The direction generated by the methodis descent for the residual function Under appropriateconditions the method is globally convergent by the use ofsome backtracking line search technique

In this paper we further study the conjugate gradientmethod We focus our attention on the modified Polak-Ribiere-Polyak (MPRP) method proposed by Zhang et al[18] The direction generated by MPRP method is given by

119889119896=

minus119892 (119909119896) 119896 = 0

minus119892 (119909119896) + 120573

PRP119896

119889119896minus1

minus 120579119896119910119896minus1

119896 gt 0

(9)

where 119892(119909119896) = nabla119891(119909

119896) 120573PRP119896

= 119892(119909119896)119879

119910119896minus1

119892(119909119896minus1

)2 120579119896=

119892(119909119896)119879

119889119896minus1

119892(119909119896minus1

)2 and 119910

119896minus1= 119892(119909

119896) minus 119892(119909

119896minus1) The

MPRP method not only reserves good properties of thePRP method but also possesses another nice property thatit is always generates descent directions for the objective

function This property is independent of the line searchused Under suitable conditions the MPRP method withthe Armoji-type line search is also globally convergent Thepurpose of this paper is to develop an MPRP type methodfor the nonnegative constraints optimization problems Com-bining the Zoutendijk feasible direction method with MPRPmethod we propose a conjugate gradient type method forsolving the nonnegative constraints optimization problemsIf the initial point is feasible the method generates a feasiblepoint sequence We also do numerical experiments to testthe proposed method and compare the performance of themethod with the Zoutendijk feasible direction method Thenumerical results show that the method that we proposeoutperforms the Zoutendijk feasible direction method

2 Algorithm

Consider the following nonnegative constraints optimizationproblems

min 119891 (119909)

st 119909 ge 0

(10)

where 119891 119877119899

rarr 119877 is continuously differentiable Let 119909119896ge 0

be the current iteration Define the index set119868119896= 119868 (119909

119896) = 119894 | 119909

119896(119894) = 0 119869

119896= 1 2 119899 119868

119896

(11)

where 119909119896(119894) is the 119894th component of 119909

119896 In fact the index set

119868119896is the active set of problem (10) at 119909

119896

The purpose of this paper is to develop a conjugategradient type method for problem (10) Since the iterativesequence is a feasible point sequence the search directionsshould be feasible descent directions Let 119909

119896ge 0 be the

current iteration By the definition of feasible direction wehave that [19] 119889 isin 119877

119899 is a feasible direction of (10) at 119909119896if and

only if 119889119868119896

ge 0 Similar to the Zoutendijk feasible directionmethod we consider the following problem

min nabla119891(119909119896)119879

119889

st 119889119868119896ge 0 119889 le 1

(12)

Next we show that if 119909119896is not a KKT point of (10) the solu-

tion of problem (12) is a feasible descent direction of 119891 at 119909119896

Lemma 1 Let 119909119896ge 0 and let 119889 be a solution of problem (12)

then nabla119891(119909119896)119879

119889 le 0 Moreover nabla119891(119909119896)119879

119889 = 0 if and only if 119909119896

is a KKT point of problem (10)

Proof Since 119889 = 0 is a feasible point of problem (12) theremust be nabla119891(119909

119896)119879

119889 le 0 Consequently if nabla119891(119909119896)119879

119889 = 0 theremust be nabla119891(119909

119896)119879

119889 lt 0 This implies that the direction 119889 is afeasible descent direction of 119891 at 119909

119896

We suppose thatnabla119891(119909119896)119879

119889 = 0 Problem (12) is equivalentto the following problem

min nabla119891(119909119896)119879

119889

st 119889119868119896ge 0 119889

2

le 1

(13)



dition holds

nabla119891 (119909119896) minus (

120582119868119896

0) + 2120583119889 = 0

120582119868119896ge 0 119889

119868119896ge 0 120582

119879

119868119896

119889119868119896= 0

120583 ge 01003817100381710038171003817100381711988910038171003817100381710038171003817le 1 120583 (

1003817100381710038171003817100381711988910038171003817100381710038171003817

2

minus 1) = 0

(14)


nabla119891(119909119896)119879

119889 minus 120582119879

119889 + 21205831003817100381710038171003817100381711988910038171003817100381710038171003817

2

= 0 (15)

where 120582 = (120582119868119896

0


119889 =



119869119896(119909119896) = 0 (16)

Let 120582119894= 0 119894 isin 119869

119896 then 120582

119894ge 0 119894 isin 119868


nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(17)




119896cup 119869119896 such that the


nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(18)



119868119896(119909119896) =

120582119868119896


119896)119879

119889 = nabla119891119868119896(119909119896)119879

119889119868119896

=

120582119879

119868119896

119889119868119896


119889 le 0 sonabla119891(119909119896)119879


119868119896(119909119896) ge 0 and


119896is a


Lemma 2 Let 119909119896ge 0 then 119909



119869119896(119909119896) = 0



119869119896(119909119896) = 0


119869119896(119909119896) = 0

Let 120582119868119896

= nabla119891119868119896(119909119896) ge 0 120582





feasible region 119909119896119868119896

= 0 we take

119889119896119894=

0 119892119894(119909119896) gt 0

minus119892119894(119909119896) 119892

119894(119909119896) le 0

forall119894 isin 119868119896 (19)

where 119892119894(119909119896) = nabla119891


119909119896119869119896


define 119889119896119869119896


119889MPRP119896119869119896

=

minus119892119869119896(119909119896) 119896 = 0

minus119892119869119896(119909119896) + 120573

PRP119896

119889119896minus1119869119896


119910119896minus1

119896 gt 0

(20)

where 119892119869119896(119909119896) = nabla119891

119869119896(119909119896) 120573PRP119896

= 119892119869119896(119909119896)119879

119910119896minus1

119892(119909119896minus1

)2

120579MPRP119896

= 119892119869119896(119909119896)119879

119889119896minus1119869119896

119892(119909119896minus1

)2 and 119910

119896minus1= 119892119869119896(119909119896) minus

119892119869119896(119909119896minus1


minus10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

le 119892119868119896(119909119896)119879

119889119896119868119896

le 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(21)


119892(119909119896)119879

119889119896= 119892119868119896(119909119896)119879

119889119896119868119896

+ 119892119869119896(119909119896)119879

119889119896119869119896

le minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(22)

119892(119909119896)119879

119889119896ge minus

10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

=1003817100381710038171003817119892(119909119896)

1003817100381710038171003817

2

(23)

where 119892(119909119896) = nabla119891(119909

119896)

Theorem 3 Let 119909119896ge 0 119889


119892(119909119896)119879

119889119896le 0 (24)


119892(119909119896)119879

119889119896= 0


119892(119909119896)119879

119889119896le 0 (25)



119889119896= 0

If 119892(119909119896)119879

119889119896= 0 by (22) we can get that

119892119868119896(119909119896)119879

119889119896119868119896

= 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

= 0

(26)

The equality 119892119868119896(119909119896)119879

119889119896119868119896


(19)imply that

119892119868119896(119909119896) ge 0 (27)


Let 120582119868119896

= 119892119868119896(119909119896) ge 0 120582









0ge 0 Let 119896 = 0


= (119889119896119868119896

119889119896119869119896

) by (19) and (20) If|119892(119909119896)119879



119896+

120572119896119889119896ge 0 and

119891 (119909119896+ 120572119896119889119896) le 119891 (119909

119896) minus 120575120572

2

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(28)


= 119909119896+ 120572119896119889119896




119896) is decreas-


infin

sum

119896=0

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

lt infin (29)


lim119896rarrinfin

120572119896

10038171003817100381710038171198891198961003817100381710038171003817 = 0 (30)



| 119891(119909) le 119891(1199090)










1003817100381710038171003817119892 (119909119896)1003817100381710038171003817 ge 120598 forall119896 (33)


1003817100381710038171003817 le 119872 forall119896 (34)


10038171003817100381710038171198891198961003817100381710038171003817 le

100381710038171003817100381710038171003817119889119896119868119896

100381710038171003817100381710038171003817+100381710038171003817100381710038171003817119889MPRP119896119869119896

100381710038171003817100381710038171003817

le 1205741+10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817+ 2

10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

1003817100381710038171003817119910119896minus11003817100381710038171003817

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1003817100381710038171003817119892 (119909119896minus1

)1003817100381710038171003817

2

le 21205741+

21205741119871120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1205982

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

(35)



21198711205741

1205982120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817le 120574 (36)


10038171003817100381710038171198891198961003817100381710038171003817 le 2120574

1+ 120574

1003817100381710038171003817119889119896minus11003817100381710038171003817

le 21205741(1 + 120574 + 120574

2


+ 120574119896minus1198960

100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

le21205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

(37)

Let

119872 = max100381710038171003817100381711988911003817100381710038171003817

100381710038171003817100381711988921003817100381710038171003817

100381710038171003817100381710038171198891198960

1003817100381710038171003817100381721205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817 (38)

Then1003817100381710038171003817119889119896

1003817100381710038171003817 le 119872 forall119896 (39)


119896 and 119889




100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816= 0 (40)


100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816ge 120598 forall119896 (41)



120572119896


rarr


|119892(119909119896)119879


assumption (41)




such that


120572119896= 0 (42)


119896does not satify119891(119909

119896+120572119896119889119896) le 119891(119909

119896)minus

1205751205722

119896 1198891198962 that is

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896) gt minus120575120588

minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(43)



119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896)

= 120588minus1

120572119896119892(119909119896+ ℎ119896120588minus1

120572119896119889119896)119879

119889119896

= 120588minus1

120572119896119892(119909119896)119879

119889119896

+120588minus1

120572119896(119892 (119909119896+ ℎ119896120588minus1

120572119896119889119896) minus 119892 (119909

119896))119879

119889119896

le 120588minus1

120572119896119892(119909119896)119879

119889119896+ 119871120588minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(44)


sufficiently large

0 le minus119892(119909119896)119879

119889119896le 120588minus1

(119871 + 120575) 120572119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(45)





|119892(119909119896)119879


contradiction





119889119896| le 00001








6


8 3 44 00150 40 0218811 3 3 00000 4 0109415 4 10 00160 20 0156318 6 322 00690 1936 12093819 11 438 05440 8338 72421923 50 12 00300 4 0500024 100 142 03750 mdash mdash25 100 38 00810 6 03438

26 100 8 00470 6 012501000 4 479060 4 1901406



VARDIM




Problem 1

1000 17 01400 8 11025782000 26 168604 8 26326603000 39 396561 11 55403104000 51 681729 30 91010905000 55 1105660 mdash mdash



min 119891 (119909)

st 119909 ge 0

(46)




119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2


+ 3 (47)



Acknowledgment


References




























Volume 2014




Journal of











Journal of


Function Spaces






Algebra











dition holds

nabla119891 (119909119896) minus (

120582119868119896

0) + 2120583119889 = 0

120582119868119896ge 0 119889

119868119896ge 0 120582

119879

119868119896

119889119868119896= 0

120583 ge 01003817100381710038171003817100381711988910038171003817100381710038171003817le 1 120583 (

1003817100381710038171003817100381711988910038171003817100381710038171003817

2

minus 1) = 0

(14)


nabla119891(119909119896)119879

119889 minus 120582119879

119889 + 21205831003817100381710038171003817100381711988910038171003817100381710038171003817

2

= 0 (15)

where 120582 = (120582119868119896

0


119889 =



119869119896(119909119896) = 0 (16)

Let 120582119894= 0 119894 isin 119869

119896 then 120582

119894ge 0 119894 isin 119868


nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(17)




119896cup 119869119896 such that the


nabla119891 (119909119896) minus (

120582119868119896

120582119869119896

) = 0

120582119894ge 0 119909

119896(119894) ge 0 120582

119894119909119896(119894) = 0 119894 isin 119868

119896cup 119869119896

(18)



119868119896(119909119896) =

120582119868119896


119896)119879

119889 = nabla119891119868119896(119909119896)119879

119889119868119896

=

120582119879

119868119896

119889119868119896


119889 le 0 sonabla119891(119909119896)119879


119868119896(119909119896) ge 0 and


119896is a


Lemma 2 Let 119909119896ge 0 then 119909



119869119896(119909119896) = 0



119869119896(119909119896) = 0


119869119896(119909119896) = 0

Let 120582119868119896

= nabla119891119868119896(119909119896) ge 0 120582





feasible region 119909119896119868119896

= 0 we take

119889119896119894=

0 119892119894(119909119896) gt 0

minus119892119894(119909119896) 119892

119894(119909119896) le 0

forall119894 isin 119868119896 (19)

where 119892119894(119909119896) = nabla119891


119909119896119869119896


define 119889119896119869119896


119889MPRP119896119869119896

=

minus119892119869119896(119909119896) 119896 = 0

minus119892119869119896(119909119896) + 120573

PRP119896

119889119896minus1119869119896


119910119896minus1

119896 gt 0

(20)

where 119892119869119896(119909119896) = nabla119891

119869119896(119909119896) 120573PRP119896

= 119892119869119896(119909119896)119879

119910119896minus1

119892(119909119896minus1

)2

120579MPRP119896

= 119892119869119896(119909119896)119879

119889119896minus1119869119896

119892(119909119896minus1

)2 and 119910

119896minus1= 119892119869119896(119909119896) minus

119892119869119896(119909119896minus1


minus10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

le 119892119868119896(119909119896)119879

119889119896119868119896

le 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(21)


119892(119909119896)119879

119889119896= 119892119868119896(119909119896)119879

119889119896119868119896

+ 119892119869119896(119909119896)119879

119889119896119869119896

le minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

(22)

119892(119909119896)119879

119889119896ge minus

10038171003817100381710038171003817119892119868119896(119909119896)10038171003817100381710038171003817

2

minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

=1003817100381710038171003817119892(119909119896)

1003817100381710038171003817

2

(23)

where 119892(119909119896) = nabla119891(119909

119896)

Theorem 3 Let 119909119896ge 0 119889


119892(119909119896)119879

119889119896le 0 (24)


119892(119909119896)119879

119889119896= 0


119892(119909119896)119879

119889119896le 0 (25)



119889119896= 0

If 119892(119909119896)119879

119889119896= 0 by (22) we can get that

119892119868119896(119909119896)119879

119889119896119868119896

= 0

119892119869119896(119909119896)119879

119889119896119869119896

= minus10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

2

= 0

(26)

The equality 119892119868119896(119909119896)119879

119889119896119868119896


(19)imply that

119892119868119896(119909119896) ge 0 (27)


Let 120582119868119896

= 119892119868119896(119909119896) ge 0 120582









0ge 0 Let 119896 = 0


= (119889119896119868119896

119889119896119869119896

) by (19) and (20) If|119892(119909119896)119879



119896+

120572119896119889119896ge 0 and

119891 (119909119896+ 120572119896119889119896) le 119891 (119909

119896) minus 120575120572

2

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(28)


= 119909119896+ 120572119896119889119896




119896) is decreas-


infin

sum

119896=0

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

lt infin (29)


lim119896rarrinfin

120572119896

10038171003817100381710038171198891198961003817100381710038171003817 = 0 (30)



| 119891(119909) le 119891(1199090)










1003817100381710038171003817119892 (119909119896)1003817100381710038171003817 ge 120598 forall119896 (33)


1003817100381710038171003817 le 119872 forall119896 (34)


10038171003817100381710038171198891198961003817100381710038171003817 le

100381710038171003817100381710038171003817119889119896119868119896

100381710038171003817100381710038171003817+100381710038171003817100381710038171003817119889MPRP119896119869119896

100381710038171003817100381710038171003817

le 1205741+10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817+ 2

10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

1003817100381710038171003817119910119896minus11003817100381710038171003817

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1003817100381710038171003817119892 (119909119896minus1

)1003817100381710038171003817

2

le 21205741+

21205741119871120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1205982

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

(35)



21198711205741

1205982120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817le 120574 (36)


10038171003817100381710038171198891198961003817100381710038171003817 le 2120574

1+ 120574

1003817100381710038171003817119889119896minus11003817100381710038171003817

le 21205741(1 + 120574 + 120574

2


+ 120574119896minus1198960

100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

le21205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

(37)

Let

119872 = max100381710038171003817100381711988911003817100381710038171003817

100381710038171003817100381711988921003817100381710038171003817

100381710038171003817100381710038171198891198960

1003817100381710038171003817100381721205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817 (38)

Then1003817100381710038171003817119889119896

1003817100381710038171003817 le 119872 forall119896 (39)


119896 and 119889




100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816= 0 (40)


100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816ge 120598 forall119896 (41)



120572119896


rarr


|119892(119909119896)119879


assumption (41)




such that


120572119896= 0 (42)


119896does not satify119891(119909

119896+120572119896119889119896) le 119891(119909

119896)minus

1205751205722

119896 1198891198962 that is

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896) gt minus120575120588

minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(43)



119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896)

= 120588minus1

120572119896119892(119909119896+ ℎ119896120588minus1

120572119896119889119896)119879

119889119896

= 120588minus1

120572119896119892(119909119896)119879

119889119896

+120588minus1

120572119896(119892 (119909119896+ ℎ119896120588minus1

120572119896119889119896) minus 119892 (119909

119896))119879

119889119896

le 120588minus1

120572119896119892(119909119896)119879

119889119896+ 119871120588minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(44)


sufficiently large

0 le minus119892(119909119896)119879

119889119896le 120588minus1

(119871 + 120575) 120572119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(45)





|119892(119909119896)119879


contradiction





119889119896| le 00001








6


8 3 44 00150 40 0218811 3 3 00000 4 0109415 4 10 00160 20 0156318 6 322 00690 1936 12093819 11 438 05440 8338 72421923 50 12 00300 4 0500024 100 142 03750 mdash mdash25 100 38 00810 6 03438

26 100 8 00470 6 012501000 4 479060 4 1901406



VARDIM




Problem 1

1000 17 01400 8 11025782000 26 168604 8 26326603000 39 396561 11 55403104000 51 681729 30 91010905000 55 1105660 mdash mdash



min 119891 (119909)

st 119909 ge 0

(46)




119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2


+ 3 (47)



Acknowledgment


References




























Volume 2014




Journal of











Journal of


Function Spaces






Algebra










Let 120582119868119896

= 119892119868119896(119909119896) ge 0 120582









0ge 0 Let 119896 = 0


= (119889119896119868119896

119889119896119869119896

) by (19) and (20) If|119892(119909119896)119879



119896+

120572119896119889119896ge 0 and

119891 (119909119896+ 120572119896119889119896) le 119891 (119909

119896) minus 120575120572

2

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(28)


= 119909119896+ 120572119896119889119896




119896) is decreas-


infin

sum

119896=0

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

lt infin (29)


lim119896rarrinfin

120572119896

10038171003817100381710038171198891198961003817100381710038171003817 = 0 (30)



| 119891(119909) le 119891(1199090)










1003817100381710038171003817119892 (119909119896)1003817100381710038171003817 ge 120598 forall119896 (33)


1003817100381710038171003817 le 119872 forall119896 (34)


10038171003817100381710038171198891198961003817100381710038171003817 le

100381710038171003817100381710038171003817119889119896119868119896

100381710038171003817100381710038171003817+100381710038171003817100381710038171003817119889MPRP119896119869119896

100381710038171003817100381710038171003817

le 1205741+10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817+ 2

10038171003817100381710038171003817119892119869119896(119909119896)10038171003817100381710038171003817

1003817100381710038171003817119910119896minus11003817100381710038171003817

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1003817100381710038171003817119892 (119909119896minus1

)1003817100381710038171003817

2

le 21205741+

21205741119871120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

1205982

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817

(35)



21198711205741

1205982120572119896minus1

100381710038171003817100381710038171003817119889MPRP119896minus1119869119896

100381710038171003817100381710038171003817le 120574 (36)


10038171003817100381710038171198891198961003817100381710038171003817 le 2120574

1+ 120574

1003817100381710038171003817119889119896minus11003817100381710038171003817

le 21205741(1 + 120574 + 120574

2


+ 120574119896minus1198960

100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

le21205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817

(37)

Let

119872 = max100381710038171003817100381711988911003817100381710038171003817

100381710038171003817100381711988921003817100381710038171003817

100381710038171003817100381710038171198891198960

1003817100381710038171003817100381721205741

1 minus 120574+100381710038171003817100381710038171198891198960

10038171003817100381710038171003817 (38)

Then1003817100381710038171003817119889119896

1003817100381710038171003817 le 119872 forall119896 (39)


119896 and 119889




100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816= 0 (40)


100381610038161003816100381610038161003816119892(119909119896)119879

119889119896

100381610038161003816100381610038161003816ge 120598 forall119896 (41)



120572119896


rarr


|119892(119909119896)119879


assumption (41)




such that


120572119896= 0 (42)


119896does not satify119891(119909

119896+120572119896119889119896) le 119891(119909

119896)minus

1205751205722

119896 1198891198962 that is

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896) gt minus120575120588

minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(43)



119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896)

= 120588minus1

120572119896119892(119909119896+ ℎ119896120588minus1

120572119896119889119896)119879

119889119896

= 120588minus1

120572119896119892(119909119896)119879

119889119896

+120588minus1

120572119896(119892 (119909119896+ ℎ119896120588minus1

120572119896119889119896) minus 119892 (119909

119896))119879

119889119896

le 120588minus1

120572119896119892(119909119896)119879

119889119896+ 119871120588minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(44)


sufficiently large

0 le minus119892(119909119896)119879

119889119896le 120588minus1

(119871 + 120575) 120572119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(45)





|119892(119909119896)119879


contradiction





119889119896| le 00001








6


8 3 44 00150 40 0218811 3 3 00000 4 0109415 4 10 00160 20 0156318 6 322 00690 1936 12093819 11 438 05440 8338 72421923 50 12 00300 4 0500024 100 142 03750 mdash mdash25 100 38 00810 6 03438

26 100 8 00470 6 012501000 4 479060 4 1901406



VARDIM




Problem 1

1000 17 01400 8 11025782000 26 168604 8 26326603000 39 396561 11 55403104000 51 681729 30 91010905000 55 1105660 mdash mdash



min 119891 (119909)

st 119909 ge 0

(46)




119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2


+ 3 (47)



Acknowledgment


References




























Volume 2014




Journal of











Journal of


Function Spaces






Algebra












such that


120572119896= 0 (42)


119896does not satify119891(119909

119896+120572119896119889119896) le 119891(119909

119896)minus

1205751205722

119896 1198891198962 that is

119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896) gt minus120575120588

minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(43)



119891 (119909119896+ 120588minus1

120572119896119889119896) minus 119891 (119909

119896)

= 120588minus1

120572119896119892(119909119896+ ℎ119896120588minus1

120572119896119889119896)119879

119889119896

= 120588minus1

120572119896119892(119909119896)119879

119889119896

+120588minus1

120572119896(119892 (119909119896+ ℎ119896120588minus1

120572119896119889119896) minus 119892 (119909

119896))119879

119889119896

le 120588minus1

120572119896119892(119909119896)119879

119889119896+ 119871120588minus2

1205722

119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(44)


sufficiently large

0 le minus119892(119909119896)119879

119889119896le 120588minus1

(119871 + 120575) 120572119896

10038171003817100381710038171198891198961003817100381710038171003817

2

(45)





|119892(119909119896)119879


contradiction





119889119896| le 00001








6


8 3 44 00150 40 0218811 3 3 00000 4 0109415 4 10 00160 20 0156318 6 322 00690 1936 12093819 11 438 05440 8338 72421923 50 12 00300 4 0500024 100 142 03750 mdash mdash25 100 38 00810 6 03438

26 100 8 00470 6 012501000 4 479060 4 1901406



VARDIM




Problem 1

1000 17 01400 8 11025782000 26 168604 8 26326603000 39 396561 11 55403104000 51 681729 30 91010905000 55 1105660 mdash mdash



min 119891 (119909)

st 119909 ge 0

(46)




119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2


+ 3 (47)



Acknowledgment


References




























Volume 2014




Journal of











Journal of


Function Spaces






Algebra












119891 (119909) =

119899

sum

119894=2

(1199092

119894minus1+ 1199092

119894)2


+ 3 (47)



Acknowledgment


References




























Volume 2014




Journal of











Journal of


Function Spaces






Algebra
















Volume 2014




Journal of











Journal of


Function Spaces






Algebra







