A NEW HYBRID WYL-AMRI CONJUGATE GRADIENT METHOD WITH SUFFICIENT

DESCENT CONDITION FOR UNCONSTRAINED OPTIMIZATION

Ibrahim S. Mohammed*Mustafa Mamat

Abdelrhaman AbasharKamil Uba Kamfa

Contents :-* Introduction * Objectives of the research* Conjugate Gradient Version* New method and Algorithm* Numerical Results* Conclusion* References

* Introduction

Conjugate gradient method (CG) are designed to solve large scale unconstrained

optimization problem. In general, the method has the form

𝒎𝒊𝒏𝒙 ∈𝑹𝒏 𝒇(𝒙) (1.1)

where 𝒇 ∶ 𝑹𝒏 → 𝑹 is continuously differentiable . Conjugate gradient methods are

iterative methods of the form

𝒙𝒌+𝟏 = 𝒙𝒌 + 𝜶𝒌𝒅𝒌 (1.2)

where 𝒙𝒌+𝟏 is the current iterate point , 𝜶𝒌 > 𝟎 is step length which is computed

by carrying out a line search , 𝒅𝒌 is the search direction of the conjugate gradient

method define by

𝒅𝒌 −𝒈𝒌 , 𝒊𝒇 𝒌 = 𝟎

−𝒈𝒌 + 𝜷𝒌𝒅𝒌, 𝒊𝒇 𝒌 ≥ 𝟏(1.3)

Some classical formula’s for 𝜷𝒌 are given as follows

𝜷𝒌𝑯𝑺 =

𝒈𝒌𝑻 (𝒈𝒌−𝒈𝒌−𝟏)

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏(1.4)

Cont : Introduction

𝜷𝒌𝑭𝑹 =

𝒈𝒌𝑻 𝒈𝒌

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.5)

𝜷𝒌𝑷𝑹𝑷 =

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.6)

𝜷𝒌𝑪𝑫 = −

𝒈𝒌𝑻 𝒈𝒌

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.7)

𝜷𝒌𝑫𝒀 =

𝒈𝒌𝑻 𝒈𝒌

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏

(1.8)

𝜷𝒌𝑳𝑺 = −

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.9)

Cont : Introduction

• Zoutendijk proved that FR Method with exact line search is

globally convergent , Al – Baali extended this results to the

strong Wolfe - Powell line search.

• Recently, Wei et al. [7], propose a new CG formula

• Abashar et al ,modified RMIL method to suggest

• Research Objective

* To proposed a new formula for solving unconstrained optimization.

* To analyze the performance these new formulas based on standard optimizations test

problem functions.

* To proof the sufficient descent conditions of our new method

• Conjugate Gradient Version

(i) Hybrid CG methods (ii) Scaled CG methods. (iii) Three terms CG methods.

.

An important class of conjugate gradient algorithm is the hybrid CG method, for

example Hu and Storey [18] propose

Dai and Yuan [19] suggested two hybrid methods,

New method and Algorithm

we propose a new hybrid CG method is a combination between CG methods

(WYL, RMIL)

Algorithm

Step 1: Given6

0 10, nRx , set 00 gd if 0g then stop.

Step 2: Compute k by applying exact line search.

Step 3: kkkk dxx 1 if 1kg then stop.

Step 4: ComputehRW

k and generated kd by (1.3).

Step 5: Set 1 kk go to Step 2.

Numerical Results• Test problem functions considered by Andrei.

• Stopping criteria as Hillstrom , 𝑔𝑘 ≤ 𝜀.

• Subroutine programming using Matlab.

• Using exact line search.

• Performance profile introduced by Dolan and More.

Cont: Numerical Results

TABLE 1: A LIST OF PROBLEM FUNCTIONS

No Function Dim Initial Points

1 Six hump 2 (1, 1), (2, 2), (5,5), (10, -10)

2 Three hump 2 (24, 24), (29, 29), (33, 33), (50, 50)

3 Booth 2 (10, -10), (20, 20), (50, 50), (100, 100)

4 Treccani 2 (5, 5), (10,10), (-20, 20), (-50, -50)

5 Matyas 2 (1, 1), (5, 5), (10, 10), (50, 50)

6 Extended Maratos 2, 4 (0,0,0,0), (0.5,5, 0.5, 5), (10, 0.5, 10, 0.5), (70, 70, 70, 70)

7 Ext FREUD & ROTH 2, 4 (13, 13, 13, 13), (21, 21, 21, 21), (25, 25, 25,25), (23, 23, 23, 23)

8 Generalized Trig 2, 4, 10 (0.5, 5, …, 5),(5, 10, …, 10),(7,7, …, 7),(50, 50, …, 50)

9 Fletcher 2, 4, 10 (23, 23, …, 23), (45, 45, …, 45), (50, 5,…, 5), (70, 70, …,70)

10 Extended Penalty 2, 4, 10, 100 (0.5,5, …,5),(10,-0.5…,-0.5), (105,105, …,105), (130,130, …,130)

11 Raydan 1 2, 4, 10, 100 (1, 1, …,1), (3, 3, …, 3), (5, 5, …, 5), (-10, -10, …, -10)

12 Hager 2, 4, 10, 100 (3, -3, …, -3),(21, 21, …, 21), (-23, 23, …, 23), (23, 23, …, 23)

13 Rosenbrock 2, 4, 10, 100, 500, 1000, 10000 (7, 7, …, 7), (13, 13, …, 13), (23, 23, …, 23), (35, 35, …, 35)

14 Shallow 2, 4, 10, 100, 500, 1000, 10000 (21, -21, …, -21), (21, 21, …, 21), (50,50, …, 50),(130, 130, …, 130)

15 Tridiagonal 1 2, 4, 10, 100, 500, 1000, 10000 (0, 0,…, 0), (1, -1, …, -1), (17, -17, …, -17), (30, 30, …, 30)

16 Ext White & Holst 2, 4, 10, 100, 500, 1000, 10000 (-5, -5, …, -5), (2, -2, …, -2), (3, -3, …, -3), (7, -7, …, -7)

17 Ext Denschnb 2, 4, 10, 100, 500, 1000, 10000 (8, 8, …, 8), (11, 11, …,11), (12, 12, …, 12),(13, 13, …, 13)

18 Diagonal 4 2, 4, 10, 100, 500, 1000, 10000 (2, 2, …, 2), (5, 5, …,5), (10, 10, …, 10), (15, 15, …, 15)

Cont: Numerical Results

• Performance Profile based on Number iterations

Cont: Numerical Results

• Performance Profile based on CPU time

• Conclusion

* AMRI was able to solve 95% of test problems.

* WYL solve 97% of test problems.

* SW-A solve all test problems.

• References* M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. Sec. B 49 1952, pp. 409–432

*Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 2006, pp. 1341–1350.

*G. Zoutendijk, Nonlinear programming computational methods, in:J.Abadie(Ed.), Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970, pp. 37–86.

*M.Rivaie,M.Mamat,L. June, M.Ismail, a new class of nonlinear conjugate gradient coefficient with global convergence properties, Appl.Math.comput.218 ,2012, pp.11323-11332.

. * Y.H. Dai, Y. Yuan. An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 2001, pp. 33–47.

* E. Dolan, J.J.More, Benchmarking optimization software with performance profile, Math. Prog. 91, 2002, pp. 201–213.

*Y. F. Hu, C. Storey . Global convergence result for conjugate gradient methods. J.Optim.Theory.Appl., 71,1991, pp. 399-405.

* N. Andrei, An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 2008, pp. 147–161.

•

Thank You

We encourage our participants to adopt a crisp 10 Minutes Presentation Format.

1 Minute: Self Introduction and Topic

2 Minutes: Problem Area/ Need of Research

2 Minutes: Inputs/ Efforts/ Methodology

2 Minutes: Outputs

1 Minute: Take Aways

2 Minutes: Discussion

This format is tried and tested, and makes the sessions verymeaningful, rather than monotonous and full of slumber.Knowledgeable participants, but with poor soft-skills also find iteasy to present. This format provides more time for networkingand discussions and idea sharing, which are the core purpose ofour conferences.