A hybrid CG algorithm for nonlinear unconstrained optimization with application in image restoration

Document Type : Research Article

Authors

1 Laboratory of Fundamental and Numerical Mathematics (LMFN), University Ferhat Abbas Setif 1, Algeria

2 Department of Mathematics, College of Computer Science and Mathematics, University of Mosul, Mosul, Iraq

Abstract

This paper presents a new hybrid conjugate gradient method for solving  nonlinear unconstrained optimization problems; it is based on a combination of $RMIL$  (Rivaie-Mustafa-Ismail-Leong)  and $hSM$  (hybrid Sulaiman- Mohammed) methods. The proposed algorithm enjoys the sufficient descent condition without depending on any line search; moreover, it is globally convergent under the usual and strong Wolfe line search assumptions.  The performance of the algorithm is demonstrated through numerical experiments on a set of 100 test functions from [1] and four image restoration problems with two noise levels. The numerical comparisons with four existing methods show that the proposed method is promising and effective.

Keywords

Main Subjects


[1] N. Andrei, An unconstrained optimization test functions, Adv. Modeling Optim. 10 (2008) 147–161.
[2] Y.H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property.
SIAM J. Optim. 10 (1999) 177–182.
[3] E.D. Dolan, J.J. Moré, Benchmarking optimization software with performance profiles, Math. Program. 91
(2002) 201–213.
[4] R. Fletcher, Practical Methods of Optimization, 2nd Ed., J. Wiley, Sons, New York, USA, 1987.
[5] R. Fletcher, C.M. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964) 149–154.
[6] J.C. Gilbert, J. Nocedal, Global convergence properties of conjugate gradient methods for optimization,
SIAM J. Optim. 2 (1992) 21–42.
[7] M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur.
Stand. 49 (1952) 409–436.
[8] Y.I. Ibrahim, H.M. Khudhur, Modified three-term conjugate gradient algorithm and its applications in image
restoration, J. Electr. Eng. Comput. 28 (2022) 1510–1517.
[9] M. Koontse, P. Kaelo, Another hybrid conjugate gradient method for unconstrained optimization, J. Nonlin-
ear Anal. Optim. 2 (2014) 127–137.
[10] Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms,Part 1, Theory, J. Optim. Theory. Appl.
69 (1991) 129–137.
[11] I.E. Livieris, V. Tampakas, P. Pintelas, A descent hybrid conjugate gradient method based on the memoryless
BFGS update, Numer. Algor. 79 (2018) 11691185.
[12] M. Lotfi, S.M. Hosseini, An efficient hybrid conjugate gradient method with sufficient descent property for
unconstrained optimization, Optim. Methods Softw. 37 (2022) 1725–1739.
[13] E. Mehamdia, Y. Chaib, T. Bechouat, Two modified conjugate gradient methods for unconstrained optimiza-
tion and applications, RAIRO-Oper. Res. 57 (2023) 333–350.
[14] P. Mtagulwa, P. Kaelo, An efficient modified PRP-FR hybrid conjugate gradient method for solving uncon-
strained optimization problems, Appl. Numer. Math. 145 (2019) 111–120.
[15] E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Revue Française
d’informatique et de Recherche Opérationnelle, Série Rouge 3 (1969) 35–43 .
[16] B.T. Polyak, The conjugate gradient method in extremal problems, USSR Comput. Math. Math. Phys. 9
(1969) 94–112.
[17] M. Rivaie, M. Mamat, L. W. June, I. Mohd, A new class of nonlinear conjugate gradient coefficients with
global convergence properties, Appl. Math. Comput. 218 (2012) 11323–11332.
[18] I. M. Sulaiman, N. A. Bakar, M. Mamat, B.A. Hassan, M. Malik, A. M. Ahmed, A new hybrid conjugate
gradient algorithm for optimization models and its application to regression analysis, J. Electr. Eng. Comput.
23 (2021) 1100–1109.
[19] D. Touati-Ahmed, C. Storey, Efficient hybrid conjugate gradient technique, J. Optim., Theory. Appl. 64
(1990) 379–397.
[20] L. Zhang, An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation,
Appl. Math. Comput. 215 (2009) 2269–2274.
[21] R. Ziadi, R. Ellaia, A. Bencherif-Madani, Global optimization through a stochastic perturbation of the Polak-
Ribiére conjugate gradient method, J. Comput. Appl. Math. 317 (2017) 672–684.
[22] R. Ziadi, A. Bencherif-Madani, A mixed algorithm for smooth global optimization, J. Math. Model. 11 (2023)
207–228.
[23] R. Ziadi, A. Bencherif-Madani, A Perturbed quasi-Newton algorithm for bound-constrained global optimiza-
tion, J. Comp. Math., 2023, doi:10.4208/jcm.2307-m2023-0016.
[24] G. Zoutendijk, Nonlinear programming computational methods, J. Integ. Nonlinear Progr. (1970) 37–86