A new damping strategy of Levenberg-Marquardt algorithm for Multilayer Perceptrons

Title:

A new damping strategy of Levenberg-Marquardt algorithm for Multilayer Perceptrons

Creator:

Kwak, Young-tae, Hwang, Ji-won, and Yoo, Cheol-jung

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:fd174f12-1263-415f-9483-ac373c29c75b
uuid:fd174f12-1263-415f-9483-ac373c29c75b
doi:10.14311/NNW.2011.21.020

Subject:

Levenberg-Marquardt algorithm, damping parameter, Gauss-Newton method, and error backpropagation

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

In this paper, a new adjustment to the damping parameter of the Levenberg-Marquardt algorithm is proposed to save training time and to reduce error oscillations. The damping parameter of the Levenberg-Marquardt algorithm switches between a gradient descent method and the Gauss-Newton method. It also affects training speed and induces error oscillations when a decay rate is fixed. Therefore, our damping strategy decreases the damping parameter with the inner product between weight vectors to make the Levenberg-Marquardt algorithm behave more like the Gauss-Newton method, and it increases the damping parameter with a diagonally dominant matrix to make the Levenberg-Marquardt algorithm act like a gradient descent method. We tested two simple classifications and a handwritten digit recognition for this work. Simulations showed that our method improved training speed and error oscillations were fewer than those of other algorithms.

Language:

English

Rights:

http://creativecommons.org/publicdomain/mark/1.0/
policy:public

Source:

Neural network world: international journal on neural and mass-parallel computing and information systems | 2011 Volume:21 | Number:4

Harvested from:

CDK

Metadata only:

false