A modified higher-order feed forward neural network with smoothing regularization

Title:

A modified higher-order feed forward neural network with smoothing regularization

Creator:

Mohamed, Kh.Sh., Wu, W., and Liu, Y.

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:27fa5028-da94-401d-bb36-23cbad3ccf53
uuid:27fa5028-da94-401d-bb36-23cbad3ccf53
doi:10.14311/NNW.2017.27.032

Subject:

convergence, pi-sigma neural network, offline (batch) gradient method, and smoothing L1/2 regularization

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

This paper proposes an offline gradient method with smoothing L1/2 regularization for learning and pruning of the pi-sigma neural networks (PSNNs). The original L1/2 regularization term is not smooth at the origin, since it involves the absolute value function. This causes oscillation in the computation and difficulty in the convergence analysis. In this paper, we propose to use a smooth function to replace and approximate the absolute value function, ending up with a smoothing L1/2 regularization method for PSNN. Numerical simulations show that the smoothing L1/2 regularization method eliminates the oscillation in computation and achieves better learning accuracy. We are also able to prove a convergence theorem for the proposed learning method.

Language:

English

Rights:

http://creativecommons.org/publicdomain/mark/1.0/
policy:public

Coverage:

577-592

Source:

Neural network world: international journal on neural and mass-parallel computing and information systems | 2017 Volume:27 | Number:6

Harvested from:

CDK

Metadata only:

false