Recurrent Neural Network Training with the Kalman Filter-based Techniques

Title:

Recurrent Neural Network Training with the Kalman Filter-based Techniques

Creator:

Trebaticky, Petr

Identifier:

https://cdk.lib.cas.cz/client/handle/uuid:232f4215-f96f-470c-a54f-9f6d95a1bbe5
uuid:232f4215-f96f-470c-a54f-9f6d95a1bbe5

Subject:

Recurrent neural network, Kalman Filter, Extended Kalman Filter, Unscented Kalman Filter, and nprKF

Type:

model:article and TEXT

Format:

bez média and svazek

Description:

Recurrent neural networks, in contrast to the classical feedforward neural networks, handle better the inputs that have space-time structure, e.g. symbolic time series. Since the classic gradient methods for recurrent neural network training on longer input sequences converge very poorly and slowly, alternative approaches are needed. We describe the training method with the Extended Kalman Filter and with its modifications Unscented Kalman Filter, nprKF and with their joint versions. The Joint Unscented Kalman Filter was not used for this purpose before. We compare the performance of these filters and of the classic Truncated Backpropagation Through Time (BPTT(h)) in two experiments for next-symbol prediction - word sequence generated by Reber automaton and the sequence generated by quantising the activations of a laser in a chaotic regime. All the new filters achieved significantly better results.

Language:

English

Rights:

http://creativecommons.org/publicdomain/mark/1.0/
policy:public

Source:

Neural network world: international journal on neural and mass-parallel computing and information systems | 2005 Volume:15 | Number:5

Harvested from:

CDK

Metadata only:

false