In this paper, a new adjustment to the damping parameter of the Levenberg-Marquardt algorithm is proposed to save training time and to reduce error oscillations. The damping parameter of the Levenberg-Marquardt algorithm switches between a gradient descent method and the Gauss-Newton method. It also affects training speed and induces error oscillations when a decay rate is fixed. Therefore, our damping strategy decreases the damping parameter with the inner product between weight vectors to make the Levenberg-Marquardt algorithm behave more like the Gauss-Newton method, and it increases the damping parameter with a diagonally dominant matrix to make the Levenberg-Marquardt algorithm act like a gradient descent method. We tested two simple classifications and a handwritten digit recognition for this work. Simulations showed that our method improved training speed and error oscillations were fewer than those of other algorithms.
In this study, a new approach based on the consideration that electroencephalogram (EEG) signals are chaotic signals was presented for automated diagnosis of electroencephalographic changes. This consideration was tested successfully using the nonlinear dynamics tools, like the computation of Lyapunov exponents. Multilayer perceptron neural network (MLPNN) architectures were formulated and used as basis for detection of electroencephalographic changes. Three types of EEG signals (EEG signals recorded from healthy volunteers with eyes open, epilepsy patients in the epileptogenic zone during a seizure-free interval, and epilepsy patients during epileptic seizures) were classified. The computed Lyapunov exponents of the EEG signals were used as inputs of the MLPNNs trained with backpropagation, delta-bar-delta, extended delta-bar-delta, quick propagation, and Levenberg-Marquardt algorithms. The performances of the MLPNN classifiers were evaluated in terms of training performance and classification accuracies. Receiver operating characteristic (ROC) curves were used to assess the performance of the detection process. The results confirmed that the proposed MLPNN trained with the Levenberg-Marquardt algorithm has potentiality in detecting the electroencephalographic changes.
In this paper, we propose a new global and fast Multilayer Perceptron Neural Network (MLP-NN) which can be used to forecast the automotive price. Nowadays, the gradient-based techniques, such as back propagation, are widely used for training neural networks. These techniques have local convergence results and, therefore, can perform poorly even on simple problems when forecasting is out of sample. On the other hand, the global search algorithms, like Tabu Search (TS), suffer from low rate convergence. Motivated by these facts, a new global and fast hybrid algorithm for training MLP-NN is provided. In our new framework, a hybridization of an extended version of TS with some local techniques is constructed in order to train the connected weights of the network. The extended version of TS in the proposed scheme consists of a simple TS together with the intensification and diversification search methods, and the local search methods are based on a direct strategy of Nelder-Mead (NM) or Levenberg-Marquardt (LM) techniques. This hybridization leads us to have a global and fast trained network in order to use in some forecasting problems. To show the efficiency and effectiveness of our new proposed network, we apply our new scheme for forecasting the automotive price in Iran Khodro Company which is the biggest car manufacturer in Iran. The results are promising compared to the cases when we apply the TS and some other forecasting techniques individually. We also compare the results with the case when we employ the gradient-based optimization techniques such as LM, and global search methods such as Genetic Algorithm (GA) and hybrid of MLP-NN with GA.