Recently, a support vector machine (SVM) has been receiving increasing attention in the field of regression estimation due to its remarkable characteristics such as good generalization performance, the absence of local minima and sparse representation of the solution. However, within the SVMs framework, there are very few established approaches for identifying important features. Selecting significant features from all candidate features is the first step in regression estimation, and this procedure can improve the network performance, reduce the network complexity, and speed up the training of the network.
This paper investigates the use of saliency analysis (SA) and genetic algorithm (GA) in SVMs for selecting important features in the context of regression estimation. The SA measures the importance of features by evaluating the sensitivity of the network output with respect to the feature input. The derivation of the sensitivity of the network output to the feature input in terms of the partial derivative in SVMs is presented, and a systematic approach to remove irrelevant features based on the sensitivity is developed. GA is an efficient search method based on the mechanics of natural selection and population genetics. A simple GA is used where all features are mapped into binary chromosomes with a bit "1" representing the inclusion of the feature and a bit of "0" representing the absence of the feature. The performances of SA and GA are tested using two simulated non-linear time series and five real financial time series. The experiments show that with the simulated data, GA and SA detect the same true feature set from the redundant feature set, and the method of SA is also insensitive to the kernel function selection. With the real financial data, GA and SA select different subsets of the features. Both selected feature sets achieve higher generation performance in SVMs than that of the full feature set. In addition, the generation performance between the selected feature sets of GA and SA is similar. All the results demonstrate that that both SA and GA are effective in the SVMs for identifying important features.
Support Vector Machines (SVM) are well known as a kernel based method mostly applied to classification. SVM-Recursive Feature Elimination (SVM- RFE) is a variable ranking and selection method dedicated to the design of SVM based classifiers. In this paper, we propose to revisit the SVM-RFE method. We study two implementations of this feature selection method that we call External SVM-RFE and Internal SVM-RFE, respectively. The two implementations are applied to rank and select acoustic features extracted from speech to design optimized linear SVM classifiers that recognize speaker emotions. To show the efficiency of the External and Internal SVM-RFE methods, an extensive experimental study is presented. The SVM classifiers were selected using a validation procedure that ensures strict speaker independence. The results are discussed and compared with those achieved when the features are ranked using the Gram-Schmidt procedure. Overall, the results achieve a recognition rate that exceeds 90%.