Support vector machine (SVM) has become one of the most popular machine-learning methods during the last years. The design of an efficient model and the proper adjustment of the SVMs parameters are integral to reducing the testing time and enhancing performance. In this paper, a new bipartite objective function consisted of the sparseness property and generalization performance is proposed. Since the proposed objective function is based on selecting fewer numbers of the support vectors, the model complexity is reduced while the performance accuracy remains at an acceptable level. Due to the model complexity reduction, the testing time is decreased and the ability of SVM in practical applications is increased Moreover, to prove the performance of the proposed objective function, a comparative study was carried out on the proposed objective function and the conventional objective function, which is only based on the generalization performance, using the Binary Genetic Algorithm (BGA) and Real-valued vectors GA (RGA). The effectiveness of the proposed cost function is demonstrated based on the results of the comparative study on four real-world datasets of UCI database.
Recently, a support vector machine (SVM) has been receiving increasing attention in the field of regression estimation due to its remarkable characteristics such as good generalization performance, the absence of local minima and sparse representation of the solution. However, within the SVMs framework, there are very few established approaches for identifying important features. Selecting significant features from all candidate features is the first step in regression estimation, and this procedure can improve the network performance, reduce the network complexity, and speed up the training of the network.
This paper investigates the use of saliency analysis (SA) and genetic algorithm (GA) in SVMs for selecting important features in the context of regression estimation. The SA measures the importance of features by evaluating the sensitivity of the network output with respect to the feature input. The derivation of the sensitivity of the network output to the feature input in terms of the partial derivative in SVMs is presented, and a systematic approach to remove irrelevant features based on the sensitivity is developed. GA is an efficient search method based on the mechanics of natural selection and population genetics. A simple GA is used where all features are mapped into binary chromosomes with a bit "1" representing the inclusion of the feature and a bit of "0" representing the absence of the feature. The performances of SA and GA are tested using two simulated non-linear time series and five real financial time series. The experiments show that with the simulated data, GA and SA detect the same true feature set from the redundant feature set, and the method of SA is also insensitive to the kernel function selection. With the real financial data, GA and SA select different subsets of the features. Both selected feature sets achieve higher generation performance in SVMs than that of the full feature set. In addition, the generation performance between the selected feature sets of GA and SA is similar. All the results demonstrate that that both SA and GA are effective in the SVMs for identifying important features.
A key physical property used in the description of a soil-water regime is a soil water retention curve, which shows the relationship between the water content and the water potential of the soil. Pedotransfer functions are based on the supposed dependence of the soil water content on the available soil characteristics, e.g., on the relative content of the particle size in the soil and the dry bulk density of the soil. This dependence could be extracted from the available data by various regression methods. In this paper, artificial neural networks (ANNs) and support vector machines (SVMs) were used to estimate a drying branch of a water retention curve. The paper compares the mentioned methods by estimating the water retention curves on regional scale for the Záhorská lowland in the Slovak Republic, where relatively small data set was available. The performance of the models was evaluated and compared. These computations did not fully confirm the superiority of SVMs over ANNs as is often proclaimed in the literature, because the results obtained show that in this study the ANN model performs somewhat better and is easier to handle in determining pedotransfer functions than the SVM models. Nevertheless, the results from both data-driven models are quite close, and the results show that they provide a significantly more precise outcome than a traditional multi-linear regression does., Autori sa v príspevku venujú určovaniu pedotransferových funkcií (PTF), ktoré umožňujú stanoviť body vlhkostných retenčných kriviek pôdy z ľahšie merateľných pôdnych vlastností a sú dôležitým prvkom modelovania vodného režimu pôdy. Ešte v minulej dekáde sa objavili snahy využívať na ich určenie umelé neurónové siete (UNS). Multi-layer perceptron (MLP) čiže viacvrstvový perceptrón je najčastejšie používaný model doprednej umelej neurónovej siete s kontrolovaným typom učenia. Vstupné signály prechádzajú sieťou typu MLP iba dopredným smerom, teda postupne od vrstvy k vrstve. MLP používa tri a viac vrstiev neurónov rozdelených na vstupnú, skrytú a výstupnú vrstvu s nelineárnou aktivačnou funkciou a vie rozpoznať alebo modelovať informácie, ktoré nie sú lineárne oddeliteľné alebo závislé. Novší vývoj v oblasti učiacich algoritmov poskytuje ďalšie možnosti, z ktorých sa v tomto príspevku venujeme tzv. mechanizmom podporných vektorov (Support Vector Machines - SVM). SVM využíva pri svojom kalibrovaní na riešený problém princíp tzv. štrukturálnej minimalizácie namiesto iba minimalizácie chyby - (Vapnik, 1995). Pri trénovaní siete MLP je jediným cieľom minimalizovať celkovú chybu. Pri SVM sa simultánne minimalizuje chyba aj zložitosť modelu. Použitie tohto princípu vedie zvyčajne k vyššej schopnosti generalizácie, t.j. umožneniu presnejších predpovedí pre dáta, ktoré neboli použité pri trénovaní SVM. Vhodnosť štandardnej umelej neurónovej siete, SVM a viacnásobnej lineárnej regresie sa v článku vyhodnocuje na základe údajov získaných z pôdnych vzoriek odobratých v lokalite Záhorskej nížiny. Pôvodné údaje a ich aplikáciu pri vyhodnocovaní vodného režimu pôd uvádza Skalová (2001, 2007), odkiaľ boli prevzaté vstupné dáta a to percentuálny obsah zrnitostných kategórií (I až IV podľa Kopeckého), redukovaná objemová hmotnosť (ρd) a vlhkosti pre vlkostné potenciály hw= -2.5, -56, -209, -558, -976, -3060, -15300 cm, ktoré boli stanovené laboratórne pre potreby určenia a testovania regresných závislostí. Vzhľadom na to, že pri odvodzovaní regionálnych PTF je častým prípadom nedostatok dát pre odvodenie dátovo riadených modelov, autori navrhli riešiť úlohu pomocou ansámblu MLP resp. SVM. Ansámbel dátovo riadených modelov bol vytvorený variabilným rozdelením údajov na trénovacie a validačné (validačnými údajmi sa testuje presnosť modelu vo fáze jeho tvorby, ešte sa používajú konečné testovacie dáta, ktoré neboli pri tvorbe modelu použité). Výsledky ukázali lepšie regresné schopnosti oboch dátovo riadených modelov (SVM aj MLP) voči multilineárnej regresii a o niečo lepšie výsledky boli získané z viacvrstvového perceptrónu než zo SVM., and Keďže v niektorých iných prácach mal zvyčajne vyššiu výpočtovú presnosť model založený na SVM než na UNS, autori odporúčajú pre budúci výskum preveriť vhodnosť kombinácie SVM a MLP modelov v dátovo riadenom skupinovom modeli.
This paper summarizes recent results on applying the method of partial least squares (PLS) in a reproducing kernel Hilbert space (RKHS). A previously proposed nonlinear kernel-based PLS regression model has proven to be competitive with other regularized regression methods in RKHS. In this paper the use of kernel PLS for discrimination is discussed. A new methodology for classification is then proposed. This is based on kernel PLS dimensionality reduction of the original data space followed by a support vector classifier. Good results using this method on a two-class classification problem are reported here.
This paper addresses the problem of probability estimation in Multiclass classification tasks combining two well-known data mining techniques: Support Vector Machines and Neural Networks. We present an algorithm which uses both techniques in a two-step procedure. The first step employs Support Vector Machines within a One-vs-All reduction from multiclass to binary approach to obtain the distances between each observation and the Support Vectors representing the classes. The second step uses these distances as inputs for a Neural Network, built with an entropy cost function and softmax transfer function for the output layer where class membership is used for training. Consequently, this network estimates probabilities of class membership for new observations. A benchmark using different databases demonstrates that the proposed algorithm is highly competitive with the most recent techniques for multiclass probability estimation.
This paper deals with the application of saliency analysis to Support
Vector Machines (SVMs) for feature selection. The importance of feature is ranked by evaluating the sensitivity of the network output to the feature input in terms of the partial derivative. A systematic approach to remove irrelevant features based on the sensitivity is developed. Two simulated non-linear time series and five real financial time series are examined in the experiment. Based on the simulation results, it is shown that that saliency analysis is effective in SVMs for identifying important features.
This article presents an application of evolutionary fuzzy rules to the modeling and prediction of power output of a real-world Photovoltaic Power Plant (PVPP). The method is compared to artificial neural networks and support vector regression that were also used to build predictors in order to analyse a time-series like data describing the production of the PVPP. The models of the PVPP are created using different supervised machine learning methods in order to forecast the short-term output of the power plant and compare the accuracy of the prediction.
Risk assessment of credit portfolios is of pivotal importance in the banking industry. The bank that has the most accurate view of its credit risk will be the most profitable. One of the main pillars in the assessing credit risk is the estimated probability of default of each counterparty, ie the probability that a counterparty cannot meet its payment obligations in the horizon of one year. A credit rating system takes several characteristics of a counterparty as inputs and assigns this counterparty to a rating class. In essence, this system is a classifier whose classes lie on an ordinal scale.
In this paper we apply linear regression ordinal logistic regression, and support vector machine techniques to the credit rating problem. The latter technique is a relatively new machine learning technique that was originally designed for the two-class problem. We propose two new techniques that incorporate the ordinal character of the credit rating problem into support vector machines. The results of our newly introduced techniques are promising.