Advances in Computer Science : an International Journal
MS-TWSVM: Mahalanobis distance-based Structural Twin Support Vector Machine
Abstract
The distribution information of data points in two classes as the structural information is inserted into the classifiers to improve their generalization performance. Recently many algorithms such as S-TWSVM has used this information to construct two non-parallel hyperplanes which each one lies as close as possible to one class and being far away from the other. It is well known that different classes have different data distribution in real world problems, thus the covariance matrices of these classes are not the same. In these situations, the Mahalanobis is often more popular than Euclidean as a measure of distance. In this paper, in addition to apply the idea of S-TWSVM, the classical Euclidean distance is replaced by Mahalanobis distance which leads to simultaneously consider the covariance matrices of the two classes. By this modification, the orientation information in two classes can be better exploited than S-TWSVM. The experiments indicate our proposed algorithm is often superior to other learning algorithms in terms of generalization performance.
Keywords
Full Text:
PDFReferences
A. Ruiz, P.L.-T. Pedro, Nonlinear kernel-based statistical pattern analysis, IEEE Transactions on Neural Networks 12 (2001) 16–32.
B. Boser, L. Guyon, V.N. Vapnik, A training algorithm for optimal margin classifiers, in: Proceedings of the 5th Annual Workshop on Computational Learning Theory, ACM Press, Pittsburgh, (1992) 144–152.
B. Haasdonk, E. Pekalska, Classification with kernel Mahalanobis distance classifiers, in: Advances in Data Analysis, Data Handling and Business Intelligence Studies in Classification, Data Analysis, and Knowledge Organization, (2010) Part 5 351–361.
D. Wang, D.S. Yeung, E.C.C. Tsang, Weighted the Mahalanobis distance kernels for support vector machines, IEEE Transactions on Neural Networks 18 (2007) 1453–1462.
D. Yeung, D. Wang, W. Ng, E. Tsang, X. Zhao, Structured large margin machines: sensitive to data distributions, Machine Learning 68 (2) (2007) 171–200.
F.R. Gantmacher, Matrix Theory, New York, Chelsea, (1990).
G. Rätsch, Benchmark Repository, datasets, http://ida.first.fhg.de/projects/bench/benchmarks.htm, (2000).
G.R.G. Lanckriet, L.E. Ghaoui, C. Bhattacharyya, M.I. Jordan, A robust minimax approach to classfication, Journal of Machine Learning & Research 3 (2002) 555–582.
H. Xue, S. Chen, Q. Yang, Structural regularized support vector machine: a framework for structural large margin classifier, IEEE Transactions on Neural Networks 22 (4) (2011) 573–587, http://dx.doi.org/10.1109/TNN.2011.2108315.
J. H. Ward, Hierarchical grouping to optimize an objective function, Journal of the American Statistical Association 58 (301) (1963) 236–244.
J.A. Hartigan, M.A. Wong, A k-means clustering algorithm, Applied Statistics 28 (1) (1979) 100–108.
J.A. Schinka, W.F. Velicer, I.B. Weiner, Research methods in psychology, Wiley, New york, (2003).
Jayadeva, R. Khemchandani, S. Chandra, Twin support vector machines for pattern classification, IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (5) (2007) 905–910.
K. Huang, H. Yang, I. King, M.R. Lyu, L. Chan, The minimum error minimax probability machine, Journal of Machine Learning Research 5 (2004) 1253–1286.
K. Huang, H. Yang, I. King, M.R. Lyu, Maxi-min margin machine-learning large margin classifiers locally and globally, IEEE Transactions on Neural Networks (2008) 260–272.
L.A. Zadeh, Fuzzy sets, Information Control 8 (1965) 338–353.
M.A. Kumar, M. Gopal, Application of smoothing technique on twin support vector machines, Pattern Recognition Letter 29 (6) (2008) 1842–1848.
M.A. Kumar, M. Gopal, Least squares twin support vector machines for pattern classification, Expert Systems with Applications 36 (4) (2009) 7535–7543.
M.A. Kumar, R. Khemchandani, M. Gopal, S. Chandra, Knowledge based least squares twin support vector machines, Information Sciences 180 (16) (2010) 4606–4618.
M.A. Woodbury, Inverting modified matrices, Memorandum Rept. 42,
Statistical Research Group, Princeton University, Princeton, NJ, (1950).
P.K. Shivaswamy, T. Jebara, Ellipsoidal kernel machines, in Proceeding of 12th International Workshop on Artificial Intelligence Statistic, (2007) 1–8.
R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, B. Kijsirikul, A new kernelization framework for Mahalanobis distance learning algorithms, Neurocomputing 73 (10–12) (2010) 1570–1579.
R. De Maesschalck, D. Jouan-Rimbaud, D.L. Massart, The Mahalanobis distance, Chemometrics and Intelligent Laboratory Systems 50 (2000) 1–18.
R. Gnanadesikan, J.R. Kettenring, Robust estimates, residuals, and outlier detection with multiresponse data, Biometrics 28 (1972) 81–124.
S. Salvador, P. Chan, Determining the Number of Clusters/Segments in
Hierarchical Clustering/Segmentation Algorithms, Tech. Rep., (2003).
S. Xiang, F. Nie, C. Zhang, Learning a Mahalanobis distance metric for data clustering and classification, Pattern Recognition 41 (2008) 3600–3612.
S.-Y. Lu, K.S. Fu, A sentence-to-sentence clustering procedure for pattern analysis, IEEE Transactions on Systems Man & Cybernetics 8 (5) (1978) 381–389.
V.N. Vapnik, Statistical Learning Theory, Wiley, New York, (1998).
V.N. Vapnik, The Natural of Statistical Learning Theory, Springer, New York, (1995).
X. Peng, A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms, Information Sciences 180 (8) (2010) 3863–3875.
X. Peng, Building sparse twin support vector machine classifiers in primal space, Information Sciences 181 (11) (2011) 3967–3980.
X. Peng, D. Xu, Twin Mahalanobis distance-based support vector machines for pattern recognition, Information Sciences 200 (1) (2013) 22–37.
X. Peng, Primal twin support vector regression and its sparse approximation, Neurocomputing 73 (16–18) (2010) 2846–2858.
X. Peng, TSVR: an efficient twin support vector machine for regression, Neural Networks 23 (3) (2010) 365–372.
Y.-H. Shao, C.-H. Zhang, X.-B. Wang, N.-Y. Deng, Improvements on twin support vector machines, IEEE Transactions on Neural Networks 22 (6) (2011) 962–968.
Z. Qi, Y. Tian, Y. Shi, Structural twin support vector machine for classification, Knowledge-Based Systems 43 (2013) 74–81.
