MS-TWSVM: Mahalanobis distance-based Structural Twin Support Vector Machine

Ramin Rezvani-KhorashadiZadeh, Reza Monsefi


The distribution information of data points in two classes as the structural information is inserted into the classifiers to improve their generalization performance. Recently many algorithms such as S-TWSVM has used this information to construct two non-parallel hyperplanes which each one lies as close as possible to one class and being far away from the other. It is well known that different classes have different data distribution in real world problems, thus the covariance matrices of these classes are not the same. In these situations, the Mahalanobis is often more popular than Euclidean as a measure of distance. In this paper, in addition to apply the idea of S-TWSVM, the classical Euclidean distance is replaced by Mahalanobis distance which leads to simultaneously consider the covariance matrices of the two classes. By this modification, the orientation information in two classes can be better exploited than S-TWSVM. The experiments indicate our proposed algorithm is often superior to other learning algorithms in terms of generalization performance.


Structural information; Non-parallel hyperplanes; Non-parallel hyperplanes; Ward`s linkage; Twin support vector machine; Structural twin support vector machine

Full Text:



A. Ruiz, P.L.-T. Pedro, Nonlinear kernel-based statistical pattern analysis, IEEE Transactions on Neural Networks 12 (2001) 16–32.

B. Boser, L. Guyon, V.N. Vapnik, A training algorithm for optimal margin classifiers, in: Proceedings of the 5th Annual Workshop on Computational Learning Theory, ACM Press, Pittsburgh, (1992) 144–152.

B. Haasdonk, E. Pekalska, Classification with kernel Mahalanobis distance classifiers, in: Advances in Data Analysis, Data Handling and Business Intelligence Studies in Classification, Data Analysis, and Knowledge Organization, (2010) Part 5 351–361.

D. Wang, D.S. Yeung, E.C.C. Tsang, Weighted the Mahalanobis distance kernels for support vector machines, IEEE Transactions on Neural Networks 18 (2007) 1453–1462.

D. Yeung, D. Wang, W. Ng, E. Tsang, X. Zhao, Structured large margin machines: sensitive to data distributions, Machine Learning 68 (2) (2007) 171–200.

F.R. Gantmacher, Matrix Theory, New York, Chelsea, (1990).

G. Rätsch, Benchmark Repository, datasets,, (2000).

G.R.G. Lanckriet, L.E. Ghaoui, C. Bhattacharyya, M.I. Jordan, A robust minimax approach to classfication, Journal of Machine Learning & Research 3 (2002) 555–582.

H. Xue, S. Chen, Q. Yang, Structural regularized support vector machine: a framework for structural large margin classifier, IEEE Transactions on Neural Networks 22 (4) (2011) 573–587,

J. H. Ward, Hierarchical grouping to optimize an objective function, Journal of the American Statistical Association 58 (301) (1963) 236–244.

J.A. Hartigan, M.A. Wong, A k-means clustering algorithm, Applied Statistics 28 (1) (1979) 100–108.

J.A. Schinka, W.F. Velicer, I.B. Weiner, Research methods in psychology, Wiley, New york, (2003).

Jayadeva, R. Khemchandani, S. Chandra, Twin support vector machines for pattern classification, IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (5) (2007) 905–910.

K. Huang, H. Yang, I. King, M.R. Lyu, L. Chan, The minimum error minimax probability machine, Journal of Machine Learning Research 5 (2004) 1253–1286.

K. Huang, H. Yang, I. King, M.R. Lyu, Maxi-min margin machine-learning large margin classifiers locally and globally, IEEE Transactions on Neural Networks (2008) 260–272.

L.A. Zadeh, Fuzzy sets, Information Control 8 (1965) 338–353.

M.A. Kumar, M. Gopal, Application of smoothing technique on twin support vector machines, Pattern Recognition Letter 29 (6) (2008) 1842–1848.

M.A. Kumar, M. Gopal, Least squares twin support vector machines for pattern classification, Expert Systems with Applications 36 (4) (2009) 7535–7543.

M.A. Kumar, R. Khemchandani, M. Gopal, S. Chandra, Knowledge based least squares twin support vector machines, Information Sciences 180 (16) (2010) 4606–4618.

M.A. Woodbury, Inverting modified matrices, Memorandum Rept. 42,

Statistical Research Group, Princeton University, Princeton, NJ, (1950).

P.K. Shivaswamy, T. Jebara, Ellipsoidal kernel machines, in Proceeding of 12th International Workshop on Artificial Intelligence Statistic, (2007) 1–8.

R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, B. Kijsirikul, A new kernelization framework for Mahalanobis distance learning algorithms, Neurocomputing 73 (10–12) (2010) 1570–1579.

R. De Maesschalck, D. Jouan-Rimbaud, D.L. Massart, The Mahalanobis distance, Chemometrics and Intelligent Laboratory Systems 50 (2000) 1–18.

R. Gnanadesikan, J.R. Kettenring, Robust estimates, residuals, and outlier detection with multiresponse data, Biometrics 28 (1972) 81–124.

S. Salvador, P. Chan, Determining the Number of Clusters/Segments in

Hierarchical Clustering/Segmentation Algorithms, Tech. Rep., (2003).

S. Xiang, F. Nie, C. Zhang, Learning a Mahalanobis distance metric for data clustering and classification, Pattern Recognition 41 (2008) 3600–3612.

S.-Y. Lu, K.S. Fu, A sentence-to-sentence clustering procedure for pattern analysis, IEEE Transactions on Systems Man & Cybernetics 8 (5) (1978) 381–389.

V.N. Vapnik, Statistical Learning Theory, Wiley, New York, (1998).

V.N. Vapnik, The Natural of Statistical Learning Theory, Springer, New York, (1995).

X. Peng, A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms, Information Sciences 180 (8) (2010) 3863–3875.

X. Peng, Building sparse twin support vector machine classifiers in primal space, Information Sciences 181 (11) (2011) 3967–3980.

X. Peng, D. Xu, Twin Mahalanobis distance-based support vector machines for pattern recognition, Information Sciences 200 (1) (2013) 22–37.

X. Peng, Primal twin support vector regression and its sparse approximation, Neurocomputing 73 (16–18) (2010) 2846–2858.

X. Peng, TSVR: an efficient twin support vector machine for regression, Neural Networks 23 (3) (2010) 365–372.

Y.-H. Shao, C.-H. Zhang, X.-B. Wang, N.-Y. Deng, Improvements on twin support vector machines, IEEE Transactions on Neural Networks 22 (6) (2011) 962–968.

Z. Qi, Y. Tian, Y. Shi, Structural twin support vector machine for classification, Knowledge-Based Systems 43 (2013) 74–81.

Lululemon Black Friday cheap nfl jerseys Lululemon factory Outlet ny Black Friday discount tiffany outlet wholesale soccer jerseys online oakley black friday cheap nhl jerseys china cheap nfl jerseys north face black friday sale cheap nfl jerseys online Jordans Black Friday Sale 2015 Cheap Moncler Cyber Monday moncler outlet cheap soccer jerseys moncler outlet black friday cheap authentic nfl jerseys north face cyber monday Louboutin Black Friday canada wholesale cheap nfl jerseys lululemon cyber monday 2015 cheap nfl jerseys from china 2015 Cheap Moncler Black Friday Sale Moncler Cyber Monday 2015 cheap jerseys Lululemon Cyber Monday Sale jordans cyber monday deals 2015 cheap nike nfl jerseys Black Friday deals Lululemon 2015 jordan black friday 2015 Moncler Jackets Black Friday Sale 2015 Louboutin Pas Cher Black Friday 2015 Canada Lululemon north face black friday cheap wholesale soccer jerseys