Probabilistic Twin Support Vector Machine for Solving Unclassifiable Region Problem

Document Type : Original Article

Authors

1 Faculty of Mathematical Sciences, Ferdowsi University of Mashhad, Mashhad, Iran

2 Department of Computer Engineering, Faculty of Engineering, Alzahra University, Tehran, Iran

Abstract

Support Vector Machine classifiers are widely used in many classification tasks. However, they have two considerable weaknesses, Unclassifiable Region (UR) in multiclass classification and outliers. In this research, we address these problems by introducing Probabilistic Least Square Twin Support Vector Machine (PLS-TSVM). The proposed algorithm introduces continuous and probabilistic outputs over the model obtained by Least-Square Twin Support Vector Machine (LS-TSVM) method with both linear and polynomial kernel functions. PLS-TSVM not only solves the unclassifiable region problem by introducing a continuous output value membership function, but it also reduces the adverse effects of noisy data and outliers. For showing the superiority of our proposed method, we have conducted experiments on various UCI datasets. In the most cases, higher or competitive accuracy to other methods have been obtained such that in some UCI datasets, PLS-TSVM could obtain up to 99.90% of classification accuracy. Moreover, PLS-TSVM has been evaluated against ”one-against-all” and ”one-against-one” approaches on several well-known video datasets such as Weizmann, KTH, and UCF101 for human action recognition task. The results show the higher accuracy of PLS-TSVM compared to its counterparts. Specifically, the proposed algorithm could improve respectively about 12.2%, 2.8%, and 12.1% of classification accuracy in three video datasets compared to the standard SVM and LS-TSVM classifiers. The final results indicate that the proposed algorithm could achieve better overall performances than the literature.

Keywords

References

  1. Cen, Feng, Xiaoyu Zhao, Wuzhuang Li, and Guanghui Wang. "Deep feature augmentation for occluded image classification." Pattern Recognition 111, (2021), 107737, doi: 10.1016/j.patcog.2020.107737.
  2. AlyanNezhadi, M. M., H. Qazanfari, A. Ajam, and Z. Amiri. "Content-based Image Retrieval Considering Colour Difference Histogram of Image Texture and Edge Orientation." International Journal of Engineering, Transactions B: Applications, 33, No. 5, (2020), 949-958, doi: 10.5829/ije.2020.33.05b.28.
  3. Sezavar, A., H. Farsi, and Sajad Mohamadzadeh. "A modified grasshopper optimization algorithm combined with cnn for content-based image retrieval." International Journal of Engineering, Transactions A: Basics, 32, No. 7, (2019), 924-930, doi: 10.5829/ije.2019.32.07a.04.
  4. Sharma, Parul, Yash Paul Singh Berwal, and Wiqas Ghai. "Performance analysis of deep learning CNN models for disease detection in plants using image segmentation." Information Processing in Agriculture 7, No. 4 (2020), 566-574, doi: 10.1016/j.inpa.2019.11.001
  5. Chen, Long, Liangxiao Jiang, and Chaoqun Li. "Modified DFS-based term weighting scheme for text classification." Expert Systems with Applications 168, (2021), 114438, doi: 10.1016/j.eswa.2020.114438
  6. Rahmanimanesh, Mohammad, Jalal A. Nasiri, Saeed Jalili, and N. Moghaddam Charkari. "Adaptive three-phase support vector data description." Pattern Analysis and Applications 22, No. 2, (2019), 491-504, doi: 10.1007/s10044-017-0646-3
  7. Refahi, Mohammad S., Jalal A. Nasiri, and S. M. Ahadi. "Ecg arrhythmia classification using least squares twin support vector machines." In Electrical Engineering (ICEE), Iranian Conference on, 1619-1623. IEEE, 2018.
  8. Okwuashi, Onuwa, and Christopher E. Ndehedehe. "Deep support vector machine for hyperspectral image classification." Pattern Recognition 103, (2020), 107298, doi: 10.1016/j.patcog.2020.107298
  9. Sivaram, M., E. Laxmi Lydia, Irina V. Pustokhina, Denis Alexandrovich Pustokhin, Mohamed Elhoseny, Gyanendra Prasad Joshi, and K. Shankar. "An optimal least square support vector machine-based earnings prediction of blockchain financial products." IEEE Access 8, (2020), 120321-120330, doi: 10.1109/ACCESS.2020.3005808
  10. Gao, Zheming, Shu-Cherng Fang, Jian Luo, and Negash Medhin. "A kernel-free double well potential support vector machine with applications." European Journal of Operational Research 290, No. 1 (2021), 248-262, doi: 10.1016/j.ejor.2020.10.040
  11. Badaghei, R., H. Hassanpour, and T. Askari. "Detection of Bikers without Helmet Using Image Texture and Shape Analysis." International Journal of Engineering, Transactions C: Aspects, 34, No. 3 (2021): 650-655, doi: 10.5829/ije.2021.34.03c.09
  12. Wang, Kuaini, Wenxin Zhu, and Ping Zhong. "Robust support vector regression with generalized loss function and applications." Neural Processing Letters 41, No. 1 (2015), 89-106, doi: 10.1007/s11063-013-9336-3
  13. Qu, Hai-Ni, Guo-Zheng Li, and Wei-Sheng Xu. "An asymmetric classifier based on partial least squares." Pattern Recognition 43, No. 10 (2010), 3448-3457, doi: 10.1016/j.patcog.2010.05.002
  14. Guo, Guodong, and Alice Lai. "A survey on still image based human action recognition." Pattern Recognition 47, No. 10 (2014), 3343-3361, doi: 10.1016/j.patcog.2014.04.018
  15. Xiao, Yanghao, Yucheng Liu, Yuanyuan Deng, and Haoxuan Li. "Enhancing Multi-Class Classification in One-Versus-One Strategy: A Type of Base Classifier Modification and Weighted Voting Mechanism." In 2021 International Conference on Communications, Information System and Computer Engineering (CISCE), 303-307. IEEE, 2021.
  16. Wu, Yuanyuan, Liyong Shen, and Sanguo Zhang. "Fuzzy multiclass support vector machines for unbalanced data." In 2017 29th Chinese Control and Decision Conference (CCDC), 2227-2231. IEEE, 2017.
  17. Pruengkarn, Ratchakoon, Kok Wai Wong, and Chun Che Fung. "Imbalanced data classification using complementary fuzzy support vector machine techniques and SMOTE." In 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 978-983. IEEE, 2017.
  18. Liu, Jie, and Enrico Zio. "A scalable fuzzy support vector machine for fault detection in transportation systems." Expert Systems with Applications 102 (2018), 36-43, doi: 10.1016/j.eswa.2018.02.017
  19. Inoue, Takuya, and Shigeo Abe. "Fuzzy support vector machines for pattern classification." In IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222), Vol. 2, 1449-1454. IEEE, 2001.
  20. Ji, Ai-bing, Songcan Chen, and Qiang Hua. "Fuzzy classifier based on fuzzy support vector machine." Journal of Intelligent & Fuzzy Systems 26, No. 1 (2014), 421-430, doi: 10.3233/IFS-130819.
  21. Yang, Libo. "Fuzzy Output Support Vector Machine Based Incident Ticket Classification." IEICE Transactions on Information and Systems 104, No. 1 (2021), 146-151, doi: 1587/transinf.2020EDP7044
  22. Thakur, Arunava Kabiraj, Palash Kumar Kundu, and Arabinda Das. "Prediction of Unknown Fault of Induction Motor using SVM following Decision-Directed Acyclic Graph." Journal of The Institution of Engineers (India): Series B, 102, No. 3, (2021), 573-583, doi: 10.1007/s40031-021-00536-2
  23. Liu, Bo, Zhifeng Hao, and Eric CC Tsang. "Nesting one-against-one algorithm based on SVMs for pattern classification." IEEE Transactions on Neural Networks 19, No. 12, (2008), 2044-2052, doi: 10.1109/TNN.2008.2003298.
  24. Li, Renbing, Aihua Li, Tao Wang, and Liang Li. "Vector projection method for unclassifiable region of support vector machine." Expert Systems with Applications 38, No. 1, (2011), 856-861, doi: 10.1016/j.eswa.2010.07.046.
  25. Khemchandani, Reshma, and Suresh Chandra. "Twin support vector machines for pattern classification." IEEE Transactions on Pattern Analysis and Machine Intelligence 29, No. 5, (2007), 905-910, doi: 1109/TPAMI.2007.1068
  26. Liu, Yi-Hung, and Yen-Ting Chen. "Face recognition using total margin-based adaptive fuzzy support vector machines." IEEE Transactions on Neural Networks 18, No. 1, (2007), 178-192, doi: 1109/TNN.2006.883013.
  27. Nasiri, Jalal A., Nasrollah Moghadam Charkari, and Saeed Jalili. "Least squares twin multi-class classification support vector machine." Pattern Recognition 48, No. 3, (2015), 984-992, doi: 1016/j.patcog.2014.09.020.
  28. Gao, Zheming, Shu-Cherng Fang, Xuerui Gao, Jian Luo, and Negash Medhin. "A novel kernel-free least squares twin support vector machine for fast and accurate multi-class classification." Knowledge-Based Systems 226 (2021), 107123, doi: 1016/j.knosys.2021.107123.
  29. Xu, Yitian, Xianli Pan, Zhijian Zhou, Zhiji Yang, and Yuqun Zhang. "Structural least square twin support vector machine for classification." Applied Intelligence 42, No. 3, (2015), 527-536, doi: 1007/s10489-014-0611-4.
  30. Mir, A., and Jalal A. Nasiri. "KNN-based least squares twin support vector machine for pattern classification." Applied Intelligence 48, No. 12, (2018), 4551-4564, doi: 10.1007/s10489-018-1225-z.
  31. Nasiri, Jalal A., Nasrollah Moghadam Charkari, and Kourosh Mozafari. "Energy-based model of least squares twin support vector machines for human action recognition." Signal Processing 104 (2014), 248-257, doi: 10.1016/j.sigpro.2014.04.010.
  32. Chen, Xiaobo, Jian Yang, Qiaolin Ye, and Jun Liang. "Recursive projection twin support vector machine via within-class variance minimization." Pattern Recognition 44, No. 10-11, (2011), 2643-2655, doi: 10.1016/j.patcog.2011.03.001.
  33. Chen, Su-Gen, and Xiao-Jun Wu. "Multiple birth least squares support vector machine for multi-class classification." International Journal of Machine Learning and Cybernetics 8, No. 6, (2017), 1731-1742, doi: 10.1007/s13042-016-0554-7.
  34. Shao, Yuan-Hai, Nai-Yang Deng, Zhi-Min Yang, Wei-Jie Chen, and Zhen Wang. "Probabilistic outputs for twin support vector machines." Knowledge-Based Systems 33, (2012), 145-151, doi: 10.1016/j.knosys.2012.04.006.
  35. Bottou, Léon, Corinna Cortes, John S. Denker, Harris Drucker, Isabelle Guyon, Larry D. Jackel, Yann LeCun et al. "Comparison of classifier methods: a case study in handwritten digit recognition." In Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3-Conference C: Signal Processing (Cat. No. 94CH3440-5), Vol. 2, 77-82. IEEE, 1994.
  36. KRESSEL, Ulrich HG. "Pairwise classification and support vector machines." Advances in Kernel Methods: Support Vector Learning (2002).
  37. Liu, Li, Ling Shao, and Peter Rockett. "Human action recognition based on boosted feature selection and naive Bayes nearest-neighbor classification." Signal Processing 93, No. 6, (2013), 1521-1530, doi: 10.1016/j.sigpro.2012.07.017.
  38. Lu, Zhiwu, and Yuxin Peng. "Latent semantic learning with structured sparse representation for human action recognition." Pattern Recognition 46, No. 7, (2013), 1799-1809, doi: 10.1016/j.patcog.2012.09.027.
  39. Laptev, Ivan, Marcin Marszalek, Cordelia Schmid, and Benjamin Rozenfeld. "Learning realistic human actions from movies." In 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1-8. IEEE, 2008.
  40. Fathi, Alireza, and Greg Mori. "Action recognition by learning mid-level motion features." In 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1-8. IEEE, 2008.
  41. Jiang, Zhuolin, Zhe Lin, and Larry Davis. "Recognizing human actions by learning and matching shape-motion prototype trees." IEEE Transactions on Pattern Analysis and Machine Intelligence 34, No. 3, (2012), 533-547, doi: 10.1109/TPAMI.2011.147.
  42. Niebles, Juan Carlos, Hongcheng Wang, and Li Fei-Fei. "Unsupervised learning of human action categories using spatial-temporal words." International Journal of Computer Vision 79, No. 3, (2008), 299-318, doi: 10.1007/s11263-007-0122-4.
  43. Liu, Jingen, Saad Ali, and Mubarak Shah. "Recognizing human actions using multiple features." In 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1-8. IEEE, 2008.
  44. Bregonzio, Matteo, Shaogang Gong, and Tao Xiang. "Recognising action as clouds of space-time interest points." In 2009 IEEE conference on computer vision and pattern recognition, 1948-1955. IEEE, 2009.
  45. Wang, Heng, Muhammad Muneeb Ullah, Alexander Klaser, Ivan Laptev, and Cordelia Schmid. "Evaluation of local spatio-temporal features for action recognition." In Bmvc 2009-british machine vision conference, pp. 124-1. BMVA Press, 2009.
  46. Chou, Kuang-Pen, Mukesh Prasad, Di Wu, Nabin Sharma, Dong-Lin Li, Yu-Feng Lin, Michael Blumenstein, Wen-Chieh Lin, and Chin-Teng Lin. "Robust feature-based automated multi-view human action recognition system." IEEE Access 6, (2018), 15283-15296, doi: 10.1109/ACCESS.2018.2809552.
  47. Goudelis, Georgios, Konstantinos Karpouzis, and Stefanos Kollias. "Exploring trace transform for robust human action recognition." Pattern Recognition 46, No. 12, (2013), 3238-3248, doi: 10.1016/j.patcog.2013.06.006.
  48. Arunnehru, J., G. Chamundeeswari, and S. Prasanna Bharathi. "Human action recognition using 3D convolutional neural networks with 3D motion cuboids in surveillance videos." Procedia Computer Science 133 (2018), 471-477, doi: 10.1016/j.procs.2018.07.059.
  49. Singh, Tej, and Dinesh Kumar Vishwakarma. "A hybrid framework for action recognition in low-quality video sequences." arXiv preprint arXiv:1903.04090 (2019),
  50. Aslan, Muhammet Fatih, Akif Durdu, and Kadir Sabanci. "Human action recognition with bag of visual words using different machine learning methods and hyperparameter optimization." Neural Computing and Applications 32, No. 12, (2020), 8585-8597, doi: 10.1007/s00521-019-04365-9.
  51. Vishwakarma, Dinesh Kumar, and Chhavi Dhiman. "A unified model for human activity recognition using spatial distribution of gradients and difference of Gaussian kernel." The Visual Computer 35, No. 11, (2019), 1595-1613, doi: 10.1007/s00371-018-1560-4.
  52. Vishwakarma, Dinesh Kumar. "A two-fold transformation model for human action recognition using decisive pose." Cognitive Systems Research 61, (2020), 1-13, doi: 10.1016/j.cogsys.2019.12.004.
  53. Ramya, P., and Rajendran Rajeswari. "Human action recognition using distance transform and entropy-based features." Multimedia Tools and Applications 80, No. 6, (2021), 8147-8173, doi: 10.1007/s11042-020-10140-z.
  54. Schuldt, Christian, Ivan Laptev, and Barbara Caputo. "Recognizing human actions: a local SVM approach." In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., Vol. 3, 32-36. IEEE, 2004.
  55. Dollár, Piotr, Vincent Rabaud, Garrison Cottrell, and Serge Belongie. "Behavior recognition via sparse spatio-temporal features." In 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 65-72. IEEE, 2005.
  56. Wong, Shu-Fai, Tae-Kyun Kim, and Roberto Cipolla. "Learning motion categories using both semantic and structural information." In 2007 IEEE Conference on Computer Vision and Pattern Recognition, 1-6. IEEE, 2007.
  57. Jhuang, Hueihan, Thomas Serre, Lior Wolf, and Tomaso Poggio. "A biologically inspired system for action recognition." In 2007 IEEE 11th international conference on computer vision, 1-8. Ieee, 2007.
  58. Klaser, Alexander, Marcin Marszałek, and Cordelia Schmid. "A spatio-temporal descriptor based on 3d-gradients." In BMVC 2008-19th British Machine Vision Conference, 275-301. British Machine Vision Association, 2008.
  59. Liu, Jingen, and Mubarak Shah. "Learning human actions via information maximization." In 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1-8. IEEE, 2008.
  60. Kovashka, Adriana, and Kristen Grauman. "Learning a hierarchy of discriminative space-time neighborhood features for human

 

 

 

action recognition." In 2010 IEEE computer society conference on computer vision and pattern recognition, 2046-2053. IEEE, 2010.

  1. Shao, Ling, Ruoyun Gao, Yan Liu, and Hui Zhang. "Transform based spatio-temporal descriptors for human action recognition." Neurocomputing 74, No. 6, (2011), 962-973, doi: 10.1016/j.neucom.2010.11.013.
  2. Ghodrati, Amir, and Shohreh Kasaei. "Human action categorization using discriminative local spatio-temporal feature weighting." Intelligent Data Analysis 16, No. 4, (2012), 537-550, doi: 10.3233/IDA-2012-0538.
  3. An, Feng-Ping. "Human action recognition algorithm based on adaptive initialization of deep learning model parameters and support vector machine." IEEE Access 6, (2018), 59405-59421, doi: 10.1109/ACCESS.2018.2874022.
  4. Soomro, Khurram, Amir Roshan Zamir, and Mubarak Shah. "UCF101: A dataset of 101 human actions classes from videos in the wild." arXiv preprint arXiv:1212.0402 (2012).
  5. Karpathy, Andrej, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. "Large-scale video classification with convolutional neural networks." In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 1725-1732. 2014.
  6. Hou, Rui, Amir Roshan Zamir, Rahul Sukthankar, and Mubarak Shah. "Damn–discriminative and mutually nearest: Exploiting pairwise category proximity for video action recognition." In European Conference on Computer Vision, 721-736. Springer, Cham, 2014.
  7. Boyraz12, Hakan, Syed Zain Masood13, Baoyuan Liu, Marshall Tappen12, and Hassan Foroosh. "Action recognition by weakly-supervised discriminative region localization." (2014).
  8. Kihl, Olivier, David Picard, and Philippe-Henri Gosselin. "A unified framework for local visual descriptors evaluation." Pattern Recognition 48, No. 4, (2015), 1174-1184, doi: 10.1016/j.patcog.2014.11.013.
  9. Peng, Xiaojiang, and Cordelia Schmid. "Multi-region two-stream R-CNN for action detection." In European conference on computer vision, pp. 744-759. Springer, Cham, 2016.
  10. Chang, Xiaojun, Yao-Liang Yu, and Yi Yang. "Robust top-k multiclass SVM for visual category recognition." In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 75-83. 2017.
  11. Hua, Michelle, Mingqi Gao, and Zichun Zhong. "SCN: Dilated silhouette convolutional network for video action recognition." Computer Aided Geometric Design 85, (2021), 101965, doi: 10.1016/j.cagd.2021.101965.
  12. dos S Silva, Francisco H., Gabriel M. Bezerra, Gabriel B. Holanda, J. Wellington M. de Souza, Paulo AL Rego, Aloísio V. Lira Neto, Victor Hugo C. de Albuquerque, and Pedro P. Rebouças Filho. "A novel feature extractor for human action recognition in visual question answering." Pattern Recognition Letters 147, (2021), 41-47, doi: 10.1016/j.patrec.2021.04.002.
  13. Leyva, Roberto, Victor Sanchez, and Chang-Tsun Li. "Compact and low-complexity binary feature descriptor and Fisher vectors for video analytics." IEEE Transactions on Image Processing 28, No. 12 (2019): 6169-6184, doi: 10.1109/TIP.2019.2922826.
  14. Sahoo, Suraj Prakash, and Samit Ari. "On an algorithm for human action recognition." Expert Systems with Applications 115, (2019), 524-534, doi: 10.1016/j.eswa.2018.08.014
International Journal of Engineering
Volume 35, Issue 1
TRANSACTIONS A: Basics
January 2022
Pages 1-13

Files

Cited by

Share

How to cite

Statistics