Semi-supervised Support Vector Machine

Authors

  • Victor Sineglazov National Aviation University, Kyiv, Ukraine https://orcid.org/0000-0002-3297-9060
  • Andriy Samoshyn National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.75.17553

Keywords:

support vector machine, semi-supervised learning, multi-class classification, multicriteria, method of global optimization

Abstract

The article considers a new approach to constructing a support vector machine with semi-supervised learning for solving a classification problem. It is assumed that the distributions of the classes may overlap. The cost function has been modified by adding elements of a penalty to it for labels not in their class. The penalty is represented as a linear function of the distance between the label and the class boundary. To overcome the problem of multicriteria, a global optimization method known as continuation is proposed. For a combination of predictions, it is suggested to use the voting method of models with different kernels. The Optuna framework was chosen as the tool for configuring hyperparameters. The following were considered as training samples: type_dataset, banana, banana_inverse, c_circles, two_moons_classic, two_moons_tight, two_moons_wide.

Author Biographies

Victor Sineglazov , National Aviation University, Kyiv, Ukraine

Doctor of Engineering Science. Professor. Head of the Department

Aviation Computer-Integrated Complexes Department

Faculty of Air Navigation Electronics and Telecommunications

Andriy Samoshyn , National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Bachelor

Educational and Scientific Institute of Applied System Analysis

References

C. M. Bishop, Pattern recognition and machine learning. 2006. Berlin: Springer.

N. Cristianini and J. Shawe-Taylor, An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press, 2000, Cambridge. https://doi.org/10.1017/CBO9780511801389

A. Gammerman, V. Vapnik, and V. Vovk, “Learning by transduction,” In Uncertainty in Artificial Intelligence, pp. 148–155, 1998.

V. N. Vapnik, Statistical learning theory. New York: John Wiley & Sons, Inc.

K. P. Bennett, A. Demiriz, and J. Shawe–Taylor, “A Column Generation Algorithm for Boosting,” (http://www.recognition.mccme.ru/pub/papers/boosting/bennett00column.pdf). In Pat Langley, editor, Proceedings of Seventeenth International Conference on Machine Learning, pp. 65–72. Morgan Kaufmann, 2000.

T. Joachims, “Transductive inference for text classification using support vector machines,” In ICML, 1999.

O. Chapelle, and A. Zien, “Semi-supervised learning by low density separation,” In AISTATS, pp. 57–64, 2005. https://doi.org/10.7551/mitpress/9780262033589.001.0001

F. Wang, & C. Zhang, “Label propagation through linear neighborhoods,” IEEE Transactions on Knowledge and Data Engineering, 20(1), 2008, pp. 55–67. https://doi.org/10.1109/TKDE.2007.190672

N. Kasabov and S. Pang, “Transductive support vector machines and applications in bioinformatics for promoter recognition,” In ICNNSP, 2004, pp. 1–6. https://doi.org/10.1109/ICNNSP.2003.1279199

C. Goutte, H. Deґjean, E. Gaussier, N. Cancedda, and J.M. Renders, “Combining labelled and unlabelled data: A case study on fisher kernels and transductive inference for biological entity recognition,” In CoNLL, 2002, pp. 1–7. https://doi.org/10.3115/1118853.1118864

T. Zhang and F. J. Oles, “A probability analysis on the value of unlabeled data for classification problems,” In ICML 00, pp. 1191–1198, 2000.

O. Chapelle, B. Schölkopf, and A. Zien, (eds.). Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006b.

O. Chapelle, V. Sindhwani, and S. S. Keerthi, “Optimization techniques for semi-supervised support vector machines,” J. Mach. Learn. Res., 9: 203–233, 2008.

O. Chapelle, M. Chi, & A. Zien, “A continuation method for semi-supervised SVMsm,” In Proceedings of the 23rd international conference on machine learning, 2006a, pp. 185–192. https://doi.org/10.1145/1143844.1143868

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama, Optuna: A Next-generation Hyperparameter Optimization Framework, 2019, https://doi.org/10.48550/arXiv.1907.10902

Victor Chukwudi Osamor & Adaugo Fiona Okezie, “Enhancing the weighted voting ensemble algorithm for tuberculosis predictive diagnosis,” Scientific Reports, vol. 11, Article number: 14806, 2021, 4922, Accesses 19. https://doi.org/10.1038/s41598-021-94347-6

Downloads

Published

2023-03-26

Issue

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES