Semi-supervised Support Vector Machine

Authors

  • Victor Sineglazov National Aviation University, Kyiv, Ukraine https://orcid.org/0000-0002-3297-9060
  • Andriy Samoshyn National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.75.17553

Keywords:

support vector machine, semi-supervised learning, multi-class classification, multicriteria, method of global optimization

Abstract

The article considers a new approach to constructing a support vector machine with semi-supervised learning for solving a classification problem. It is assumed that the distributions of the classes may overlap. The cost function has been modified by adding elements of a penalty to it for labels not in their class. The penalty is represented as a linear function of the distance between the label and the class boundary. To overcome the problem of multicriteria, a global optimization method known as continuation is proposed. For a combination of predictions, it is suggested to use the voting method of models with different kernels. The Optuna framework was chosen as the tool for configuring hyperparameters. The following were considered as training samples: type_dataset, banana, banana_inverse, c_circles, two_moons_classic, two_moons_tight, two_moons_wide.

Author Biographies

Victor Sineglazov , National Aviation University, Kyiv, Ukraine

Doctor of Engineering Science. Professor. Head of the Department

Aviation Computer-Integrated Complexes Department

Faculty of Air Navigation Electronics and Telecommunications

Andriy Samoshyn , National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Bachelor

Educational and Scientific Institute of Applied System Analysis

References

C. M. Bishop, Pattern recognition and machine learning. 2006. Berlin: Springer.

N. Cristianini and J. Shawe-Taylor, An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press, 2000, Cambridge. https://doi.org/10.1017/CBO9780511801389

A. Gammerman, V. Vapnik, and V. Vovk, “Learning by transduction,” In Uncertainty in Artificial Intelligence, pp. 148–155, 1998.

V. N. Vapnik, Statistical learning theory. New York: John Wiley & Sons, Inc.

K. P. Bennett, A. Demiriz, and J. Shawe–Taylor, “A Column Generation Algorithm for Boosting,” (http://www.recognition.mccme.ru/pub/papers/boosting/bennett00column.pdf). In Pat Langley, editor, Proceedings of Seventeenth International Conference on Machine Learning, pp. 65–72. Morgan Kaufmann, 2000.

T. Joachims, “Transductive inference for text classification using support vector machines,” In ICML, 1999.

O. Chapelle, and A. Zien, “Semi-supervised learning by low density separation,” In AISTATS, pp. 57–64, 2005. https://doi.org/10.7551/mitpress/9780262033589.001.0001

F. Wang, & C. Zhang, “Label propagation through linear neighborhoods,” IEEE Transactions on Knowledge and Data Engineering, 20(1), 2008, pp. 55–67. https://doi.org/10.1109/TKDE.2007.190672

N. Kasabov and S. Pang, “Transductive support vector machines and applications in bioinformatics for promoter recognition,” In ICNNSP, 2004, pp. 1–6. https://doi.org/10.1109/ICNNSP.2003.1279199

C. Goutte, H. Deґjean, E. Gaussier, N. Cancedda, and J.M. Renders, “Combining labelled and unlabelled data: A case study on fisher kernels and transductive inference for biological entity recognition,” In CoNLL, 2002, pp. 1–7. https://doi.org/10.3115/1118853.1118864

T. Zhang and F. J. Oles, “A probability analysis on the value of unlabeled data for classification problems,” In ICML 00, pp. 1191–1198, 2000.

O. Chapelle, B. Schölkopf, and A. Zien, (eds.). Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006b.

O. Chapelle, V. Sindhwani, and S. S. Keerthi, “Optimization techniques for semi-supervised support vector machines,” J. Mach. Learn. Res., 9: 203–233, 2008.

O. Chapelle, M. Chi, & A. Zien, “A continuation method for semi-supervised SVMsm,” In Proceedings of the 23rd international conference on machine learning, 2006a, pp. 185–192. https://doi.org/10.1145/1143844.1143868

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama, Optuna: A Next-generation Hyperparameter Optimization Framework, 2019, https://doi.org/10.48550/arXiv.1907.10902

Victor Chukwudi Osamor & Adaugo Fiona Okezie, “Enhancing the weighted voting ensemble algorithm for tuberculosis predictive diagnosis,” Scientific Reports, vol. 11, Article number: 14806, 2021, 4922, Accesses 19. https://doi.org/10.1038/s41598-021-94347-6

Downloads

Published

2023-03-26

Issue

Vol. 1 No. 75 (2023)

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES

License

Authors who publish with this journal agree to the following terms:

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).