Structural-parametric Synthesis of Capsule Neural Networks

Authors

  • Victor Sineglazov National Aviation University, Kyiv, Ukraine https://orcid.org/0000-0002-3297-9060
  • Denys Kudriev National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.78.18261

Keywords:

сapsule neural network, structural and parametric synthesis, genetic algorithm, adaptive estimation of moment (Adam), classification problem

Abstract

This work is dedicated to the structural-parametric synthesis of capsule neural networks. A methodology for structural-parametric synthesis of capsule neural networks has been developed, which includes the following algorithms: determining the most influential parameters of the capsule neural network, a hybrid machine learning algorithm. Using the hybrid algorithm, the optimal structure and values of weight coefficients are determined. The hybrid algorithm consists of a genetic algorithm and a gradient algorithm (Adam). 150 topologies of capsule neural networks were evaluated, with an average evaluation time of one generation taking 10 hours. Chromosomes and weights are stored in the generation folder. The chromosome storage format is JSON, using the jsonpickle library for writing. Also, when forming a new generation, chromosome files from previous generations are used as a "cache". If a chromosome of the same structure exists, the accuracy is assigned immediately to avoid unnecessary training of neural networks. As a result of using the hybrid algorithm, the optimal topology and parameters of the capsule neural network for classification tasks have been found.

Author Biographies

Victor Sineglazov , National Aviation University, Kyiv, Ukraine

Doctor of Engineering Science

Professor

Head of the Department Aviation Computer-Integrated Complexes

Faculty of Air Navigation Electronics and Telecommunications

Denys Kudriev , National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Graduate Student

Department of Artificial Intelligence

Institute of Applied System Analysis

References

G. Hinton, А. Krizhevsky, and S. Wang, “Transforming Auto-Encoders,” Artificial Neural Networks and Machine Learning: ICANN 2011, 21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proc., Part I. 44–51. https://doi.org/10.1007/978-3-642-21735-7_6.

S. Sabour, N. Frosst, and G. E. Hinton, Dynamic Routing Between Capsules. arXiv:1710.09829. https://doi.org/10.48550/arXiv.1710.09829

Edgar Xi, Selina Bing, and Yang Jin, Capsule network performance on complex data. arXiv: 1712.03480. https://doi.org/10.48550/arXiv.1712.03480

Dilin Wang and Qiang Liu, An optimization view on dynamic routing between capsules, 2018. URL: https://openreview.net/forum?id=HJjtFYJDf

Jan Eric Lenssen, Matthias Fey, and Pascal Libuschewski, Group equivariant capsule networks. arXiv: 1806.05086. https://doi.org/10.48550/arXiv.1806.05086

Geoffrey E Hinton, Sara Sabour, and Nicholas Frosst, Matrix capsules with EM routing, 2018. URL: https://openreview.net/pdf?id=HJWLfGWRb

Mohammad Taha Bahadori, Spectral capsule networks, 2018. URL: https://openreview.net/pdf?id=HJuMvYPaM

Fabio De Sousa Ribeiro1, Georgios Leontidis, and Stefanos D Kollias, Capsule routing via variational bayes. arXiv: 1905.11455. https://doi.org/10.48550/arXiv.1905.11455

Jindong Gu and Volker Tresp, Improving the robustness of capsule networks to image affine transformations. arXiv: 1911.07968. https://doi.org/10.48550/arXiv.1911.07968

Inyoung Paik, Taeyeong Kwak, and Injung Kim, Capsule networks need an improved routing algorithm. arXiv: 1907.13327. https://doi.org/10.48550/arXiv.1907.13327

Sai Raam Venkatraman, Ankit Anand, S Balasubramanian, and R Raghunatha Sarma, Learning compositional structures for deep learning: Why routing-by-agreement is necessary. arXiv: 2010.01488. https://doi.org/10.48550/arXiv.2010.01488

Adam Byerly, Tatiana Kalganova, and Ian Dear, No Routing Needed Between Capsules. arXiv: 2001.09136. https://doi.org/10.48550/arXiv.2001.09136

Jaewoong Choi, Hyun Seo, Suii Im, and Myungjoo Kang, Attention routing between capsules. arXiv: 1907.01750. https://doi.org/10.48550/arXiv.1907.01750

Yao-Hung Hubert Tsai, Nitish Srivastava, Hanlin Goh, and Ruslan Salakhutdinov. Capsules with inverted dot-product attention routing. arXiv: 2002.04764. https://doi.org/10.48550/arXiv.2002.04764

Dunlu Peng, Dongdong Zhang, Cong Liu, and Jing Lu, “Bg-sac: Entity relationship classification model based on self-attention supported capsule networks,” Appl. Sof Comput. 91, 106186, 2020. https://doi.org/10.1016/j.asoc.2020.106186

V. Mazzia, F. Salvetti, & M. Chiaberge, “Efficient-CapsNet: capsule network with self-attention routing,” Sci Rep 11, Article number 14634, 2021. https://doi.org/10.1038/s41598-021-93977-0.

Downloads

Published

2023-12-27

Issue

Vol. 4 No. 78 (2023)

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES

License

Authors who publish with this journal agree to the following terms:

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).