OPTIMAL GENETIC ALGORITHM SELECTION FOR DEEP NEURAL NETWORK SETTINGS

Authors

  • O. I. Chumachenko National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”
  • M. O. Liubachenko National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.66.15222

Keywords:

Multiobjective genetic algorithm, deep neural networks, structural-parametric synthesis, optimal selection

Abstract

The problem of construction of deep neural networks with the use of genetic algorithms is considered. The problem of structural-parametric synthesis with creation of neural networks is defined. The main purpose of the study is to find a deep neural network that is optimal for solving urgent problems. The classification problem is chosen as the urgent problem to solve. Also the classification of genetic algorithms is given, which are used as a basis for establishing the parameters of deep neural networks A system for the optimal tuning of the parameters of deep neural networks is proposed, which includes a two-stage algorithm. At the first stage of the algorithm, a multicriteria genetic algorithm is selected from a set of possible ones (genetic algorithm of vector estimation, genetic algorithm of Fonseca and Fleming, genetic algorithm of Pareto approximation with niche, genetic algorithm of sorting without dominance, genetic algorithm of Pareto force, genetic algorithm of Pareto-2 force ) that best fits the given training sample. At the second step, the problem of structural-parametric synthesis of a neural network is solved according to the criteria of accuracy and complexity. As a result of training, the values of the neural network parameters are found, such as: the number of layers, the number of neurons in each layer, the values of the weight coefficients.The modeling of the proposed system is carried out. The results of modeling, comparison of results with similar software packages are presented. The obtained results show the possibilities of wide use.

Author Biographies

O. I. Chumachenko, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Doctor of Engineering Science. Professor

orcid.org/0000-0003-3006-7460

M. O. Liubachenko, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Bachelor

References

Ghosh Ashish & Dehuri Satchidananda, "Evolutionary Algorithms for Multi-Criterion Optimization: A Survey," International Journal of Computing & Information Sciences, 2, 2004.

C. A. C. Coello, "Evolutionary multi-objective optimization: a historical view of the field," Comput. Intell. Mag. IEEE 1 (1), 28–36, 2006. https://doi.org/10.1109/MCI.2006.1597059

K. Deb, "Multi-Objective Optimization Using Evolutionary Algorithms," vol. 16, John Wiley & Sons, 2001.

A. Zhou, B.-Y. Qu, H. Li, S.-Z. Zhao, P. N. Suganthan, and Q. Zhang, "Multi-objective evolutionary algorithms: a survey of the state of the art," Swarm Evol. Comput, 1 (1), 32–49, 2011. https://doi.org/10.1016/j.swevo.2011.03.001.

B. Li, J. Li, K. Tang, and X. Yao, "Many-objective evolutionary algorithms: a survey," ACM Comput. Surv. 48 (1), pp. 1–35, 2015. https://doi.org/10.1145/2792984

H. Ishibuchi, N. Tsukamoto, and Y. Nojima, "Evolutionary many-objective optimization: a short review," Proceedings of the IEEE Congress on Evolutionary Computation, 2008, pp. 2419–2426. https://doi.org/10.1109/CEC.2008.4631121

M. Farina, and P. Amato, "A fuzzy definition of “optimality” for many-criteria optimization problems," IEEE Trans. Syst. Man Cybern, Part A: Syst. Hum., 34 (3), 2004, pp. 315–326. https://doi.org/10.1007/s40747-019-0113-4

Mario Köppen, Raul Vicente-Garcia, and Bertram Nickolay, "Fuzzy-Pareto-dominance and its application in evolutionary multi-objective optimization," in: Proceedings of the Evolutionary Multi-criterion Optimization, Springer, 2005, pp. 399–412. https://doi.org/10.1007/978-3-540-31880-4_28

S. Yang, M. Li, X. Liu, and J. Zheng, "A grid-based evolutionary algorithm for many-objective optimization," IEEE Trans. Evol. Comput. 17 (5), 2013, pp. 721–736. https://doi.org/10.1109/TEVC.2012.2227145.

R. Wang, R. C. Purshouse, and P. J. Fleming, "Preference-inspired coevolutionary algorithm for many-objective optimization," IEEE Trans. Evol. Comput. 17 (4), 2013, pp. 474–494. https://doi.org/10.1109/TEVC.2012.2204264.

M. Li, S. Yang, and X. Liu, "Shift-based density estimation for Pareto-based algorithm in many-objective optimization," IEEE Trans. Evol. Comput., 18 (3), 2014, pp. 348–365. https://doi.org/10.1109/TEVC.2013.2262178

K. Tan, T. Lee, & E. Khor, "Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons," Artificial Intelligence Review 17, pp. 251–290, 2002. https://doi.org/10.1023/A:1015516501242

kaggle.com/c/mnist-classification

https://github.com/EpistasisLab/tpot

Downloads

Issue

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES