OPTIMAL GENETIC ALGORITHM SELECTION FOR DEEP NEURAL NETWORK SETTINGS

Authors

  • O. I. Chumachenko National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”
  • M. O. Liubachenko National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.66.15222

Keywords:

Multiobjective genetic algorithm, deep neural networks, structural-parametric synthesis, optimal selection

Abstract

The problem of construction of deep neural networks with the use of genetic algorithms is considered. The problem of structural-parametric synthesis with creation of neural networks is defined. The main purpose of the study is to find a deep neural network that is optimal for solving urgent problems. The classification problem is chosen as the urgent problem to solve. Also the classification of genetic algorithms is given, which are used as a basis for establishing the parameters of deep neural networks A system for the optimal tuning of the parameters of deep neural networks is proposed, which includes a two-stage algorithm. At the first stage of the algorithm, a multicriteria genetic algorithm is selected from a set of possible ones (genetic algorithm of vector estimation, genetic algorithm of Fonseca and Fleming, genetic algorithm of Pareto approximation with niche, genetic algorithm of sorting without dominance, genetic algorithm of Pareto force, genetic algorithm of Pareto-2 force ) that best fits the given training sample. At the second step, the problem of structural-parametric synthesis of a neural network is solved according to the criteria of accuracy and complexity. As a result of training, the values of the neural network parameters are found, such as: the number of layers, the number of neurons in each layer, the values of the weight coefficients.The modeling of the proposed system is carried out. The results of modeling, comparison of results with similar software packages are presented. The obtained results show the possibilities of wide use.

Author Biographies

O. I. Chumachenko, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Doctor of Engineering Science. Professor

orcid.org/0000-0003-3006-7460

M. O. Liubachenko, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Bachelor

References

Ghosh Ashish & Dehuri Satchidananda, "Evolutionary Algorithms for Multi-Criterion Optimization: A Survey," International Journal of Computing & Information Sciences, 2, 2004.

C. A. C. Coello, "Evolutionary multi-objective optimization: a historical view of the field," Comput. Intell. Mag. IEEE 1 (1), 28–36, 2006. https://doi.org/10.1109/MCI.2006.1597059

K. Deb, "Multi-Objective Optimization Using Evolutionary Algorithms," vol. 16, John Wiley & Sons, 2001.

A. Zhou, B.-Y. Qu, H. Li, S.-Z. Zhao, P. N. Suganthan, and Q. Zhang, "Multi-objective evolutionary algorithms: a survey of the state of the art," Swarm Evol. Comput, 1 (1), 32–49, 2011. https://doi.org/10.1016/j.swevo.2011.03.001.

B. Li, J. Li, K. Tang, and X. Yao, "Many-objective evolutionary algorithms: a survey," ACM Comput. Surv. 48 (1), pp. 1–35, 2015. https://doi.org/10.1145/2792984

H. Ishibuchi, N. Tsukamoto, and Y. Nojima, "Evolutionary many-objective optimization: a short review," Proceedings of the IEEE Congress on Evolutionary Computation, 2008, pp. 2419–2426. https://doi.org/10.1109/CEC.2008.4631121

M. Farina, and P. Amato, "A fuzzy definition of “optimality” for many-criteria optimization problems," IEEE Trans. Syst. Man Cybern, Part A: Syst. Hum., 34 (3), 2004, pp. 315–326. https://doi.org/10.1007/s40747-019-0113-4

Mario Köppen, Raul Vicente-Garcia, and Bertram Nickolay, "Fuzzy-Pareto-dominance and its application in evolutionary multi-objective optimization," in: Proceedings of the Evolutionary Multi-criterion Optimization, Springer, 2005, pp. 399–412. https://doi.org/10.1007/978-3-540-31880-4_28

S. Yang, M. Li, X. Liu, and J. Zheng, "A grid-based evolutionary algorithm for many-objective optimization," IEEE Trans. Evol. Comput. 17 (5), 2013, pp. 721–736. https://doi.org/10.1109/TEVC.2012.2227145.

R. Wang, R. C. Purshouse, and P. J. Fleming, "Preference-inspired coevolutionary algorithm for many-objective optimization," IEEE Trans. Evol. Comput. 17 (4), 2013, pp. 474–494. https://doi.org/10.1109/TEVC.2012.2204264.

M. Li, S. Yang, and X. Liu, "Shift-based density estimation for Pareto-based algorithm in many-objective optimization," IEEE Trans. Evol. Comput., 18 (3), 2014, pp. 348–365. https://doi.org/10.1109/TEVC.2013.2262178

K. Tan, T. Lee, & E. Khor, "Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons," Artificial Intelligence Review 17, pp. 251–290, 2002. https://doi.org/10.1023/A:1015516501242

kaggle.com/c/mnist-classification

https://github.com/EpistasisLab/tpot

Downloads

Issue

Vol. 4 No. 66 (2020)

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES

License

Authors who publish with this journal agree to the following terms:

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).