TWO-LEVEL SYSTEM FOR TUNING PARAMETERS OF ARTIFICIAL NEURAL NETWORKS

Authors

  • O. I. Chumachenko National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”
  • S. V. Shymkov National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”
  • A. T. Kot National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

DOI:

https://doi.org/10.18372/1990-5548.63.14517

Keywords:

Neural networks, parametric tuning, training, optimization, genetic algorithms

Abstract

This paper focuses on the process of adjusting weights and biases of feed-forward ANN during their training process. A new algorithm for tuning artificial neural networks parameters has been proposed to overcome some limitations of existing optimization algorithms and to improve the training process of neural networks. This proposed algorithm combines the benefits of genetic algorithm and gradient-based optimization algorithms to improve the speed of training artificial neural networks and to increase the prediction accuracy of resulting network. The results of artificial neural networks training for classification task using two-level algorithm are presented and compared in performance with various gradient-based optimization algorithms.

Author Biographies

O. I. Chumachenko, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Doctor of Engineering Science. Associate Professor

orcid.org/0000-0003-3006-7460

S. V. Shymkov, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Bachelor

A. T. Kot, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Technical Cybernetic Department

Post-graduate student

References

Kurt Hornik, Maxwell Stinchcombe, Halbert White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, Issue 5, pp. 359–366, 1989. Print. doi:10.1016/0893-6080(89)90020-8

Simon Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed., Upper Saddle River, NJ: Prentice Hall PTR, 1998. Print; ISBN: 0132733501

Nick McClure, TensorFlow Machine Learning Cookbook: Over 60 recipes to build intelligent machine learning systems with the power of Python, 2nd ed., Birmingham, UK: Packt Publishing, 2018. Print; ISBN: 1789131685

David Kriesel, A Brief Introduction to Neural Networks [Online]. Available: http://www.dkriesel.com/en/science/neural_network

Yurii Nesterov, “A method for unconstrained convex minimization problem with the rate of convergence o(1/k2),” Soviet. Math. Docl., vol. 269, pp. 543–547, 1983. Print.

Kurt Hornik, Maxwell Stinchcombe, Halbert White, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,” The Journal of Machine Learning Research, vol. 12, pp. 2121–2159, 2011. Print.

Matthew D. Zeiler, ADADELTA: An Adaptive Learning Rate Method, 2012 [Online]. Available: https://arxiv.org/abs/1212.5701

Diederik P. Kingma and Jimmy Ba, “Adam: A Method for Stochastic Optimization,” Presented at at the 3rd International Conference for Learning Representations, San Diego, 2015 [Online]

Available: https://arxiv.org/abs/1412.6980

David E. Goldberg, Genetic Algorithms in Search, Optimization & Machine Learning, Boston, MA: Addison-Wesley Longman Publishing Co., 1989. Print; ISBN: 0201157675

Dan Simon, Evolutionary Optimization Algorithms, Hoboken, NJ: John Wiley & Sons, Inc., 2013. Print; ISBN: 0470937416

Eckart Zitzler, Marco Laumanns, and Lothar Thiele (May 2001), SPEA2: Improving the Strength Pareto Evolutionary Algorithm, Swiss Federal Institute of Technology, Zurich, Switzerland [Online] Available: https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/145755/eth-24689-01.pdf, doi: 10.3929/ethz-a-004284029

Downloads

Issue

Vol. 1 No. 63 (2020)

Section

COMPUTER ENGINEERING

License

Authors who publish with this journal agree to the following terms:

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).