Linear Algebraic Systems Neural Network Solution. Part 1

Authors

  • Volodymyr Mamonov National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
  • Yevhen Shatikhin National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”
  • Yurii Tymoshenko National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute" https://orcid.org/0000-0001-7812-6437

DOI:

https://doi.org/10.18372/1990-5548.75.17542

Keywords:

linear algebraic systems, condition number, ill-conditioned systems, neural network, gradient descent, TensorFlow

Abstract

In recent years, neural networks have become increasingly popular due to their versatility in solving complex problems. One area of interest is their application in solving linear algebraic systems, especially those that are ill-conditioned. The solutions of such systems are highly sensitive to small changes in their coefficients, leading to unstable solutions. Therefore, solving these types of systems can be challenging and require specialized techniques. This article explores the use of neural network methodologies for solving linear algebraic systems, focusing on ill-conditioned systems. The primary goal is to develop a model capable of directly solving linear equations and to evaluate its performance on a range of linear equation sets, including ill-conditioned systems. To tackle this problem, neural network implementing iterative algorithm was built. Error function of linear algebraic system is minimized using stochastic gradient descent. This model doesn’t require extensive training other than tweaking learning rate for particularly large systems. The analysis shows that the suggested model can handle well-conditioned systems of varying sizes, although for systems with large coefficients some normalization is required. Improvements are necessary for effectively solving ill-conditioned systems, since researched algorithm is shown to be not numerically stable. This research contributes to the understanding and application of neural network techniques for solving linear algebraic systems. It provides a foundation for future advances in this field and opens up new possibilities for solving complex problems. With further research and development, neural network models can become a powerful tool for solving ill-conditioned linear systems and other related problems.

Author Biographies

Volodymyr Mamonov , National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Bachelor's degree student.

Department of Artificial Intelligence

Institute of Applied Systems Analysis 

Yevhen Shatikhin , National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”

Bachelor's degree student

Department of Artificial Intelligence

Institute of Applied Systems Analysis

Yurii Tymoshenko , National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Candidate of Sciences (Engineering). Associated Professor

Institute of Applied Systems Analysis

References

G. E. Forsythe, M. A. Malcolm, & C. B. Moler, Computer Solution of Linear Algebraic Systems. New York: Prentice-Hall, Inc., 1967, 259 р. https://doi.org/10.1002/zamm.19790590235

Hilbert, David, “Ein Beitrag zur Theorie des Legendre'schen Polynoms“, (1894) Acta Mathematica, 18: 155–159, https://doi.org/10.1007/BF02418278

M. Zgurovsky, V. Sineglazov and E. Chumachenko, Artificial Intelligence Systems Based on Hybrid Neural Networks. Theory and Applications, Springer Cham, 2020, 512 p. https://doi.org/10.1007/978-3-030-48453-8

A. Cichocki and R. Unbehauen, “Neural networks for solving systems of linear equations and related problems,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. vol. 39, no. 2, 1992, рp. 124–138. https://doi.org/10.1109/81.167018

C. Darken, J. Chang, and J. Moody, “Learning rate schedules for faster stochastic gradient search,”. Neural Networks for Signal Processing, Proceedings of the 1992 IEEE Workshop, vol. 2, Sep 1992, pp. 1–11. https://doi.org/10.1109/NNSP.1992.253713

TensorFlow documentation. https://www.tensorflow.org/api_docs/python/tf

Keras documentation. https://keras.io/api/

NumPy documentation. https://numpy.org/doc/stable/reference/index.html#reference

Matplotlib documentation. https://matplotlib.org/stable/api/index.html

Downloads

Published

2023-03-26

Issue

Section

COMPUTER SCIENCES AND INFORMATION TECHNOLOGIES