ANALYSIS OF PECULIARITIES OF THE EM-ALGORITHM IN THE IMPLEMENTATION OF CLUSTERING OF SIGNAL CONSTRUCTIONS
DOI:
https://doi.org/10.18372/2310-5461.42.13758Keywords:
cluster analysis, expectation-maximization algorithm, mathematical singularity, Gaussian mixture model, machine learning, data processingAbstract
The expectation-maximization (EM) algorithm is a well-known statistical method, which is used in the field of data and signal processing for cluster analysis, parameter estimation and other machine learning techniques. The EM-algorithm is characterized by some specific peculiarities, e.g., a sensitivity to initial parameters. The article aims to analyze peculiarities of the EM-algorithm, which arise due to the detection of empty clusters in solving the problem of clustering of signal constructions. These peculiarities boil down to the formation of uncertainties (mathematical singularities) in the log-likelihood function, the maximization of which is performed during the iterative procedure of the EM-algorithm. An example of a possible clustering problem in the field of signal analysis is shown. The example uses an approach based on an analysis of correlations between signals using the Gaussian mixture model for these correlations and an estimation of the Gaussian mixture model parameters and hidden variables (the probabilities for belonging of elements of the mixture to certain components of this mixture, which is decisive for a criteria whether a particular element belongs to a particular cluster) using the EM-algorithm. As a result of the analysis of considered type of mathematical singularity, it is shown that it is possible to remove an empty cluster in such a way that the structure and values of the log-likelihood function are adjusted to those that would be in the case of the a priori absence of this specified empty cluster. The peculiarity, which is analyzed in the article, is more typical for the modification of the EM-algorithm with deletion the components of the Gaussian mixture model. This peculiarity may also arise in cases of relatively small number of elements that need to be analyzed through clustering or relatively large number of components of the Gaussian mixture model (clusters), which is the structural parameter of the EM-algorithmReferences
Jain A. K., Murty M. N., Flynn P. J. Data clustering: a review. ACM Computing Surveys. 1999. Vol. 31. No. 3. P. 264-323. DOI: 10.1145/331499.331504.
Bakir G. H. et al. Predicting Structured Data (Neural Information Processing series) 1st Edition. The MIT Press, 2007. 360 p.
Dempster A. P., Laird N. M., Rubin D. B. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Socie-ty. Series B. 1977. Vol. 39. No. 1. P. 1-38.
Gupta M. R., Chen Y. Theory and Use of the EM Algorithm. Foundations and Trends® in Signal Processing. 2011. Vol. 4. No. 3. P. 223-296. DOI: 10.1561/2000000034.
Vlassis N., Likas A. A Greedy EM Algorithm for Gaussian Mixture Learning. Neural Processing Letters. 2002. Vol. 15. P. 77-87.
Yu D., Deng L. Gaussian Mixture Models. Au-tomatic Speech Recognition. A Deep Learning Ap-proach. London: Springer-Verlag, 2015. P. 13-21. DOI: 10.1007/978-1-4471-5779-3_2
Huang T., Peng H., Zhang K. Model Selection for Gaussian Mixture Models. Statistica Sinica. 2017. Vol. 27. P. 147-169.
Wang Xi et al. An Adaptive Prognostic Ap-proach for Newly Developed System with Three-Source Variability. IEEE Access. 2019. Vol. 7. P. 53091-53102. DOI: 10.1109/ACCESS.2019.2911307
Wei B., Nener B. D. Multi-Sensor Space De-bris Tracking for Space Situational Awareness with Labeled Random Finite Sets. IEEE Access. 2019. Vol. 7. P. 36991-37003. DOI: 10.1109/ACCESS.2019.2904545
Li C., Okamura H., Dohi T. Parameter Esti-mation of Mt/M/1/K Queueing Systems with Utiliza-tion Data. IEEE Access. 2019. Vol. 7. P. 42664-42671. DOI: 10.1109/ACCESS.2019.2906796
Голубничий А. Г., Конахович Г. Ф. Муль-типликативно комплементарные бинарные сиг-нально-кодовые конструкции. Известия высших учебных заведений. Радиоэлектроника. 2018. Т. 61. № 10. С. 551-565. DOI: 10.20535/S0021347018100011.
Голубничий О. Г. Синтез аналітичних форм опису автокореляційної функції узагальне-них бінарних послідовностей Баркера типу 1 на основі її декомпозиції з використанням лінійних складових. Наукоємні технології. 2019. Т. 41. № 1. С. 10-15. DOI: 10.18372/2310-5461.41.13523.
Голубничий О. Г. Синтез систем корельо-ваних сигналів з використанням доповненої про-цедури Грама-Шмідта. Наукоємні технології. 2018. Т. 40. № 4. С. 405-408. DOI:
18372/2310-5461.40.13265.
