Recommender Systems Based on Reinforced Learning
DOI:
https://doi.org/10.18372/1990-5548.76.17668Keywords:
machine learning, reinforcement learning, recommendation systems, recommender agent, collaborative filtering, Actor-Critic, explicit feedbackAbstract
This article is devoted to the problem of building recommender systems based on the use of artificial intelligence methods. The paper analyzes the algorithms of recommender systems. Analyzes the Markov decision-making process in the context of recommender systems. Approaches to the adaptation of reinforcement learning algorithms to the task of recommendations (transition from the task of supervised learning to the task of reinforcement learning) are considered. Reinforcement learning algorithms Deep Deterministic Policy Gradient and Twin Delayed DDPG were implemented with their own environment simulating the user's reaction, and the results were compared. The structure of a recommender system has been developed, in which the recommender agent generates a list of offers for an individual user, using his previous history of ratings. In the system itself, the user has the ability to interact only with the space of recommended films. This can be compared to the main YouTube page, which is a feed with suggestions, but we have a user interacting only with this feed and his reaction to objects in the recommendation space falls into recommender agent, which regulates the parameters of the model in the learning process.
References
J. A. Konstan and J. Riedl, “Recommender systems: from algorithms to user experience,” User Model User-Adapt Interact, 22: 101–23. 2012. https://doi.org/10.1007/s11257-011-9112-x
P. Pu, L. Chen, and R. Hu, “A user-centric evaluation framework for recommender systems,” In: Proceedings of the fifth ACM conference on Recommender Systems (RecSys’11), ACM, New York, NY, USA; 2011, pp. 57–164. https://doi.org/10.1145/2043932.2043962
R. Hu and P. Pu, “Potential acceptance issues of personality-ASED recommender systems,” In: Proceedings of ACM conference on recommender systems (RecSys’09), New York City, NY, USA; October 2009. pp. 22–5. https://doi.org/10.1145/1639714.1639753
B. Pathak, R.Garfinkel R. Gopal, R. Venkatesan, and F. Yin, “Empirical analysis of the impact of recommender systems on sales,” J Manage In form Syst, 27(2): 159–88, 2010. https://doi.org/10.2753/MIS0742-1222270205
Soufiene Jaffali, Salma Jamoussi, Kamel Smaili, and Abdelmajid Ben Hamadou, “Like-tasted user groups to predict ratings in recommender systems,” Social Netw. Analys, Mining, vol. 10, no. 1, p. 42, 2020. https://doi.org/10.1007/s13278-020-00643-w
K. Zhou, S.-H. Yang, and H. Zha, “Functional matrix factorizations for cold-start recommendation,” in Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’11, (New York, NY, USA), pp. 315–324, ACM, 2011. https://doi.org/10.1145/2009916.2009961
X. Zhang, J. Cheng, T. Yuan, B. Niu, and H. Lu, “Semi-supervised discriminative preference elicitation for cold-start recommendation,” in Proceedings of the 22Nd ACM International Conference on Information & Knowledge Management, CIKM ’13, (New York, NY, USA), pp. 1813–1816, ACM, 2013. https://doi.org/10.1145/2505515.2507869
P. Mazumdar, B. K. Patra, and K. S. Babu, “Cold-star point-of-interest recommendation through crowd sourcing,” ACM Trans. Web, vol. 14, Aug. 2020. https://doi.org/10.1145/3407182
N. K. Mishra, V. Mishra, and S. Chaturvedi, “Solving cold start problem using MBA,” in 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI), pp. 1598–1601, 2017. https://doi.org/10.1109/ICPCSI.2017.8391981
N. Mishra, V. Mishra, and S. Chaturvedi, “Tools and techniques for solving cold start recommendation,” in Proceedings of the 1st International Conference on Internet of Things and Machine Learning, IML ’17, (New York, NY, USA), Association for Computing Machinery, 2017. https://doi.org/10.1145/3109761.3109772
M. Saveskiand A. Mantrach, “Item cold-start recommendations: Learning local collective embeddings,” in Proceedings of the 8th ACM Conference on Recommender Systems, RecSys ’14, (New York, NY, USA), pp. 89–96, ACM, 2014. https://doi.org/10.1145/2645710.2645751
Y. Rong, X. Wen, and H. Cheng, “A monte-carlo algorithm for cold start-recommendation,” in Proceedings of the 23rd international conference on Worldwide web, pp. 327–336, ACM, 2014. https://doi.org/10.1145/2566486.2567978
A. da Costa, E. Fressato, F. Neto, M. Manzato, and R. Campello, “Case recommender: A flexible and extensible python frame work for recommender systems,” in Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, (New York, NY, USA), p. 494–495, Association for Computing Machinery, 2018. https://doi.org/10.1145/3240323.3241611
T. Kitazawaand M. Yui, “Query-based simple and scalable recommender systems with apache hive mall,” in Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, (New York, NY, USA), p. 502–503, Association for Computing Machinery, 2018. https://doi.org/10.1145/3240323.3241592
N. Mishra, S. Chaturvedi, V. Mishra, R. Srivastava, and P. Bargah, “Solving sparsity problem in rating-based movie recommendation system,” in Computational Intelligence in Data Mining (H. S. Beheraand D. P. Mohapatra, eds.), (Singapore), pp. 111–117, Springer Singapore, 2017. https://doi.org/10.1007/978-981-10-3874-7_11
W. Pan, E. W. Xiang, N. N. Liu, and Q. Yang, "Transfer learning in collaborative filtering for sparsity reduction.," in AAAI, vol. 10, pp. 230–235, 2010. https://doi.org/10.1609/aaai.v24i1.7578.
P. Adamopoulos and A. Tuzhilin, “On over-specialization and concentration bias of recommendations: Probabilistic neighborhood selection in collaborative filtering systems,” in Proceedings of the 8th ACM Conference on Recommender systems, pp. 153–160, ACM, 2014. https://doi.org/10.1145/2645710.2645752
Z.-K. Zhang, C. Liu, Y.-C. Zhang, and T. Zhou, “Solving the cold-start problem in recommender systems with social tags,” EPL (Euro physics Letters), vol. 92, no. 2, p. 28002, 2010. https://doi.org/10.1209/0295-5075/92/28002
N. Lathia, S. Hailes, L. Capra, and X. Amatriain, “Temporal diversity in recommender systems,” in Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pp. 210–217, ACM, 2010. https://doi.org/10.1145/1835449.1835486
Lalita Sharma and Anju Gera. "A Survey of Recommendation System: Research Challenges," International Journal of Engineering Trends and Technology (IJETT), vol. 4(5) 2013, pp.1989–1992.
J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez, "Hybrid recommender systems: survey and experiments," User Model User-adapted Interact, 12 (4) (2002), pp. 331–370
N. Friedman, D. Geiger, and M. Goldszmidt, ”Recommender systems survey Knowl-Based Syst,” Bayesian net work classifiers, vol.46, pp. 109–132, 2013, https://doi.org/10.1016/j.knosys.2013.03.012
Mach Learn, 29 (2–3), 1997, pp. 131–163. https://doi.org/10.1023/A:1007465528199
R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification John Wiley & Sons (2012).
J. L. Herlocker, J. A. Konstan, L. G. Terveen, J. T. Riedl, “Evaluating collaborative filtering recommender systems ACM Trans ,” In form Syst, 22 (1), 2004. https://doi.org/10.1145/963770.963772
M. Montaner, B. Lopez, and J. L. Dela Rosa, “A Taxonomy of Recommender Agent son the Internet,” Artificial Intelligence Review, Kluwer Academic Publisher, 2003.
Yan-Martin Tamm, Rinchin Damdinov, and Alexey Vasilev, Quality Metrics in Recommender Systems: Do We Calculate Metrics Consistently?
Downloads
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
