A Comprehensive Benchmark of Collaborative Filtering Methods on Implicit Feedback Datasets

Abstract

Collaborative filtering is a foundational technique in modern recommender systems, especially when dealing with implicit feedback signals such as clicks, purchases, or listening behavior. Despite the abundance of сollaborative filtering models, including classical, probabilistic, and neural approaches, there is a lack of standardized, large-scale evaluations across diverse datasets. This study presents a comprehensive empirical benchmark of 13 сollaborative filtering algorithms encompassing matrix factorization, pairwise ranking, variational and non-variational autoencoders, graph-based neural models, and probabilistic methods. Using four representative implicit feedback datasets from different domains, we evaluate models under a unified experimental protocol using ranking-based metrics (MAP@10, NDCG@10, Precision@10, Recall@10, MRR), while also reporting training efficiency. Our results reveal that neural architectures such as NeuMF, VAECF, and LightGCN offer strong performance in dense and moderately sparse scenarios, but may face scalability constraints on larger datasets. Simpler models like EASEᴿ and BPR often achieve a favorable balance between performance and efficiency. This benchmark offers actionable insights into the trade-offs of modern сollaborative filtering methods and guides future research in implicit recommender systems.