Abstract
In recent recommendation systems, the image information of items is often used in conjunction with deep convolution network to directly learn the visual features of items. However, the existing approaches usually use only one image to represent an item. These approaches are inadequate for an item with multi-view related images. For a restaurant, it has visual information of food, drink, environment, and so on. Each view of an item can be represented by multiple images. In this paper, we propose a new factorization model that combines multi-view visual information with the implicit feedback data for restaurant prediction and ranking. The visual features (visual information) of images are extracted by using a deep convolution network and are integrated into a collaborative filtering framework. In order to conduct personalized recommendation better, the multi-view visual features are fused through user related weights. User related weights reflect the personalized visual preference for restaurants and the weights are different and independent between users. We applied this model to make personalized recommendations for users on two real-world restaurant review datasets. Experimental results show that our model with multi-view visual information achieves better performance than models without or with only single-view visual information.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions [J]. IEEE Trans Knowl Data Eng 17(6):734–749
Cheng H T, Koc L, Harmsen J et al (2016) Wide & deep learning for recommender systems [C]. Proceedings of the 1st workshop on deep learning for recommender systems, pp 7–10
Chiliguano P, Fazekas G (2016) Hybrid music recommender using content-based and social information [C]. IEEE International Conference on Acoustics, Speech and Signal Processing, pp 2618–2622
Chu W T, Tsai Y L (2017) A hybrid recommendation system considering visual information for predicting favorite restaurants [J]. World Wide Web 20(6):1313–1331
Deng J, Dong W, Socher R et al (2009) Imagenet: a large-scale hierarchical image database [C]. IEEE conference on computer vision and pattern recognition, p 248–255
Fu Y, Liu B, Ge Y et al (2014) User preference learning with multiple information fusion for restaurant recommendation [C]. Proceedings of the 2014 SIAM international conference on data mining, pp 470–478
Gupta A, Singh K (2013) Location based personalized restaurant recommendation system for mobile environments [C]. International Conference on Advances in Computing, Communications and Informatics, pp 507–511
Hao P Y, Cheang W H, Chiang J H (2019) Real-time event embedding for POI recommendation [J]. Neurocomputing 349:1–11
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
He R, McAuley J (2016) VBPR: Visual bayesian personalized ranking from implicit feedback [C]. The Thirtieth AAAI Conference on Artificial Intelligence, pp 144–151
He X, Liao L, Zhang H et al (2017) Neural collaborative filtering [C]. Proceedings of the 26th international conference on world wide web, pp 173–182
Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets [C]. Eighth IEEE International Conference on Data Mining, pp 263–272
Jaworek-Korjakowska J, Kleczek P, Gorgon M (2019) Melanoma Thickness Prediction Based on Convolutional Neural Network with VGG-19 Model Transfer Learning [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
Koren Y, Bell R (2015) Advances in collaborative filtering [M]. Recommender systems handbook. Springer, pp pp 77–118
Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems [J]. Computer 42(8):30–37
Lee B H, Kim H N, Jung J G et al (2006) Location-based service with context data for a restaurant recommendation [C]. International Conference on Database and Expert Systems Applications, pp 430–438
Lin T Y, RoyChowdhury A, Maji S (2015) Bilinear cnn models for fine-grained visual recognition [C]. Proceedings of the IEEE international conference on computer vision, pp 1449–1457
Luo H, Zhang X, Chen B et al (2018) Multi-view visual bayesian personalized ranking from implicit feedback [C]. Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, pp 361–362
McAuley J, Targett C, Shi Q et al (2015) Image-based recommendations on styles and substitutes [C]. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 43–52
Miao D, Lang F (2017) A recommendation system based on text mining [C]. In: 2017 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, pp 318–321
Pan R, Zhou Y, Cao B et al (2008) One-class collaborative filtering [C]. Eighth IEEE International Conference on Data Mining, pp 502–511
Park M H, Park H S, Cho S B (2008) Restaurant recommendation for group of people in mobile environments using probabilistic multi-criteria decision making [C]. sia-Pacific Conference on Computer Human Interaction, pp 114–122
Ren S, He K, Girshick R et al (2015) Faster r-cnn: Towards real-time object detection with region proposal networks [C]. Advances in neural information processing systems, pp 91–99
Rendle S, Freudenthaler C, Gantner Z et al (2012) BPR: Bayesian personalized ranking from implicit feedback. arXiv:1205.2618
Shin H C, Roth H R, Gao M et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning [J]. IEEE Trans Med Imaging 35(5):1285–1298
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision [C]. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Walter F E, Battiston S, Schweitzer F (2008) A model of a trust-based recommendation system on a social network [J]. Auton Agent Multi-Agent Syst 16(1):57–74
Wang S, Wang Y, Tang J et al (2017) What your images reveal: Exploiting visual contents for point-of-interest recommendation [C]. Proceedings of the 26th International Conference on World Wide Web, pp 391–400
Wen L, Li X, Li X et al (2019) A new transfer learning based on VGG-19 network for fault diagnosis [C]. IEEE 23rd International Conference on Computer Supported Cooperative Work in Design, pp 205–209
Xia B, Li Y, Li Q et al (2017) Attention-based recurrent neural network for location recommendation [C]. 12th International Conference on Intelligent Systems and Knowledge Engineering, pp 1–6
Xue H J, Dai X, Zhang J et al (2017) Deep matrix factorization models for recommender systems [C]. Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp 3203–3209
Yang Z, Yang D, Dyer C et al (2016) Hierarchical attention networks for document classification [C]. Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1480–1489
Yin H, Zhou X, Cui B et al (2016) Adapting to user interest drift for poi recommendation [J]. IEEE Trans Knowl Data Eng 28(10):2566–2581
Zhang J, Yang Y, Tian Q et al (2017) Personalized social image recommendation method based on user-image-tag model [J]. IEEE Trans Multimed 19(11):2439–2449
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (Grant No. 61602313, 61702084).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, X., Luo, H., Chen, B. et al. Multi-view visual Bayesian personalized ranking for restaurant recommendation. Appl Intell 50, 2901–2915 (2020). https://doi.org/10.1007/s10489-020-01703-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01703-6