Skip to main content
Springer Nature Link
Log in

Social image tag enrichment based on textual similarity modeling

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In social image sharing websites, users provide several descriptive tags to annotate their shared images. Usually, the user annotated tags are noisy, biased and incomplete. How to improve tag quality is very important for tag based applications. The content relevant tags have certain similarities or connections with each other. Thus from some highly relevant tags, we can infer the other content relevant tags for an image. In this paper, a social image tag enrichment approach is proposed. Considering the diversity of content relevant tags for the image, we first determine some seed tags which are highly relevant to image content and cover wide range of semantics. Then the seed tags are utilized to adopt semantic similarity tags for the input image. Experiments demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Ames M, Naaman M (2007) Why We Tag: Motivations for Annotation in Mobile and Online Media. In Proc. SIGCHI Conference on Human Factors in Computing System

  2. Chang X, Yang Y (2016) Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2016.2582746

    Article  MathSciNet  Google Scholar 

  3. Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound Rank-k Projections for Bilinear Analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513

    Article  MathSciNet  Google Scholar 

  4. Chang X, Nie F, Yang Y, Zhang C, Huang H (2016) Convex Sparse PCA for Unsupervised Feature Analysis. ACM Trans Knowl Discov Data 11(1):3:1–3:16

    Article  Google Scholar 

  5. Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound Rank-k Projections for Bilinear Analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513

    Article  MathSciNet  Google Scholar 

  6. Chang X, Ma Z, Yang Y, Zeng Z (2017) Alexander G. Hauptmann: Bi-Level Semantic Representation Analysis for Multimedia Event Detection. IEEE Trans Cybern 47(5):1180–1197

    Article  Google Scholar 

  7. Chang X, Yu Y, Yang Y, Xing EP (2017) Semantic Pooling for Complex Event Analysis in Untrimmed Videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632

    Article  Google Scholar 

  8. Chang X, Ma Z, Lin M, Yang Y, Hauptmann AG (2017) Feature Interaction Augmented Sparse Learning for Fast Kinect Motion Detection. IEEE Trans Image Process 26(8):3911–3920

    Article  MathSciNet  MATH  Google Scholar 

  9. Chua T, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: A real-world web image database from national university of Singapore. In Proc. CIVR

  10. Datta R, Joshi D, Li J, Wang JZ (2007) Tagging over time: Realworld image annotation by lightweight meta-learning. In: Proc. ACM Mutlimedia, p 393–402

  11. Feng S, Lang C, Xu D (2010) Beyond tag relevance: integrating visual attention model and multi-instance learning for tag saliency ranking. CIVR, p 288–295

  12. Gao Y, Wang M, Zha Z, Shen J, Li X, Wu X (2013) Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search. IEEE Trans Image Process 22(1):363–376

    Article  MathSciNet  MATH  Google Scholar 

  13. Gu Y, Qian X, Li Q, Wang M, Hong R, Tian Q (2015) Image Annotation by Latent Community Detection and Multi-Kernel Learning. IEEE Trans Image Process 24(11):3450–3463

    Article  MathSciNet  MATH  Google Scholar 

  14. Han Y, Wu F, Tian Q, Zhuang Y (2012) Image Annotation by Input-Output Structural Grouping Sparsity. IEEE Trans Image Process 21(6):3066–3079

    Article  MathSciNet  MATH  Google Scholar 

  15. Jiang S, Qian X, Shen J, Fu Y, Mei T (2015) Author Topic Model-Based Collaborative Filtering for Personalized POI Recommendations. IEEE Trans Multimedia 17(6):907–918

    Google Scholar 

  16. Jiang S, Qian X, Fu Y, Mei T (2016) Personalized Travel Sequence Recommendation on Multi-Source Big Social Media. IEEE Trans Big Data 1(2):43–56

    Article  Google Scholar 

  17. Joshi D, Luo J, Yu J, Lei P, Gallagher A (2011) Using Geotags to Derive Rich Tag-Clouds for Image Annotation, Social Media Modeling and Computing. Springer, Berlin

    Google Scholar 

  18. Kleban J, Moxley E, Xu J, Manjunath BS (2009) Global annotation on georeferenced photographs. In: Proc. CIVR

  19. Lei X, Qian X, Zhao G (2016) Rating Prediction based on Social Sentiment from Textual Reviews. IEEE Trans Multimedia 18(9):1910–1921

    Article  Google Scholar 

  20. Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002

    Article  Google Scholar 

  21. Li J, Qian X, Lan K, Qi P, Sharma A (2015) Improved image GPS location estimation by mining salient features. Sig. Proc.: Image Comm. 38:141–150

    Google Scholar 

  22. Li X, Chen L, Zhang L, Ma W, Lin F (2006) Image annotation by large-scale content-based image retrieval. ACM MM

  23. Li X, Snoek CGM, Worring M (2008) Learning tag relevance by neighbor voting for social image retrieval. ACM MIR, p 180–187

  24. Li X, Snoek C, Worring M (2009) Learning Social Tag Relevance by Neighbor Voting. IEEE Trans Multimedia 11(7):1310–1322

    Article  Google Scholar 

  25. Li G, Wang M, Lu Z, Hong R, Chua T (2012) In-Video Product Annotation with Web Information Mining. ACM Trans Multimed Comput Commun Appl 8(4)

    Article  Google Scholar 

  26. Li J, Qian X, Tang Y, Yang L, Mei T (2013) GPS estimation for places of interest from social users’ uploaded photos. IEEE Trans Multimedia 15(8):2058–2071

    Article  Google Scholar 

  27. Li X, Guo Q, Lu X (2016) Spatiotemporal Statistics for Video Quality Assessment. IEEE Trans Image Process 25(7):3329–3342

    Article  MathSciNet  MATH  Google Scholar 

  28. Li X, Mou L, Lu X (2016) Surveillance Video Synopsis via Scaling Down Objects. IEEE Trans Image Process 25(2):740–755

    Article  MathSciNet  MATH  Google Scholar 

  29. Liu D, Hua X, Yang L, Wang M, Zhang H (2009) Tag ranking. In: Proc. WWW

  30. Liu D, Hua X-S, Wang M, Zhang H-J (2010) Retagging social images based on visual and semantic consistency. In: Proc. ACM WWW, p 1149–1150

  31. Liu D, Hua X-S, Wang M, Zhang H-J (2010) Image retagging. In: Proc. ACM Multimedia

  32. Liu D, Wang M, Hua X, Zhang H (2011) Semi-Automatic Tagging of Photo Albums via Exemplar Selection and Tag Inference. IEEE Trans Multimedia 13(1):82–91

    Article  Google Scholar 

  33. Liu D, Yan S, Hua X, Zhang H (2011) Image Retagging Using Collaborative Tag Propagation. IEEE Trans Multimedia

  34. Lu X, Li X (2014) Multiresolution Imaging. IEEE Trans Cybern 44(1):149–160

    Article  Google Scholar 

  35. Lu X, Wang Y, Yuan Y (2013) Graph Regularized Low-Rank Representation for Destriping of Hyperspectral Images. IEEE Trans Geosci Remote Sens 51(7):4009–4018

    Article  Google Scholar 

  36. Lu X, Wu H, Yuan Y, Yan P, Li X (2013) Manifold Regularized Sparse NMF for Hyperspectral Unmixing. IEEE Trans Geosci Remote Sens 51(5):2815–2826

    Article  Google Scholar 

  37. Lu X, Li X, Li M (2015) Semi-Supervised Multi-task Learning for Scene Recognition. IEEE Trans Cybern 45(9):1967–1976

    Article  Google Scholar 

  38. Lu D, Liu X, Qian X (2016) Tag based Image Search by Social Re-Ranking. IEEE Trans Multimedia 18(8):1628–1639

    Article  Google Scholar 

  39. Lu X, Yuan Y, Zhang X (2016) Jointly Dictionary Learning for Change Detection in Multispectral Imagery. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2016.2531179

    Article  Google Scholar 

  40. Lu X, Li X, Zheng X (2017) Latent Semantic Minimal Hashing for Image Retrieval. IEEE Trans Image Process 26(1):355–368

    Article  MathSciNet  MATH  Google Scholar 

  41. Mei T, Wang Y, Hua X, Gong S, Li S (2008) Coherent image annotation by learning semantic distance. In: Proc. CVPR

  42. Moxley E, Mei T, Manjunath B (2010) Video annotation through search and graph reinforcement mining. IEEE Trans Multimedia 12(3):184–193

    Article  Google Scholar 

  43. Qian X, Hua X (2011) Graph-cut based tag enrichment. In: Proc. SIGIR, p 1111–1112

  44. Qian X, Hua X, Hou X (2012) Tag Filtering based on Similar Compatible Principle. In: Proc. ICIP

  45. Qian X, Liu X, Zheng C, Du Y, Hou X (2013) Tagging photos using users’ vocabularies. Neurocomputing 111:144–153

    Article  Google Scholar 

  46. Qian X, Hua X, Tang Y, Mei T (2014) Social Image Tagging with Diverse Semantics. IEEE Trans Cybern 44(12):2493–2508

    Article  Google Scholar 

  47. Qian X, Feng H, Zhao G, Mei T (2014) Personalized Recommendation Combining User Interest and Social Circle. IEEE Trans Knowl Data Eng 26(7):1487–1502

    Article  Google Scholar 

  48. Qian X, Xue Y, Tang Y, Hou X, Mei T (2015) Landmark Summarization with Diverse Viewpoints. IEEE Trans Circuits and Syst Video Technol 25(11):1857–1869

    Article  Google Scholar 

  49. Qian X, Zhao Y, Han J (2015) Image Location Estimation by Salient Region Matching. IEEE Trans Image Process 24(6):4348–4358

    Article  MathSciNet  MATH  Google Scholar 

  50. Qian X, Wang H, Zhao Y, Hou X, Hong R, Wang M, Tang YY (2017) Image Location Inference by Multisaliency Enhancement. IEEE Trans Multimedia 19(4):813–821

    Article  Google Scholar 

  51. Qian X, Lu D, Wang Y, Zhu L, Tang YY, Wang M (2017) Image Re-Ranking Based on Topic Diversity. IEEE Trans Image Process 26(8):3734–3747

    Article  MathSciNet  MATH  Google Scholar 

  52. Shen J, Meng W, Yan S, Pang H, Hua X (2010) Effective music tagging through advanced statistical modeling. SIGIR

  53. Wang C, Jing F, Zhang L, Zhang H (2007) Content-based image annotation refinement. In: Proc. CVPR

  54. Wang X, Zhang L, Li X, Ma W (2008) Annotating images by mining image search results. IEEE Trans Pattern Anal Mach Intell 30(11):1919–1932

    Article  Google Scholar 

  55. Wang X-J, Yu M, Zhang L, Cai R, Ma W-Y (2009) Argo: Intelligent Advertising by Mining a User's Interest from His Photo Collections. ACM Data Mining and Audience Intelligence for Advertising, p 18–26

  56. Wang X, Zhang L, Liu M, Li Y, Ma W (2010) ARISTA - image search to annotation on billions of web photos. In: Proc. CVPR, p 2987–2994

  57. Wang M, Yang K, Hua X, Zhang H (2010) Towards a Relevant and Diverse Search of Social Images. IEEE Trans Multimedia 12(8):829–842

    Article  Google Scholar 

  58. Wang M, Ni B, Hua X, Chua T (2012) Assistive Tagging: A Survey of Multimedia Tagging with Human-Computer Joint Exploration. ACM Comput Surv 44(4)

    Article  Google Scholar 

  59. Wang M, Hong R, Li G, Zha Z, Yan S, Chua T (2012) Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification. IEEE Trans Multimedia 14(4):975–985

    Article  Google Scholar 

  60. Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal Graph-Based Reranking for Web Image Search. IEEE Trans Image Process 21(11):4649–4661

    Article  MathSciNet  MATH  Google Scholar 

  61. Weinberger K, Slaney M, van Zwol R (2008) Resolving tag ambiguity. In: Proc. ACM Multimedia, p 111–119

  62. Wu L, Hua X-S, Yu N, Ma W-Y, Li S (2008) Flickr distance. In: Proc. ACM Multimedia, p 31–40

  63. Wu L, Yang LJ, Yu NH, Hua XS (2009) Learning to Tag. In: Proc. of ACM WWW

  64. Xu H, Wang J, Hua X, Li S (2009) Tag refinement by regularized lda. In: Proc. ACM Multimedia

  65. Yan Y, Nie F, Li W, Gao C, Yang Y, Xu D Image classification by cross-media active learning with privileged information. IEEE Trans Multimedia 18(12):2494–2502

    Article  Google Scholar 

  66. Yang K, Hua X, Wang M, Zhang H (2011) Tag Tagging: Towards More Descriptive Keywords of Image Content. IEEE Trans Multimedia 13(4):662–673

    Article  Google Scholar 

  67. Yang Y, Wu F, Nie F, Shen H, Zhuang Y, Hauptmann A (2012) Web and Personal Image Annotation by Mining Label Correlation with Relaxed Visual Graph Embedding. IEEE Trans Image Process 21(3):1339–1351

    Article  MathSciNet  MATH  Google Scholar 

  68. Yang Y, Nie F, Xu D, Luo J, Zhuang Y, Pan Y (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell :723–742

  69. Yang X, Qian X, Xue Y (2015) Scalable Mobile Image Retrieval by Exploring Contextual Saliency. IEEE Trans Image Process 24(6):1709–1721

    Article  MathSciNet  MATH  Google Scholar 

  70. Yang Y, Ma Z, Nie F, Chang X, Hauptmann AG Multi-class active learning by uncertainty sampling with diversity maximization. Int J Comput Vis 113(2):113–127

    Article  MathSciNet  Google Scholar 

  71. Yang Y, Ma Z, Hauptmann AG, Sebe N Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Trans Multimedia 15(3):661–669

    Article  Google Scholar 

  72. Yuan Y, Zheng X, Lu X (2017) Hyperspectral Band Selection by Discovering Diverse Subset in Multiple Graphs. IEEE Trans Image Process 26(1)

  73. Zha Z, Hua X, Mei T, Wang J, Qi G, Wang Z (2008) Joint multi-label multi-instance learning for image classification. In: Proc. CVPR

  74. Zha Z, Wang M, Zheng Y, Yang Y, Hong R, Chua T (2012) Interactiv e Video Indexing With Statistical Active Learning. IEEE Trans Multimedia 14(1):17–27

    Article  Google Scholar 

  75. Zhang S, Huang J, Li H, Metaxas D (2012) Automatic Image Annotation and Retrieval Using Group Sparsity. IEEE Trans Syst Man Cybern Part B Cybern 42(3):838–849

    Article  Google Scholar 

  76. Zhang D, Han J, Jiang L, Ye S, Chang X (2017) Revealing Event Saliency in Unconstrained Video Collection. IEEE Trans Image Process 26(4):1746–1758

    Article  MathSciNet  MATH  Google Scholar 

  77. Zhao G, Qian X, Lei X (2016) Objective Evaluation for Service by Deep Exploring Social Users’ Contextual Information. IEEE Trans Knowl Data Eng 28(12):3382–3394

    Google Scholar 

  78. Zhao G, Qian X, Xie X (2016) User-Service Rating Prediction by Exploring Social Users’ Rating Behaviors. IEEE Trans Multimedia 18(3):496–506

    Article  Google Scholar 

  79. Zhao G, Qian X, Kang C (2017) Service Rating Prediction by Exploring Social Mobile Users' Geographical Locations. IEEE Trans Big Data 3(1):67–78

    Article  Google Scholar 

  80. Zhou N, Cheung WK, Qiu G, Xue X (2011) A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging. IEEE Trans Pattern Anal Mach Intell 33(7):1281–1294

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Xi’an Jiaotong University City College, Xi’an, 710049, China

    Miao Shen

Authors
  1. Miao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miao Shen.

Electronic supplementary material

ESM 1

(PDF 73 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, M. Social image tag enrichment based on textual similarity modeling. Multimed Tools Appl 77, 3659–3676 (2018). https://doi.org/10.1007/s11042-017-5184-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5184-x

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy