research-article

You are what emojis say about your pictures: language-independent gender inference attack on Facebook

Authors:

Bizhan Alipour Pijani,

Abdessamad Imine,

Michaël RusinowitchAuthors Info & Claims

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

Pages 1826 - 1834

https://doi.org/10.1145/3341105.3373943

Published: 30 March 2020 Publication History

Abstract

The picture owner's gender has a strong influence on individuals' emotional reactions to the picture. In this study, we investigate gender inference attacks on their owners from pictures meta-data composed of: (i) alt-texts generated by Facebook to describe the content of pictures, and (ii) Emojis/Emoticons posted by friends, friends of friends or regular users as a reaction to the picture. Specifically, we study the correlation of picture owner gender with alt-text, and Emojis/Emoticons used by commenters when reacting to these pictures. We leverage this image sharing and reaction mode of Facebook users to derive an efficient and accurate technique for user gender inference. We show that such a privacy attack often succeeds even when other information than pictures published by their owners is either hidden or unavailable.

References

[1]

Chaabane Abdelberi, Gergely Ács, and Mohamed Ali Kâafar. 2012. You are what you like! Information leakage through users' Interests. In 19th Annual Network and Distributed System Security Symposium, NDSS. The Internet Society, San Diego, California, USA.

[2]

Wei Ai, Xuan Lu, Xuanzhe Liu, Ning Wang, Gang Huang, and Qiaozhu Mei. 2017. Untangling Emoji Popularity Through Semantic Embeddings. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM. AAAI Press, Montréal, Québec, Canada, 2--11.

[3]

Bizhan Alipour, Abdessamad Imine, and Michaël Rusinowitch. 2019. Gender Inference for Facebook Picture Owners. In International Conference on Trust, Privacy and Security in Digital Business, Trust Bus. Springer, Linz, Austria, 145--160.

[4]

Jiaxin An, Tian Li, Yifei Teng, and Pengyi Zhang. 2018. Factors Influencing Emoji Usage in Smartphone Mediated Communications. In Transforming Digital Worlds - 13th International Conference, iConference. Springer, Sheffield, UK, 423--428.

[5]

Francesco Barbieri, Germán Kruszewski, Francesco Ronzano, and Horacio Saggion. 2016. How Cosmopolitan Are Emojis?: Exploring Emojis Usage and Meaning over Different Languages with Distributional Semantics. In Proceedings of the Conference on Multimedia Conference, MM. ACM, Amsterdam, The Netherlands, 531--535.

Digital Library

[6]

Tena Belinic. 2009. Personality profile of social media users how to get maximum from it. https://medium.com/krakensystems-blog/personality-profile-of-social-media-users-how-to-get-maximum-from-it-5e8b803efb30.

[7]

Robyn L. Brouer, Michael Stefanone, Rebecca L. Badawy, and Michael J. Egnoto. 2017. Gender (In)Consistent Communication via Social Media and Hireability: An Exploratory Study. In 50th Hawaii International Conference on System Sciences, HICSS. ScholarSpace / AISeL, Hawaii, USA, 1--10.

[8]

Sarah E Butterworth, Traci A Giuliano, Justin White, Lizette Cantu, and Kyle C Fraser. 2019. Sender Gender Influences Emoji Interpretation in Text Messages. Frontiers in psychology 10 (2019), 784.

[9]

Zhenpeng Chen, Xuan Lu, Wei Ai, Huoran Li, Qiaozhu Mei, and Xuanzhe Liu. 2018. Through a Gender Lens: Learning Usage Patterns of Emojis from Large-Scale Android Users. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW. ACM, Lyon, France, 763--772.

Digital Library

[10]

Zhenpeng Chen, Sheng Shen, Ziniu Hu, Xuan Lu, Qiaozhu Mei, and Xuanzhe Liu. 2019. Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification. In The World Wide Web Conference, WWW. ACM, San Francisco, CA, USA, 251--262.

[11]

Ming Cheung and James She. 2017. An Analytic System for User Gender Identification through User Shared Images. TOMCCAP 13, 3 (2017), 30.

Digital Library

[12]

Munmun De Choudhury, Sanket S. Sharma, Tomaz Logar, Wouter Eekhout, and René Clausen Nielsen. 2017. Gender and Cross-Cultural Differences in Social Media Disclosures of Mental Illness. In Proceedings of the Conference on Computer Supported Cooperative Work and Social Computing, CSCW. ACM, Portland, OR, USA, 353--369.

Digital Library

[13]

Reza Farahbakhsh, Xiao Han, Ángel Cuevas, and Noël Crespi. 2017. Analysis of publicly disclosed information in Facebook profiles. CoRR abs/1705.00515 (2017).

[14]

Lucie Flekova, Jordan Carpenter, Salvatore Giorgi, Lyle H. Ungar, and Daniel Preotiuc-Pietro. 2016. Analyzing Biases in Human Perception of User Age and Gender from Text. In Proceedings of the 54th Annual Meeting of the Association for Computational. ACL, Berlin, Germany, 843--854.

[15]

Neil Zhenqiang Gong and Bin Liu. 2016. You Are Who You Know and How You Behave: Attribute Inference Attacks via Users' Social Friends and Behaviors. In 25th Security Symposium. USENIX, Austin, TX, USA, 979--995.

[16]

Fariba Karimi, Claudia Wagner, Florian Lemmerich, Mohsen Jadidi, and Markus Strohmaier. 2016. Inferring Gender from Names on the Web: A Comparative Evaluation of Gender Detection Methods. In Proceedings of the 25th International Conference on World Wide Web, WWW. ACM, Montreal, Canada, 53--54.

Digital Library

[17]

Abbey Lenton. 2019. Facebook Wants You To Search Photos Of Your Female Friends At The Beach, But Not Your Male Mates. https://www.whimn.com.au/talk/people/facebook-wants-you-to-search-photos-of-your-female-friends-at-the-beach-but-not-your-male-mates/news-Story/bbc21ee6883bd07bfbbbe76a0c8ca54c.

[18]

Xuan Lu, Wei Ai, Xuanzhe Liu, Qian Li, Ning Wang, Gang Huang, and Qiaozhu Mei. 2016. Learning from the ubiquitous language: an empirical analysis of emoji usage of smartphone users. In Proceedings of the International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp. ACM, Heidelberg, Germany, 770--780.

Digital Library

[19]

Xuan Lu, Yanbin Cao, Zhenpeng Chen, and Xuanzhe Liu. 2018. A First Look at Emoji Usage on GifHub: An Empirical Study. CoRR abs/1812.04863 (2018).

[20]

Puneet Singh Ludu. 2014. Inferring gender of a Twitter user using celebrities it follows. CoRR abs/1405.6667(2014).

[21]

Hannah Jean Miller, Daniel Kluver, Jacob Thebault-Spieker, Loren G. Terveen, and Brent J. Hecht. 2017. Understanding Emoji Ambiguity in Context: The Role of Text in Emoji-Related Miscommunication. In Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM. AAAI Press, Montréal, Québec, Canada, 152--161.

[22]

Dong Nguyen, Dolf Trieschnigg, A. Seza Dogruöz, Rilana Gravel, Mariët Theune, Theo Meder, and Franciska de Jong. 2014. Why Gender and Age Prediction from Tweets is Hard: Lessons from a Crowdsourcing Experiment. In 25th International Conference on Computational Linguistics, Proceedings of the Conference, COLING. ACL, Dublin, Ireland, 1950--1961.

[23]

Claudia Peersman, Walter Daelemans, and Leona Van Vaerenbergh. 2011. Predicting age and gender in online social networks. In Proceedings of the 3rd International Workshop on Search and Mining User-Generated Contents, SMUC. ACM, Glasgow, United Kingdom, 37--44.

Digital Library

[24]

Daniel Preotiuc-Pietro, Wei Xu, and Lyle H. Ungar. 2016. Discovering User Attribute Stylistic Differences via Paraphrasing. In Proceedings of the Thirtieth Conference on Artificial Intelligence., AAAI Press, Phoenix, Arizona, USA, 3030--3037.

[25]

Lucía Santamaría and Helena Mihaljevic. 2018. Comparison and benchmark of name-to-gender inference services. PeerJ Computer Science (2018).

[26]

Maarten Sap, Gregory J. Park, Johannes C. Eichstaedt, Margaret L. Kern, David Stillwell, Michal Kosinski, Lyle H. Ungar, and H. Andrew Schwartz. 2014. Developing Age and Gender Predictive Lexica over Social Media. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP. ACL, Doha, Qatar, 1146--1151.

[27]

Shutterstock. 2015. The Psychology Behind Why We Share on Social Media. Retrieved January 13, 2015 from https://www.shutterstock.com/blog/the-psychology-behind-why-we-share-on-social-media

[28]

Shutterstock. 2017. 6 Types of Images That Elicit an Emotional Response. Retrieved May 5, 2017 from https://www.shutterstock.com/blog/6-types-of-images-that-elicit-an-emotional-response

[29]

Shaomei Wu, Jeffrey Wieland, Omid Farivar, and Julie Schiller. 2017. Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW. ACM, Portland, OR, USA, 1180--1192.

Digital Library

Cited By

Tricomi PFacciolo LApruzzese GConti MShehab MFernandez MLi N(2023)Attribute Inference Attacks in Online Multiplayer Video Games: A Case Study on DOTA2Proceedings of the Thirteenth ACM Conference on Data and Application Security and Privacy10.1145/3577923.3583653(27-38)Online publication date: 24-Apr-2023
De SImine A(2022)Privacy Risk Analysis of Online Social NetworksundefinedOnline publication date: 7-Mar-2022
Eidizadehakhcheloo SAlipour Pijani BImine ARusinowitch M(2021)Divide-and-Learn: A Random Indexing Approach to Attribute Inference Attacks in Online Social NetworksData and Applications Security and Privacy XXXV10.1007/978-3-030-81242-3_20(338-354)Online publication date: 19-Jul-2021
Show More Cited By

Index Terms

You are what emojis say about your pictures: language-independent gender inference attack on Facebook
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Inferring attributes with picture metadata embeddings

Users in online social networks are vulnerable to attribute inference attacks due to some published data. Thus, the picture owner's gender has a strong influence on individuals' emotional reactions to the photo. In this work, we present a graph-...
Gender Inference for Facebook Picture Owners
Trust, Privacy and Security in Digital Business
Abstract
Social media such as Facebook provides a new way to connect, interact and learn. Facebook allows users to share photos and express their feelings by using comments. However, its users are vulnerable to attribute inference attacks where an attacker ...
Emojis influence emotional communication, social attributions, and information processing
Abstract
Many emojis symbolize nonverbal cues that are used during face-to-face communication. Despite their popularity, few studies have examined how emojis influence digital interactions. The present study addresses this gap by measuring the ...
Highlights
- Emojis convey information about the sender's affect.
- Senders that use positive ...

Comments

comments powered by Disqus.

Information & Contributors

Information

Published In

SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

March 2020

2348 pages

ISBN:9781450368667

DOI:10.1145/3341105

Conference Chairs:
Chih-Cheng Hung
Kennesaw State University
,
Tomas Cerny
Baylor University
,
Program Chairs:
Dongwan Shin
New Mexico Tech
,
Alessio Bechini
University of Pisa, Italy

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAC '20

Sponsor:

SIGAPP

SAC '20: The 35th ACM/SIGAPP Symposium on Applied Computing

March 30 - April 3, 2020

Brno, Czech Republic

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
280
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)4

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tricomi PFacciolo LApruzzese GConti MShehab MFernandez MLi N(2023)Attribute Inference Attacks in Online Multiplayer Video Games: A Case Study on DOTA2Proceedings of the Thirteenth ACM Conference on Data and Application Security and Privacy10.1145/3577923.3583653(27-38)Online publication date: 24-Apr-2023
De SImine A(2022)Privacy Risk Analysis of Online Social NetworksundefinedOnline publication date: 7-Mar-2022
Eidizadehakhcheloo SAlipour Pijani BImine ARusinowitch M(2021)Divide-and-Learn: A Random Indexing Approach to Attribute Inference Attacks in Online Social NetworksData and Applications Security and Privacy XXXV10.1007/978-3-030-81242-3_20(338-354)Online publication date: 19-Jul-2021
Pijani BImine ARusinowitch M(2020)Inferring attributes with picture metadata embeddingsACM SIGAPP Applied Computing Review10.1145/3412816.341281920:2(36-45)Online publication date: 27-Jul-2020

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

Alternative Proxy