Abstract
Sentiment analysis is the process of determining whether a text or a writing is positive, negative, or neutral. A lot of research has been done to improve the accuracy of sentiment analysis methods, varying from simple linear models to more complex deep neural network models. Lately, the transformer-based model showed great success in sentiment analysis and was considered as the state-of-the-art model for various languages (English, german, french, Turk, Arabic, etc.). However, the accuracy for Arabic sentiment analysis still needs improvements especially in tokenization level during data processing. In fact, the Arabic language imposes many challenges, due to its complex structure, various dialects, and resource scarcity. The improvement of the proposed approach consists of integrating an Arabic BERT tokenizer instead of a basic BERT Tokenizer. Various tests were carried out with different instances (dialect and standard). We used hyperparameters optimization by random search method to obtain the best result with different datasets. The experimental study proves the efficiency of the proposed approach in terms of classification quality and accuracy compared to Arabic BERT and AraBERT models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dragoni, M., Poria, S., Cambria, E.: OntoSenticNet: a commonsense ontology for sentiment analysis. IEEE Intell. Syst. 33(3), 77–85 (2018)
Oueslati, O., Cambria, E., Ben HajHmida, M., Ounelli, H.: A review of sentiment analysis research in Arabic language. Future Generation Comput. Syst. 112, 408–430 (2020)
Abdul-Mageed, M., Diab, M., Korayem, M.: Subjectivity and sentiment analysis of modern standard Arabic. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics Human Language Technologies short papers-Volume 2 Association for Computational Linguistics, pp. 587–591 (2011)
Shoukry, A., Rafea, A.: Sentence-level Arabic sentiment analysis. In: Collaboration Technologies and Systems (CTS) 2012 International Conference on IEEE, pp. 546-550 (2012)
Zaghouani, W.: Critical survey of the freely available Arabic corpora (2017). https://arxiv.org/abs/1702.07835
Imran, A., Faiyaz, M., Akhtar, F.: An enhanced approach for quantitative prediction of personality in Facebook posts. Int. J. Educ. Manage. Eng. (IJEME) 8(2), 8–19 (2018)
Alsayat, A., Elmitwally, N.: A comprehensive study for Arabic sentiment analysis (challenges and applications). Egyptian Inform. J. 21(1), 7–12 (2020). Elsevier
Al-Rubaiee, H., Qiu, R., Li, D.: Identifying Mubasher software products through sentiment analysis of Arabic tweets. In: 2016 International Conference on Industrial Informatics and Computer Systems (CIICS). IEEE, pp. 1–6 (2016)
Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)
Ghanem, B., Karoui, J., Benamara, F., Moriceau, V., Rosso, P.: IDAT at FIRE2019: overview of the track on irony detection in Arabic tweets. Proceedings of the 11th Forum for Information Retrieval Evaluation, pp. 10–13 (2019)
Shoukry, A., Rafea, A.: Sentence-level Arabic sentiment analysis. In: Collaboration Technologies and Systems (CTS), 2012 International Conference on IEEE, pp. 546–550 (2012)
Alhumoud, S., Albuhairi, T., Alohaideb, W.: Hybrid sentiment analyser for Arabic tweets using R. In: 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management. IC3K. IEEE, pp. 417–424 (2015)
Safaya, A., Abdullatif, M., Yuret, D.: KUISAIL at SemEval-2020 Task 12: BERT-CNN for offensive speech identification in social media (2020). arXiv:2007.13184v1 [cs.CL]
ElJundi, O., Antoun, W., El Droubi, N., Hajj, H., El-Hajj, W., Shaban, K.: Hulmona: the universal language model in Arabic. In: Proceedings of the Fourth Arabic Natural Language Processing Workshop. 68–77, (2019)
Antoun, W., Baly, F., and Hajj, H.: AraBERT: transformer-based model for Arabic language understanding (2020). arXiv preprint arXiv:2003.00104
Zahran, M.A., Magooda, A., Mahgoub, A.Y., Raafat, H., Rashwan, M., Atyia, A.: Word representations in vector space and their applications for Arabic. In: Gelbukh, A. (ed.) CICLing 2015. LNCS, vol. 9041, pp. 430–443. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18111-0_32
Grave, E., Bojanowski, P., Gupta, P., Joulin, A., Mikolov, T.: Learning word vectors for 157 languages. In: Proceedings of the International Conference on Language Resources and Evaluation, LREC 2018 (2018)
Devlin, J., Chang, Ming-W., Lee, K., Toutanova, Kristina.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, June. Association for Computational Linguistics, vol. 1, pp. 4171–4186 (2019)
Saidi, R., Jarray, F., Mansour, M.: A BERT based approach for Arabic POS tagging. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2021. LNCS, vol. 12861, pp. 311–321. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85030-2_26
Attia, M.: Arabic tokenization system. In: Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, pp. 65-72 (2007)
Abdelali, A., Darwish, K., Durrani, N., Mubarak, H.: Farasa: a fast and furious segmenter for Arabic. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. Association for Computational Linguistics, pp. 11–16 (2016). https://www.aclweb.org/anthology/N16-3003https://doi.org/10.18653/v1/N16-3003
Nabil, M., Aly, M., Atiya, A.: ASTD: Arabic sentiment tweets dataset. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2515-2519 (2015)
Elnagar, A., Khalifa, Y.S., Einea, A.: Hotel Arabic-reviews dataset construction for sentiment analysis applications. In: Shaalan, K., Hassanien, A.E., Tolba, F. (eds.) Intelligent Natural Language Processing: Trends and Applications. SCI, vol. 740, pp. 35–52. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-67056-0_3
Aly, M., Atiya, A.: LABR: A Large Scale Arabic Book Reviews Dataset. Sofia, Bulgaria, Meetings of the Association for Computational Linguistics (ACL) At (2013)
Alomari, K.M., ElSherif, H.M., Shaalan, K.: Arabic tweets sentimental analysis using machine learning. In: Benferhat, S., Tabia, K., Ali, M. (eds.) IEA/AIE 2017. LNCS (LNAI), vol. 10350, pp. 602–610. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60042-0_66
Baly, R., Khaddaj, A., Hajj, H., El-Hajj, W., Shaban, B.: Arsentd-lev: a multi-topic corpus for target-based sentiment analysis in Arabic levantine tweets (2019). arXiv preprint arXiv:1906.01830
Eskander, R., Rambow, O.: SLSA: a sentiment lexicon for standard Arabic. EMNLP 2545–2550 (2015)
Abdelghani, D., Mohamed, A.E., Junwei, Z.: Arabic sentiment classification using convolutional neural network and differential evolution algorithm. Comput. Intell. Neurosci. (2019)
Harrat, S., Meftouh, K., Smaili, K.: Machine translation for Arabic dialects (survey). Inf. Process. Manage. 56(2), 262–273 (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chouikhi, H., Chniter, H., Jarray, F. (2021). Arabic Sentiment Analysis Using BERT Model. In: Wojtkiewicz, K., Treur, J., Pimenidis, E., Maleszka, M. (eds) Advances in Computational Collective Intelligence. ICCCI 2021. Communications in Computer and Information Science, vol 1463. Springer, Cham. https://doi.org/10.1007/978-3-030-88113-9_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-88113-9_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88112-2
Online ISBN: 978-3-030-88113-9
eBook Packages: Computer ScienceComputer Science (R0)