Abstract
Recent approaches for knowledge-graph embeddings aim at connecting quantitative data structures used in machine learning to the qualitative structures of logics. Such embeddings are of a hybrid nature, they are data models that also exhibit conceptual structures inherent to logics. One motivation to investigate embeddings is to design conceptually adequate machine learning (ML) algorithms that learn or incorporate ontologies expressed in some logic. This paper investigates a new approach to embedding ontologies into geometric models that interpret concepts by geometrical structures based on convex cones. The ontologies are assumed to be represented in an orthologic, a logic with a full (ortho)negation. As a proof of concept this cone-based embedding was implemented within two ML algorithms for weak supervised multi-label learning. Both algorithms rely on cones but the first addresses ontologies expressed in classical propositional logic whereas the second addresses a weaker propositional logic, namely a weak orthologic that does not fulfil distributivity. The algorithms were evaluated and showed promising results that call for investigating other (sub)classes of cones and developing fine-tuned algorithms based on them.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Availability of data and material
All data used for the experiments are publicly available (as indicated by pointers to the literature in the paper).
Code availability
Software code for the algorithms is available on https://github.com/mleemhuis/AMAI_conelearning
References
Ashburner, M., Ball, C.A., Blake, J.A., Botstein, D., Butler, H., Cherry, J.M., Davis, A.P., Dolinski, K., Dwight, S.S., Eppig, J.T., et al.: Gene ontology: tool for the unification of biology. Nature genetics 25(1), 25 (2000). https://doi.org/10.1038/75556
Baader, F., Calvanese, D., McGuinness, D., Nardi, D., Patel-Schneider, P.F. (eds.): The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press (2003)
Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: C.J.C. Burges, L. Bottou, Z. Ghahramani, K.Q. Weinberger (eds.) Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States., pp. 2787–2795 (2013). http://papers.nips.cc/paper/5071-translating-embeddings-for-modeling-multi-relational-data
Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998). https://doi.org/10.1023/a:1009715923555
Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST) 2(3), 1–27 (2011)
Changpinyo, S., Chao, W.L., Sha, F.: Predicting visual exemplars of unseen classes for zero-shot learning. 2017 IEEE International Conference on Computer Vision (ICCV) pp. 3496–3505 (2017)
Conradie, W., Palmigiano, A., Robinson, C., Wijnberg, N.: Non-distributive logics: from semantics to meaning. arXiv e-prints arXiv:2002.04257 (2020)
Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., Adam, H.: Large-scale object classification using label relation graphs. In: D. Fleet, T. Pajdla, B. Schiele, T. Tuytelaars (eds.) Computer Vision — ECCV 2014, Lecture Notes in Computer Science, vol. 8689, pp. 48–64. Springer International Publishing (2014). https://doi.org/10.1007/978-3-319-10590-1_4
Fofanova, T.: Encyclopedia of Mathematics, chap. Semi-modular lattice. Springer Science+Business Media B.V. / Kluwer Academic Publishers (2001)
Gärdenfors, P.: Conceptual Spaces: The Geometry of Thought. The MIT Press, Cambridge, Massachusetts (2000)
Garg, D., Ikbal, S., Srivastava, S.K., Vishwakarma, H., Karanam, H., Subramaniam, L.V.: Quantum embedding of knowledge for reasoning. In: H. Wallach, H. Larochelle, A. Beygelzimer, F. Alché-Buc, E. Fox, R. Garnett (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)
Gibaja, E., Ventura, S.: Multi-label learning: a review of the state of the art and ongoing research. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 4(6), 411–444 (2014). https://doi.org/10.1002/widm.1139
Goldblatt, R.I.: Semantic analysis of orthologic. Journal of Philosophical Logic 3(1), 19–35 (1974). https://doi.org/10.1007/BF00652069
Gutiérrez-Basulto, V., Schockaert, S.: From knowledge graph embedding to ontology embedding? an analysis of the compatibility between vector space representations and rules. In: M. Thielscher, F. Toni, F. Wolter (eds.) Principles of Knowledge Representation and Reasoning: Proceedings of the Sixteenth International Conference, KR 2018, Tempe, Arizona, 30 October - 2 November 2018., pp. 379–388. AAAI Press (2018)
Ji, S., Tang, L., Yu, S., Ye, J.: Extracting shared subspace for multi-label classification. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’08, pp. 381—389. Association for Computing Machinery, New York, NY, USA (2008). https://doi.org/10.1145/1401890.1401939.
Kulmanov, M., Liu-Wei, W., Yan, Y., Hoehndorf, R.: El embeddings: Geometric construction of models for the description logic el++. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) (2019)
Leemhuis, M., Özçep, Ö.L., Wolter, D.: Multi-label learning with a cone-based geometric model. In: M. Alam, T. Braun, B. Yun (eds.) Ontologies and Concepts in Mind and Machine - 25th International Conference on Conceptual Structures, ICCS 2020, Bolzano, Italy, September 18-20, 2020, Proceedings, Lecture Notes in Computer Science, vol. 12277, pp. 177–185. Springer (2020). https://doi.org/10.1007/978-3-030-57855-8_13.
Matoušek, J. (ed.): Lectures on Discrete Geometry. Springer New York (2002). https://doi.org/10.1007/978-1-4613-0039-7
Mehran Kazemi, S., Poole, D.: SimplE Embedding for Link Prediction in Knowledge Graphs. arXiv e-prints arXiv:1802.04868 (2018)
Özçep, Ö.L., Leemhuis, M., Wolter, D.: Cone semantics for logics with negation. In: C. Bessiere (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020 [scheduled for July 2020, Yokohama, Japan, postponed due to the Corona pandemic], pp. 1820–1826. ijcai.org (2020). https://doi.org/10.24963/ijcai.2020/252.
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) Machine Learning and Knowledge Discovery in Databases, pp. 254–269. Springer, Berlin Heidelberg, Berlin, Heidelberg (2009)
Redei, M.: Quantum Logic in Algebraic Approach. Fundamental Theories of Physics. Springer Netherlands (1998). https://books.google.de/books?id=7ltemAP8MDUC
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NJ (1997)
Śmieja, M., Tabor, J., Spurek, P.: SVM with a neutral class. Pattern Analysis and Applications 22(2), 573–582 (2017). https://doi.org/10.1007/s10044-017-0654-3
Uzilov, A.V., Keegan, J.M., Mathews, D.H.: Detection of non-coding rnas on the basis of predicted secondary structure formation free energy change. BMC Bioinformatics 7(1), 173 (2006). https://doi.org/10.1186/1471-2105-7-173
Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Machine Learning 73(2), 185 (2008). https://doi.org/10.1007/s10994-008-5077-3
Wan, S.P., Xu, J.H.: A multi-label classification algorithm based on triple class support vector machine. In: 2007 International Conference on Wavelet Analysis and Pattern Recognition, vol. 4, pp. 1447–1452 (2007). https://doi.org/10.1109/ICWAPR.2007.4421677
Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29(12), 2724–2743 (2017). https://doi.org/10.1109/TKDE.2017.2754499
Xian, Y., Lampert, C.H., Schiele, B., Akata, Z.: Zero-shot learning–a comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 2251–2265 (2017)
Yih, W., Zweig, G., Platt, J.C.: Polarity inducing latent semantic analysis. In: J. Tsujii, J. Henderson, M. Pasca (eds.) Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, July 12-14, 2012, Jeju Island, Korea, pp. 1212–1222. ACL (2012). http://www.aclweb.org/anthology/D12-1111
Zhang, M.L., Li, Y.K., Liu, X.Y., Geng, X.: Binary relevance for multi-label learning: an overview. Frontiers of Computer Science 12(2), 191–202 (2018). https://doi.org/10.1007/s11704-017-7031-7
Funding
Open Access funding enabled and organized by Projekt DEAL. The research of Mena Leemhuis is partly funded by AI-Lab Lübeck https://ai-lab.digital-hub-luebeck.de/en and by the BMBF-funded project SmaDi. Diedrich Wolter acknowledges financial support by Technologie-Allianz Oberfranken and the BMBF-funded “Dependable Intelligent Systems Lab”.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
This paper is a considerably extended version of the conference paper [17].
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Leemhuis, M., Özçep, Ö.L. & Wolter, D. Learning with cone-based geometric models and orthologics. Ann Math Artif Intell 90, 1159–1195 (2022). https://doi.org/10.1007/s10472-022-09806-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-022-09806-1