Abstract
With tens if not hundreds of billions of logical statements, the Linked Open Data (LOD) is one of the biggest knowledge bases ever built. As such it is a gigantic source of information for applications in various domains, but also given its size an ideal test-bed for knowledge representation and reasoning, heterogeneous nature, and complexity.
However, making use of this unique resource has proven next to impossible in the past due to a number of problems, including data collection, quality, accessibility, scalability, availability and findability. The LOD Laundromat and LOD Lab are recent infrastructures that addresses these problems in a systematic way, by automatically crawling, cleaning, indexing, analysing and republishing data in a unified way. Given a family of simple tools, LOD Lab allows researchers to query, access, analyse and manipulate hundreds of thousands of data documents seamlessly, e.g. facilitating experiments (e.g. for reasoning) over hundreds of thousands of (possibly integrated) datasets based on content and meta-data.
This chapter provides the theoretical basis and practical skills required for making ideal use of this large scale experimental platform. First we study the problems that make it so hard to work with Semantic Web data in its current form. We’ll also propose generic solutions and introduce the tools the reader needs to get started with their own experiments on the LOD Cloud.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For example, Virtuoso by default limits both the result set size and the number of rows within a sorting operation.
- 2.
- 3.
- 4.
Version 2.0.14, retrieved from http://librdf.org/raptor/rapper.html.
- 5.
- 6.
See http://ckan.org/.
- 7.
- 8.
- 9.
- 10.
- 11.
The default ranking function in ElasticSearch. See https://www.elastic.co/guide/en/elasticsearch/guide/current/scoring-theory.html.
- 12.
References
Hogan, A., Harth, A., Passant, A., Decker, S., Polleres, A.: Weaving the pedantic web. In: Linked Data on the Web Workshop (2010)
Hogan, A., Umbrich, J., Harth, A., Cyganiak, R., Polleres, A., Decker, S.: An empirical survey of linked data conformance. Web Semant.: Sci. Serv. Agents World Wide Web 14, 14–44 (2012)
Beek, W., Rietveld, L., Bazoobandi, H.R., Wielemaker, J., Schlobach, S.: LOD laundromat: a uniform way of publishing other people’s dirty data. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 213–228. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11964-9_14
Rietveld, L., Verborgh, R., Beek, W., Vander Sande, M., Schlobach, S.: Linkeddata-as-a-service: the semantic web redeployed. In: Gandon, F., Sabou, M., Sack, H., d’Amato, C., Cudré-Mauroux, P., Zimmermann, A. (eds.) ESWC 2015. LNCS, vol. 9088, pp. 471–487. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18818-8_29
Verborgh, R., et al.: Querying datasets on the web with high availability. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 180–196. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11964-9_12
Ermilov, I., Martin, M., Lehmann, J., Auer, S.: Linked open data statistics: collection and exploitation. In: Klinov, P., Mouromtsev, D. (eds.) KESW 2013. CCIS, vol. 394, pp. 242–249. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41360-5_19
Auer, S., Demter, J., Martin, M., Lehmann, J.: LODStats – an extensible framework for high-performance dataset analytics. In: Teije, A., Völker, J., Handschuh, S., Stuckenschmidt, H., d’Acquin, M., Nikolov, A., Aussenac-Gilles, N., Hernandez, N. (eds.) EKAW 2012. LNCS (LNAI), vol. 7603, pp. 353–362. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33876-2_31
Buil-Aranda, C., Hogan, A., Umbrich, J., Vandenbussche, P.-Y.: SPARQL web-querying infrastructure: ready for action? In: Alani, H., et al. (eds.) ISWC 2013. LNCS, vol. 8219, pp. 277–293. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41338-4_18
Cheng, G., Gong, S., Qu, Y.: An empirical study of vocabulary relatedness and its application to recommender systems. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 98–113. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25073-6_7
Ge, W., Chen, J., Hu, W., Qu, Y.: Object link structure in the semantic web. In: Aroyo, L., Antoniou, G., Hyvönen, E., Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC 2010. LNCS, vol. 6089, pp. 257–271. Springer, Heidelberg (2010). doi:10.1007/978-3-642-13489-0_18
Alexander, K., Cyganiak, R., Hausenbals, M., Zhao, J.: Describing linked datasets with the VoID vocabulary, March 2011. http://www.w3.org/TR/2011/NOTE-void-20110303/
Millard, I., Glaser, H., Salvadores, M., Shadbolt, N.: Consuming multiple Linked Data sources: challenges and experiences. In: First International Workshop on Consuming Linked Data (COLD), November 2010
Prud’hommeaux, E., Buil-Aranda, C.: SPARQL 1.1 Federated Query (2013). http://www.w3.org/TR/sparql11-federated-query/
Fernández, J.D., Martínez-Prieto, M.A., Gutiérrez, C., Polleres, A., Arias, M.: Binary RDF representation for publication and exchange (HDT). Web Seman.: Sci. Serv. Agents World Wide Web 19, 22–41 (2013)
Mäkelä, E.: Aether – generating and viewing extended void statistical descriptions of RDF datasets. In: Presutti, V., Blomqvist, E., Troncy, R., Sack, H., Papadakis, I., Tordai, A. (eds.) ESWC 2014. LNCS, vol. 8798, pp. 429–433. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11955-7_61
Callahan, A., Cruz-Toledo, J., Ansell, P., Dumontier, M.: Bio2RDF release 2: improved coverage, interoperability and provenance of life science linked data. In: Cimiano, P., Corcho, O., Presutti, V., Hollink, L., Rudolph, S. (eds.) ESWC 2013. LNCS, vol. 7882, pp. 200–212. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38288-8_14
Bazoobandi, H.R., Rooij, S., Urbani, J., Teije, A., Harmelen, F., Bal, H.: A compact in-memory dictionary for RDF data. In: Gandon, F., Sabou, M., Sack, H., d’Amato, C., Cudré-Mauroux, P., Zimmermann, A. (eds.) ESWC 2015. LNCS, vol. 9088, pp. 205–220. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18818-8_13
Christophides, V., Efthymiou, V., Stefanidis, K.: Entity Resolution in the Web of Data. Morgan & Claypool Publishers, San Rafael (2015)
Rietveld, L., Beek, W., Schlobach, S.: LOD lab: experiments at LOD scale. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 339–355. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25010-6_23
Schmachtenberg, M., Bizer, C., Paulheim, H.: Adoption of the linked data best practices in different topical domains. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 245–260. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11964-9_16
Isele, R., Umbrich, J., Bizer, C., Harth, A.: LDSpider: an open-source crawling framework for the Web of Linked Data. In: 9th International Semantic Web Conference. Citeseer (2010)
Harris, S., Seaborne, A.: SPARQL 1.1 query language, March 2013
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Beek, W., Rietveld, L., Ilievski, F., Schlobach, S. (2017). LOD Lab: Scalable Linked Data Processing. In: Pan, J., et al. Reasoning Web: Logical Foundation of Knowledge Graph Construction and Query Answering. Reasoning Web 2016. Lecture Notes in Computer Science(), vol 9885. Springer, Cham. https://doi.org/10.1007/978-3-319-49493-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-49493-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49492-0
Online ISBN: 978-3-319-49493-7
eBook Packages: Computer ScienceComputer Science (R0)