TsQuality: Measuring Time Series Data Quality in Apache IoTDB
Authors: 
Yuanhui Qiu, Chenguang Fang, Shaoxu Song, Xiangdong Huang, Chen Wang, Jianmin WangAuthors Info & Claims
Yuanhui Qiu, Chenguang Fang, Shaoxu Song, Xiangdong Huang, Chen Wang, Jianmin WangAuthors Info & ClaimsPages 3982 - 3985
Abstract
Time series has been found with various data quality issues, e.g., owing to sensor failure or network transmission errors in the Internet of Things (IoT). It is highly demanded to have an overview of the data quality issues on the millions of time series stored in a database. In this demo, we design and implement TsQuality, a system for measuring the data quality in Apache IoTDB. Four time series data quality measures, completeness, consistency, timeliness, and validity, are implemented as functions in Apache IoTDB or operators in Apache Spark. These data quality measures are also interpreted by navigating dirty points in different granularity. It is also well-integrated with the big data eco-system, connecting to Apache Zeppelin for SQL query, and Apache Superset for an overview of data quality.
References
[1]
X. Ding, H. Wang, J. Su, Z. Li, J. Li, and H. Gao. Cleanits: A data cleaning system for industrial time series. Proc. VLDB Endow., 12(12):1786--1789, 2019.
[2]
C. Fang, S. Song, and Y. Mei. On repairing timestamps for regular interval time series. Proc. VLDB Endow., 15(9):1848--1860, 2022.
[3]
M. K. Shende, A. E. Feijóo-Lorenzo, and N. D. Bokde. cleants: Automated (automl) tool to clean univariate time series at microscales. Neurocomputing, 500:155--176, 2022.
[4]
S. Song, F. Gao, A. Zhang, J. Wang, and P. S. Yu. Stream data cleaning under speed and acceleration constraints. ACM Trans. Database Syst., 46(3):10:1--10:44, 2021.
[5]
S. Song and A. Zhang. Iot data quality. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19--23, 2020, pages 3517--3518. ACM, 2020.
[6]
Y. Su, Y. Gong, and S. Song. Time series data validity. In ACM SIGMOD International Conference on Management of Data, SIGMOD, 2023.
[7]
C. Wang, J. Qiao, X. Huang, S. Song, H. Hou, T. Jiang, L. Rui, J. Wang, and J. Sun. Apache IoTDB: A time series database for IoT applications. In ACM SIGMOD International Conference on Management of Data, SIGMOD, 2023.
Recommendations
Apache IoTDB: time-series database for internet of things
The amount of time-series data that is generated has exploded due to the growing popularity of Internet of Things (IoT) devices and applications. These applications require efficient management of the time-series data on both the edge and cloud side ...
Comments
Information & Contributors
Information
Published In
Publisher
VLDB Endowment
Publication History
Published: 01 August 2023
Published in PVLDB Volume 16, Issue 12
Check for updates
Badges
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 126Total Downloads
- Downloads (Last 12 months)74
- Downloads (Last 6 weeks)5
Reflects downloads up to 22 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in