Performing Range Aggregate Queries in Stream Data Warehouse

Abstract

The number of various areas of everyday life where data warehouse systems find their application grows rapidly. They are used not only for business purposes, as it used to be a few years ago, but also in many various domains where huge and rapidly changing data volumes must be stored and processed. More and more often we think about such data volumes as endless data streams which require continuous loading and processing. The concept of data streams results in emerging a new type of data warehouse - a steam data warehouse. Stream data warehouse, when compared to standard data warehouse, differs in many ways, the examples can be a continuous ETL process or data mining models which are always kept upto- date. The process of building stream data warehouse poses many new challenges to algorithms and memory structures designers. The most important concern efficiency and memory complexity of the designed solutions. In this paper we present a stream data warehouse cooperating with a network of sensors monitoring utilities consumption. We focus on a problem of performing range aggregate queries over the sensors and processing data streams generated by the chosen objects.We present a solution which, basing on the results of our previous work, integrate a dedicated memory structure with a spatial aggregating index. The paper includes also practical tests results which show high efficiency and scalability of the proposed solution.