Análise de dados científicos baseada em algoritmos de indexação bitmap

Abstract

Computer simulations in large-scale often consume and produce a large volume of raw data files, which can be presented in different formats. Users usually need to analyze domain-specific data based on data elements related through multiple files generated along the computer simulation execution. Different existing solutions, like FastBit and NoDB, intend to support this analysis by indexing raw data in order to allow direct access to specific elements in raw data files regions of interest. However, those solutions are limited to analyze a single raw data file at once, while they are used only after computer simulation execution. The ARMFUL architecture proposes a solution capable of guarantee dataflow management, record related raw data elements in a provenance database and combine techniques of raw data file analysis at runtime. Through a data model that supports integration between computer simulation execution data and domain data, the architecture allows for queries on data elements related by multiple files. This dissertation proposes the implementation of instances of raw data indexing and query processor components presented by ARMFUL architecture, aiming to reduce the elapsed time of data ingestion in the provenance database and support raw data exploratory analysis.