ProvDeploy: apoio à coleta de dados de proveniência em scripts de execução de códigos científicos

Abstract

Large-scale scientific applications are characterized by using many software libraries and producing large amounts of data through scripts. Usually, those scripts require High Performance Computing (HPC) on scientific code execution. The complexity to prepare those scripts for execution is high due to the amount of third-party software, that compose the software stack. Container-based virtualization helps on configuration and execution of script applications. However, the use of containers in HPC environments faces challenges of environment security and overhead on the application execution. Another complicating component that increases the software stack size are the provenance data capture services. The capture of provenance data provides analytical and monitoring capabilities. To ease the adoption of provenance data capture services in scripts using HPC environments, this dissertation presents P rovDeploy. The main goal of P rovDeploy is to guide the composition of the script’s virtualization, for execution in HPC with containers, integrating the provenance data capture services in a systematic way and with a few configuration steps. The experiments performed with P rovDeploy over diverse scripts of scientific code showed the reduction of the necessary effort. The adoption of provenance data capture services in HPC environments has been facilitated and there was no overhead on the performance of applications executed in containers with P rovDeploy.