Uma adaptação do escalonador estático da biblioteca dataflow sucuri para computação in-situ

Abstract

In the dataflow computation model, instructions or tasks are executed according to data dependencies, instead of following program order, thus allowing parallelism to be exposed naturally. Sucuri is a dataflow library for Python that allows each user to specify their application as a dependency graph and execute it transparently in clusters of multicores, while taking care of scheduling issues. Recent trends in Fog and In-situ computing assume that storage and network devices will be equipped with processing elements that usually have lower power consumption and performance. An important decision for such systems is whether to move data to traditional processors (paying the communication costs), or to perform the computation where the data sits, using a potentially slower processor. Hence, runtime environments that deal with that trade-off are of extreme importance. This work presents a study on different factors that should be considered when running dataflow applications in a In-situ environment. We use Sucuri to manage the execution in a small system with a regular PC and a Parallella board, emulating a smart storage, and a set of experiments was performed to show how data transfer size, network latency, packet loss rates and computational complexity affect execution time when outsourcing computation to the smart storage. Then, a static scheduling solution is presented, allowing Sucuri to take the best decision where to execute the application, avoiding outsourcing when there would be no performance gains.