Application-driven metrics of online action recognition

Abstract

This work tries to understand the gap between research and applications by evaluating the performance of some action classification methods in a video streaming scenario, which is essential for several applications. An overview of the action recognition area, its main tasks, methods and metrics are presented. Based on the current situation of the area and considering that several action classification applications need to run over a video streaming, two questions are raised: a) How is the performance of classification models in a streaming environment? b) What are the best metrics to evaluate a streaming environment? To answer them, we evaluated the performance of some action classification methods in a video streaming scenario. The main metrics and evaluation protocols used in the literature were applied and adapted to the case of streaming when necessary. The results obtained fail to satisfactorily capture the actual performance of the methods, pointing out the necessity of new and more tangible metrics. Therefore, new forms of assessment are studied and proposed. Finally, we provide an overview of the challenges currently faced in the area of action recognition and some insight for those who wish to use its methods in current solutions.