Machine learning and deep learning models for tone-mapped image quality assessment

Abstract

High dynamic range (HDR) images are increasingly more common in today consumer applications. Such images need to be processed by so-called tone-mapping operators (TMOs), in order to be properly exhibited on standard display devices. Objective tone-mapped image quality assessment (TMIQA) metrics are desirable to aid selecting the best TMO for an HDR image. Only few relatively small benchmark databases exist for TMIQA task, namely ESPL-LIVE and TMID. We present a new database for TMIQA, called PBTDB, which contains approximately 175000 samples. Sample quality is labeled by four TMIQA metrics. We also conduct subjective experiments to assess quality of 3009 PBTDB samples. Current state-of-the-art noreference (NR) IQA metrics, which are the focus of our study, are unable to reliably assess the quality of tone-mapped images that do not come from specific databases. We design cross-dataset experiments to investigate two approaches for developing new generic NR TMIQA metrics. In the first one, we use ESPL-LIVE samples to train machine-learning based regression models that combine scores from multiple IQA metrics into a single quality score. The best performing model outputs quality scores that correlate relatively poorly with TMID sample mean opinion scores (PLCC of 0.65 and SRCC of 0.55). In the second one, we use PBTDB samples to train deep learning architectures, and ESPL-LIVE and TMID samples to test them. Models can achieve moderate performance in TMID, but all perform poorly in ESPL-LIVE (best performances: PLCC of 0.48 and SRCC of 0.43 in ESPL-LIVE, and PLCC of 0.79 and SRCC of 0.72 in TMID). Performance is heavily influenced by quality score types (subjective versus objective) used as targets to train the models