Shima Mohammadi, Mohsen Jenadeleh, Michela Testolina, Jon Sneyers, Touradj Ebrahimi, Dietmar Saupe, Joao Ascenso
Instituto de Telecomunicações, Instituto Superior Técnico, University of Lisbon
Download the paper
This paper presents a novel double stimulus subjective assessment methodology for the evaluation of high-quality images aimed at addressing the limitations of existing protocols to capture near visually lossless perceptual quality differences. The In-place Double Stimulus Quality Scale (IDSQS) allows subjects to alternately view a reference and a distorted image at the same spatial location, facilitating a more intuitive detection of differences in quality. A large-scale crowdsourcing study employing this methodology was conducted, generating a comprehensive public dataset to evaluate perceived image quality across several compression algorithms and distortion levels. Another novelty is the fitting of the Beta distribution to the quality scores obtained to study their variability and thus, the agreement between subjects. Our findings demonstrate the effectiveness of the IDSQS methodology in achieving high correlation with more precise subjective evaluation benchmarks.
This repository contains data from a study conducted as part of JPEG AIC-3 activities using Inplace Double Stimulus Continous Quality Scale (IDSQS). For more information about the JPEG AIC-3, see: JPEG AIC-3.
The study was conducted on Amazon Mechanical Turk (MTurk). The data includes responses from crowd workers to pairs of images (reference and distorted), as well as demographic information about the workers.
This repository contains the folowing directories:
- Interface: Contains the IDSQS interface
- Subjective data: Contains raw data obtain from the subjective test
- Software: Contains a software to process the raw data
This research was funded by FCT/MCTES through national funds under the project DARING with reference PTDC/EEI-COM/7775/2020 and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 251654672 – TRR 161.