Skip to content

Subjective data collection and analysis. This repository includes a graphical user interface, associated software, and collected data, all based on the IDSQS subjective test methodology.

Notifications You must be signed in to change notification settings

shimamohammadi/IDSQS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

In-place Double Stimulus Methodology for Subjective Assessment of High Quality Images

Shima Mohammadi, Mohsen Jenadeleh, Michela Testolina, Jon Sneyers, Touradj Ebrahimi, Dietmar Saupe, Joao Ascenso

Instituto de Telecomunicações, Instituto Superior Técnico, University of Lisbon

Download the paper

About the Paper

This paper presents a novel double stimulus subjective assessment methodology for the evaluation of high-quality images aimed at addressing the limitations of existing protocols to capture near visually lossless perceptual quality differences. The In-place Double Stimulus Quality Scale (IDSQS) allows subjects to alternately view a reference and a distorted image at the same spatial location, facilitating a more intuitive detection of differences in quality. A large-scale crowdsourcing study employing this methodology was conducted, generating a comprehensive public dataset to evaluate perceived image quality across several compression algorithms and distortion levels. Another novelty is the fitting of the Beta distribution to the quality scores obtained to study their variability and thus, the agreement between subjects. Our findings demonstrate the effectiveness of the IDSQS methodology in achieving high correlation with more precise subjective evaluation benchmarks.

This repository contains data from a study conducted as part of JPEG AIC-3 activities using Inplace Double Stimulus Continous Quality Scale (IDSQS). For more information about the JPEG AIC-3, see: JPEG AIC-3.

The study was conducted on Amazon Mechanical Turk (MTurk). The data includes responses from crowd workers to pairs of images (reference and distorted), as well as demographic information about the workers.

Files in the Repository

This repository contains the folowing directories:

  1. Interface: Contains the IDSQS interface
  2. Subjective data: Contains raw data obtain from the subjective test
  3. Software: Contains a software to process the raw data

Acknowledgements

This research was funded by FCT/MCTES through national funds under the project DARING with reference PTDC/EEI-COM/7775/2020 and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 251654672 – TRR 161.

About

Subjective data collection and analysis. This repository includes a graphical user interface, associated software, and collected data, all based on the IDSQS subjective test methodology.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published