QualTex Database

Contact: Lina Karam ([email protected]) and Milind Gide ([email protected])
Image, Video, and Usabilty (IVU) Lab, http://ivulab.asu.edu , Arizona State University

Abstract:

Textures are being extensively used in next-generation video coding, image restoration and computer graphics wherein the stochastic and perceptual properties of textures are exploited. Given the prominence of texture content in image and video processing applications, texture quality assessment and objective quality metrics geared specifically towards texture are of particular importance. This paper outlines an extensive subjective study in which a total of 340 distorted images extracted from 10 reference images were evaluated by a group of 20 human subjects. The subjective Mean Opinion Scores (MOS) thus obtained from the study were used to evaluate several existing mainstream full reference quality metrics with the help of a recently developed quality assessment framework (IVQUEST). Based on the results obtained, it can be concluded that traditional objective metrics do not perform well on texture images and there is a need for specialized texture quality metrics.

Database:

Figure 1 shows the 10 different reference images used in the QualTex database. These images have been taken from the freely available VisTex database [1]. These images cover the range of different texture types such as regular, non-regular and stochastic textures as well as varying ability to mask different types of noise and distortions. These reference images are subjected to the following different types and levels of distortions:

JPEG2000 compression: The distorted images were created by compressing the reference images using JPEG2000 at compression ratios varying from 20:1 to 200:1 in 8 linear steps to create 80 test images for this type of distortion. The JasPer 1.900.1 software [2] was used for generating the JPEG2000 images.
Additive white Gaussian Noise: White Gaussian noise with standard deviation σN varying from 0.0005 to 0.005 in 8 linear steps was added to the R,G and B components of the reference images to create 80 test images for this type of distortion.
Gaussian Blur: The R, G and B components were ﬁltered using a 2-D circularly symmetric Gaussian kernel having a radius of 7 pixels by using the fspecial function in Matlab. The standard deviation of the Gaussian ﬁlter σB was varied from 0.42 to 7 in 8 linear steps to get 80 such distorted images for this type
Sub-pixel shifts: The distorted images were created by shifting the reference images by 0.25, 0.5, 1.5 and 5 pixels horizontally, vertically and diagonally to produce 8 test images per reference image resulting in a total of 80 distortions of this type. The combinations used were (0,0.25), (0.25,0), (0.25,0.25), (0,0.5), (0.5,0), (0.5,0.5),(1.5,1.5), and (5,5), where the ﬁrst value represents a horizontal shift and the second value represents a vertical shift. The motivation behind this distortion was to observe the performance of shift-variant quality metrics for sub-pixel shifts.
Synthesis with parameter distortion: For this type of distortion, the test images were synthesized from the reference images using the Portilla-Simoncelli [3] texture model by keeping first the parameters unchanged to get a similar looking test image and then quantizing the parameters by using a uniform quantizer to get a distorted version of the image. Thus for this type of distortion, 20 test images were obtained.

Subjective Testing:

Hosted by imgur.com

A single-stimulus methodology was adopted in which the reference and test images were shown side by side and the subjects were asked to compare the quality of the test (right) image with the reference (left) image using a 5-point scale marked ‘Bad’,‘Poor’,‘Fair’,‘Good’ and ‘Excellent’ as shown in Figure 2. However within the same session, subjects were also shown reference images side-by-side to get an estimate of their judgment for zero distortion and also to enable the use of differential mean opinion scores (DMOS) in the future. A total of 20 subjects were recruited for the test. The images were shown to the subjects in a randomized fashion. The rating scale was a 5-point scale, where a score of 1 was assigned to the lowest rating of ‘Bad’ and a score of 5 was assigned to the highest ‘Excellent’ rating with an increment of 1 between consecutive ratings. Once the raw scores were obtained for all the 20 subjects, they were averaged over all the reliable subjects to obtain the mean opinion scores (MOS).

Citation:

If you plan to use the database, please cite the following paper

M.S. Gide and L.J. Karam, “On the assessment of the quality of textures in visual media”, 44th Annual Conference on Information Sciences and Systems (CISS), pp.1,5, March 2010. Available online at http://goo.gl/WbiyE4

Bibtex Entry :

@INPROCEEDINGS{QualTex,
author={M. S. Gide and L. J. Karam},
booktitle={44th Annual Conference on Information Sciences and Systems (CISS)},
title={On the assessment of the quality of textures in visual media},
year={2010},
month={March},
pages={1-5},
}

References:

[1] http://vismod.media.mit.edu/vismod/imagery/VisionTexture/vistex.html.

[2] M. Adams and R. Ward, “JASPER: a portable flexible open-source software tool kit for image coding/processing,” Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, pp. 241–244, May 2004.

[3] J. Portilla and E. P. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,” Int’l Journal of Computer Vision, vol. 40, no. 1, pp. 49–71, December 2000.

Copyright:

—— COPYRIGHT NOTICE, CONDITIONS, AND DISCLAIMER START WITH THIS LINE —————–

Contact: Lina Karam ([email protected]) and Milind Gide ([email protected])

Image, Video, and Usabilty (IVU) Lab, http://ivulab.asu.edu , Arizona State University

This copyright notice must also be included in any file or product that is derived from the database.

Redistribution and use of the QualTEX database, without modification, are permitted provided that the following conditions are met:

– Redistribution’s of the database must retain the above copyright notice, this list of conditions and the following disclaimer.

– Redistribution’s in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

– The Image, Video, and Usability Laboratory (IVU Lab, http://ivulab.asu.edu) is acknowledged in any publication that reports research results using this database, copies of this database, or modifications of this database.

Our paper is to be cited as follows:

M.S. Gide and L.J. Karam, “On the assessment of the quality of textures in visual media”, 44th Annual Conference on Information Sciences and Systems (CISS), pp.1-5, March 2010.

Bibtex Entry :

DISCLAIMER:

This database is provided by the copyright holders and contributors “as is” and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed.

In no event shall the Arizona Board of Regents, Arizona State University, IVU Lab members, authors or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this database, even if advised of the possibility of such damage.

—— COPYRIGHT NOTICE, CONDITIONS, AND DISCLAIMER END WITH THIS LINE —————–

Downloading the Database:

By downloading the QualTEX database, you agree to the above COPYRIGHT NOTICE, CONDITIONS, AND DISCLAIMER.

The entire database which includes all the images and the raw subjective scores can be downloaded by clicking here.