QLFW – Image, Video, and Usability Lab

Quality Labeled Faces in the Wild (QLFW)

Contact: Lina Karam ([email protected]) and Tong Zhu ([email protected])
Image, Video, and Usabilty (IVU) Lab, http://ivulab.asu.edu , Arizona State University

Citation: L. J. Karam and T. Zhu, “Quality labeled faces in the wild (QLFW): a database for studying face recognition in real-world environments,” Proc. SPIE 9394, Human Vision and Electronic Imaging XX, 93940B, 10 pages, March 2015; doi: 10.1117/12.208039

Acknowledgment: This work was supported in part by Qualcomm.

Face recognition is an active research area with broad applications. The most important challenge that limits the effectiveness of face recognition technology is image quality [FedTech2013]. In real-world environment, the acquired face image quality varies due to lens resolution, focus, distance, compression, noise, luminance, etc. This may greatly affect the performance of face recognition algorithms. It is of great value to simulate changes of image quality in a comprehensive and systematic manner. This allows the evaluation of the performance of face recognition algorithms on images with different distortions and quality levels. This performance evaluation is crucial to boost the effectiveness and robustness of current face recognition technology. The goal of the Quality Labeled Face in the Wild (QLFW) database is to enable automated performance evaluation of face recognition technologies in the presence of different types and levels of visual distortions. This will consequently enable the development of face recognition systems that can operate reliably on real-world visual content in the presence of real-world visual distortions. A description of the QLFW database follows below.

DESCRIPTION OF THE QLFW DATABASE

The QLFW database is derived from the LFW database [LFWTech].In the QLFW database, the LFW images are subjected to different types and levels of distortions, simulating distortions that can occur under real-world conditions, including impairments due to compression, blur, noise, and contrast. For each visual impairment type, the level of impairments were chosen such that the whole range of visual quality is represented from Poor (strong perceived impairment as compared to original source) to Excellent (no perceived impairment, original source). Four different levels of impairments are used for each impairment type, besides the original source, resulting a total of 277,809 face images.

Distortion types in QLFW

In the QLFW database, the original LFW original images before impairments are named with an ending as ‘_or_0.jpg’. We chose to change the quality of the source images using five different image distortion types that could occur in real-world applications. The distortion types are:

JPEG2000 compression: The distorted images were generated by compressing the reference images using JPEG2000 at bit rates ranging from 1 bits per pixel (bpp) to 0.15 bpp. Kakadu version 7.3.3 [JP2K] was used to generate the JPEG2000 compressed images. Images after JPEG2000 compression are named with an ending as ‘_j2_*.jpg’, in which * ranges from 1 to 4, representing the level of impairment with the impairment level increasing from 1 to 4.

JPEG compression: The distorted images were generated by compressing the reference images using JPEG at different quality factors Q ranging from 40 to 5. The implementation used was the MATLAB imwrite function (www.mathworks.com). Images after JPEG compression are named with an ending as ‘_jp_*.jpg’, in which * ranges from 1 to 4, representing the level of impairment.

White noise: White Gaussian noise of standard deviation sigma was added to the RGB components of the images. The same sigma was used for the R, G, and B components. The values of sigma used were between 5 and 40. The distorted components were clipped between 0 and 255. Images after adding white noise are named with an ending as ‘_wn_*.jpg’, in which * ranges from 1 to 4, representing the level of impairment.

Gaussian blur: The R, G, and B color components were filtered using a circular-symmetric 2-D Gaussian. The three color components of the image were blurred using the same kernel. The implementation makes use of the MATLAB fspecial and imfilter functions. For these functions, the filter size is set as 5,5,7 or 9 pixels and the sigma is set as 1,2,4 or 8 pixels, for different levels of blur. Images after Gaussian blur are named ending as ‘_gb_*.jpg’, in which * ranges from 1 to 4, representing the level of impairment.

Contrast change: The R, G, and B components were converted into the CIE LAB color space. Then different contrast adjustments are applied to the L component. The first kind of contrast change is done by histogram equalization. The rest are completed by adjusting the histogram to the mid range, higher end, and lower end, respectively. After that the CIE LAB values were converted back to RGB. Images after contrast modification are named ending as ‘_co_*.jpg’, in which * ranges from 1 to 4, representing different kinds of contrast change.

These distortions represent a broad range of real-world image distortions, from smoothing, contrast change, compression, and random noise. As indicated previously, the levels of distortions were carefully chosen to generate images covering a broad range of quality, from imperceptible levels to high levels of impairment.

Download the database: All images as zip file QLFW.zip

Citation:

Please cite as:

L. J. Karam and T. Zhu, “Quality labeled faces in the wild (QLFW): a database for studying face recognition in real-world environments,” Proc. SPIE 9394, Human Vision and Electronic Imaging XX, 93940B, 10 pages, March 2015; doi: 10.1117/12.208039

PDF file of preliminary non-final version of above paper: [pdf]

PDF file of final version of above paper downloadable here

Training, Validation, and Testing

In the image-restricted setting, researchers should only use matched and mismatched pairs provided in corresponding pairsDevTrain_xx.txt, pairsDevTest_xx.txt and pairs_xx.txt, where xx is replaced by two letters referring to the impairment type.

In the image-unrestricted setting, researchers are allowed to use the names of people to generate additional training examples, through the given files peopleDevTrain_xx.txt, peopleDevTest_xx.txt and people_xx.txt where xx is replaced by two letters referring to the impairment type.

Restricted Setting

View 1:

For development purposes, we recommend using the below training/testing split, For instance, these sets may be viewed as a model selection set and a validation set. The QLFW View 1 consists of the source images in View 1 of LFW (excluding errors and mislabeled images which are now corrected in this QLFW database) together with the associated visual distortions introduced in QLFW. When testing the effectiveness of face recognition algorithms, users can choose to limit the training and testing to a single kind of impairment or to include all impairements. The training image list attempts to cover broad range of image impairments, from imperceptible levels to high levels of impairments. The testing image list uses random level of impairments. In the image list, lines such as

Aaron_Sorkin 1_or_0.jpg 2_gb_2.jpg

means it is an matched pair. The test images are Aaron_Sorkin_0001_or_0.jpg (undistorted original source) and Aaron_Sorkin_0002_gb_2.jpg (Level 2 Gaussian blur impairment) under folder Aaron_Sorkin.

while lines such as

Gonzalo_Barrientos 1_or_0.jpg Lionel_Hampton 1_gb_4.jpg

means it is an unmatched pair. The test images are Gonzalo_Barrientos_0001_or_0.jpg (undistorted original source) under folder Gonzalo_Barrientos and Lionel_Hampton_0001_gb_4.jpg (Level 4 Gaussian blur impairment) under folder Lionel_Hampton.

Download the training sets for View 1 (restricted setting):

This set is for the training/model selection part of the algorithm development.

pairsDevTrain_j2.txt (JPEG2000 compression)

pairsDevTrain_jp.txt (JPEG compression)

pairsDevTrain_wn.txt (White noise)

pairsDevTrain_gb.txt (Gaussian blur)

pairsDevTrain_co.txt (Contrast change)

pairsDevTrain_al.txt (All impairment types)

Download the validation/testing sets for View 1 (restricted setting):

This set is for the testing/validation part of the algorithm development. Researchers should choose
one of the lists below and the chosen list should correspond to the same impairment type that was selected for training:

pairsDevTest j2.txt (JPEG2000 compression)

pairsDevTest jp.txt (JPEG compression)

pairsDevTest wn.txt (White noise)

pairsDevTest gb.txt (Gaussian blur)

pairsDevTest co.txt (Contrast change)

pairsDevTest al.txt (all impairment types)

View 2:

As a benchmark for comparison, we suggest reporting performance as 10-fold cross validation using splits we have randomly generated. The QLFW View 2 consists of the source images in View2 of LFW together with associated visual impairements introduced in QLFW. When testing the effectiveness of face recognition algorithms, users can choose to limit the training and testing to a single kind of impairment or to include all impairments. The image list uses a random level of impairments.

As for the View 1 testing set, the chosen list in View 2 should correspond to to the same impairment type that was selected for training in View 1.

Download the sets for View 2 (restricted setting):

pairs_j2.txt (JPEG2000 compression),

pairs_jp.txt (JPEG compression),

pairs_wn.txt (White noise),

pairs_gb.txt (Gaussian blur),

pairs_co.txt (Contrast change),

pairs_al.txt (All impairment types)

Unresticted Setting

View 1:

In the training and testing set of View 1 (unrestricted setting), the first line of the people list gives the total number N of people involoved in the set. The next N lines give the matched pairs in the example format:

AJ_Lamas 5

The example above means the individual AJ_Lamas has 5 images which can be used to generate additional training and/or testing pairs.

Download the training sets for View 1 (unrestricted setting):

Researchers should choose one of the lists below for View 1 (unrestricted setting) training:

peopleDevTrain_j2.txt (JPEG2000 compression)

peopleDevTrain_jp.txt (JPEG compression)

peopleDevTrain_wn.txt (White noise)

peopleDevTrain_gb.txt (Gaussian blur)

peopleDevTrain_co.txt (Contrast change)

peopleDevTrain_al.txt (all impairment types)

Download the validation/testing sets for View 1 (unrestricted setting):

peopleDevTest_j2.txt (JPEG2000 compression),

peopleDevTest_jp.txt (JPEG compression),

peopleDevTest_wn.txt (White noise),

peopleDevTest_gb.txt (Gaussian blur),

peopleDevTest_co.txt (Contrast change),

peopleDevTest_al.txt (all impairment types)

View 2:

As a benchmark for comparison, we suggest reporting performance as 10-fold cross validation using 10 sets (splits). The QLFW View 2 consists of the source images in View2 of LFW (excluding errors and mislabeled images that have been corrected in this QLFW database) together with associated visual impairements introduced in QLFW. When testing the effectiveness of face recognition algorithms, users can choose to limit the training and testing to a single kind of impairment or to include all impairments. The chosen list in View 2 (unrestricted setting) should correspond to to the same impairment type that was selected for training in View 1 (unrestricted setting).

The View 2 (unrestricted setting) people list is formatted similar to View 1 (unrestricted setting), except that the first line gives the number of sets (splits) followed by the number of available people N. The next N lines give the corresponding individual’s name and total number of images for that indiividual. This is then repeated nine more times to provide the people list for the other nine sets (splits).

Download the sets for View 2 (unrestricted setting):

people_j2.txt (JPEG2000 compression)

people_jp.txt (JPEG compression)

people_wn.txt (White noise)

people_gb.txt (Gaussian blur)

people_co.txt (Contrast change)

people_al.txt (all impairment types)

Results

Accuracy and ROC curves for various methods will be made available on results page.

Information

13229 images* (5 impairment type* 4 level impairment per type+ original)
5749 people

References

[QLFW] L. J. Karam and T. Zhu, “Quality labeled faces in the wild (QLFW): a database for studying face recognition in real-world environments,” Proc. SPIE 9394, Human Vision and Electronic Imaging XX, 93940B, 10 pages, March 2015; doi: 10.1117/12.208039

[FedTech2013] 4 Limitations of Facial Recognition Technology. Nov 22,2013 http://www.fedtechmagazine.com/article/2013/11/4-limitations-facial-recognition-technology

[LFWTech] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments.University of Massachusetts, Amherst, Technical Report 07-49, October, 2007. http://vis-www.cs.umass.edu/lfw/

[JP2K] D. S. Taubman and M. W. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice. Kluwer Academic Publishers, 2001.

[MTW] MathWorks, www.mathworks.com