The Facial Recognition Technology (FERET) Database

 

  I.          Introduction

 

The FERET program ran from 1993 through 1997. Sponsored by the Department of Defense's Counterdrug Technology Development Program through the Defense Advanced Research Products Agency (DARPA), its primary mission was to develop automatic face recognition capabilities that could be employed to assist security, intelligence and law enforcement personnel in the performance of their duties.

 

The FERET image corpus was assembled to support government monitored testing and evaluation of face recognition algorithms using standardized tests and procedures. The final corpus, presented here, consists of 14051 eight-bit grayscale images of human heads with views ranging from frontal to left and right profiles.

 

The CDROM distribution that accompanies this document constitutes the first and final release of all the FERET imagery. A previous collection provided to academic and industrial concerns from 1998 through to February 2001 held just 3737 images. Results from the FERET evaluations have been published; see the bibliography below.

 

 

II.          Nomenclature

 

The naming convention for the FERET imagery in this distribution is of the form nnnnnxxfffq_yymmdd.ext where:

 

  1. nnnnn is a five digit integer that uniquely identifies the subject

 

  1. xx is a two lowercase character string that indicates the kind of imagery:

 

Two letter code

Pose Angle (degrees)

Description

Number in Database

Number of Subjects

Fa

0 = frontal

Regular facial expression

1762

1010

Fb

0

Alternative facial expression

1518

1009

ba

0

Frontal "b" series

200

200

bj

0

Alternative expression to ba

200

200

bk

0

Different illumination to ba

200

200

bb

+60

Subject faces to his left which is the photographer's right

200

200

bc

+40

200

200

bd

+25

200

200

be

+15

200

200

bf

-15

Subject faces to his right which is the photographer's left

200

200

bg

-25

200

200

Bh

-40

200

200

bi

-60

200

200

ql

-22.5

Quarter left and right

763

508

qr

+22.5

763

508

hl

-67.5

Half left and right

1246

904

hr

+67.5

1298

939

pl

-90

Profile left and right

1318

974

pr

+90

1342

980

Ra

+45

Random images. See note below. Positive angles indicate subject faces to photographer's right

322

264

Rb

+10

322

264

Rc

-10

613

429

Rd

-45

292

238

Re

-80

292

238

 

Notes:

1.       fa indicates a regular frontal image

2.       fb indicates an alternative frontal image, taken seconds after the corresponding fa

3.       ba is a frontal images which is entirely analogous to the fa series

4.       bj is an alternative frontal image, corresponding to a ba image, and analogous to the fb image

5.       bk is also a frontal image corresponding to ba, but taken under different lighting

6.       bb through bi is a series of images taken with the express intention of investigating pose angle effects (see below). Specfically, bf - bi are symmetric analogues of bb - be.

7.       ra through re are "random" orientations. Their precise angle is unknown. It appears that the pose angles are random but consistent. The pose angles in the table were derived by manual measurement of inter-eye distances in the image, and in their corresponding frontal image.

 

  1. fff is a set of three binary (zero or one) single character flags. In order these denote:
    1. Indicates whether the image is releasable for publication. The flag has fallen into disuse: All images are available via this CDROM distribution, but still none may be published without the explicit written permission of the government.
    2. Image is histogram adjusted if this flag is 1
    3. Indicates whether the image was captured using ASA 200 or 400 film, 0 implies 200.

 

  1. q is a modifier that is not always present. When it is, the meanings are as follows:
    1. Glasses worn. Note that this flag is a sufficient condition only, images of subjects wearing glasses do not necessarily carry this flag. Some retroactive re-truthing of such images to fix this problem is warranted. See also "c" below.
    2. Duplicate with different hair length.
    3. Glasses worn and different hair length
    4. Electronically scaled (resized) and histogram adjusted.
    5. Clothing has been electronically retouched.
    6. Image brightness has been reduced by 40%
    7. Image brightness has been reduced by 80%
    8. Image size has been reduced by 10%, with white border replacement
    9. Image size has been reduced by 20%, with white border replacement
    10. Image size has been reduced by 30%, with white border replacement

 

Note that the modifications d through j are the result of applying various off-line operations to real images in the database; the "parent" image is that image without the "q" modifier present at all.

 

5.       The three fields are the date that the picture was taken in year, month, day format.

6.       The filename extension is .tif. The images on the CDROMs carry an additional .bz2 suffix that indicates that the files have been losslessly compressed using the free bzip2 compressor, supplied with the database in misc/bzip2/.

 

 

III.          Standard Testing Subsets

 

Empirical testing of pattern recognition algorithms is predicated on standardized data sets. Previously publications using FERET images have reported performance on the images described below. The concepts of gallery and probe sets apply: Each probe image[1] is matched against those in a gallery, and the ranked matches can be analyzed to produce recognition performance measures such a cumulative match score for identification, and receiver operating characteristic for verification applications.

 

Users are strongly encouraged to utilize the specific gallery and probe sets included in this FERET distribution. The reports detailed in the previous footnotes quote performance measures for contemporary algorithms on exactly these standardized test sets. These partitions of the corpus defined by the *.names files in the directories underneath the to-level folder partitions/.

 

The standard subsets have two distinct provenances. The first was employed originally in the FERET tests themselves, and were used again in the Face Recognition Vendor Test 2000 (FRVT2000, see footnote 5). The second series were designed to evaluate the sensitivity of algorithms to head direction (pose angle) as part of the FRVT 2000 tests.

 

In the tables below the parenthesized numbers indicate the number of images in the respective sets.

 

 

1.      FERET Tests September 1996

 

These tests employed frontal images gathered between 1993 and 1996. The image sets are in the partitions/by_previously_reported/feret/ directory.

 

Evaluation Task

Recognized Names

Gallery (1196)

Probe Set

Aging of subjects

Duplicate I or T1

gallery.names

probe_dup_1_*.names (722)

Aging of subjects

Duplicate II or T2

gallery.names

probe_dup_2_*.names (234)

Facial Expression

fafb

gallery.names

probe_fafb_*.names (1195)

Illumination

fafc

gallery.names

probe_fafc_*.names (194)

 

 

Note that all the above tests used a single gallery containing 1196 images[2]. The Duplicate I probe images were obtained anywhere between one minute and 1031 days after[3] their respective gallery matches. The harder Duplicate II probe images re a strict subset of the Duplicate I images; they are those taken only at least 18 months[4] after their gallery entries. For assessment of the effect of facial expression two probe sets have been used. There is usually only a few seconds between the capture of the gallery-probe pairs.

 

 

2.      FRVT 2000 Tests May 2000

 

The following four tests were conduced as part of the Facial Recognition Vendor Test 2000[5] and used a single frontal gallery, and four non-frontal probe sets taken at increasing azimuthal[6] angles. The intention of these tests was to quantify the effect of non-frontal image capture on recognition performance. The images are named in the partitions/by_previously_reported/frvt2000/ directory.

 

 

Evaluation Task

Recognized Names

Gallery (200)

Probe Set

Pose Azimuth 15 deg

P1

gallery.names

P1_probe.names (400)

Pose Azimuth 25 deg

P2

gallery.names

P2_probe.names (400)

Pose Azimuth 40 deg

P3

gallery.names

P3_probe.names (400)

Pose Azimuth 60 deg

P4

gallery.names

P4_probe.names (400)

 

 

The FRVT 2000 evaluation repeated the tests using the FERET Duplicate I and II imagery, used the working names T1 and T2 (see FRVT 2000 report in footnote 5).

 

The table and the accompanying figure show the relationships between the sets defined in the preceeding sections.

 

FERET Subset 1

 

Feret Subset 2

Cardinality

FERET Gallery

Training CD

270

Duplicate I

Training CD

184

Duplicate II

Training CD

0

Probe set fafb

Training CD

270

Probe set fafc

Training CD

0

Probe set fafb

Probe set fafc

0

Probe set fafb

Duplicate I

0

Duplicate II

Duplicate I

234

 

 

 

 

IV.          Ground Truth Information

 

The previous section describes the naming of the FERET image files. Those file names encode the entire known ground truth information for each image. We have also included on the CDROM two alternative ground truth formats that may assist the user in selection and inspection of FERET imagery. The directories data/cd_*/ground_truths/ contain those two alternative markups, namely name_value and xml. Those directories contain precisely one file for each image in the images directories.

 

         The name_value tree holds *.gnd files which contain ground truth entries of the form "name = value". For example "date_taken = 8 May 1982".

 

         The XML files are similar but are constrained syntactically by a Document Type Definition (DTD) which resides on the FERET website. Similarly linked is a stylesheet that controls browser function.

 

 

 

V.          Contents of the CDROMs

 

Each CDROM contains nine top-level directories, and two files, each of which is described below. The content of all the top-level directories is identical for each CDROM with the obvious exception that the data directories contain disjoint subsets of the FERET imagery. In detail:

 

         Makefile top-level recursive Makefile for eigenface implementation

         bin various executables for eigenface recognition

         data main repository of images and associated ground truth files

         doc documentation, FERET related publications and this file

         include top level C/C++ include files for eigenface implementation

         lib library files for eigenface executables

         misc collection of files used in assembly and final preparation of this CDROM; eye-mouth coordinate information for some images; the bzip2 decompressor

         partitions lists of image files relevant to certain tasks: by pose, expression, and subject; probe and gallery sets suitable for assessing recognition performance

         src C/C++ source code hierarchy for the eigenface implementation

         test directory containing scripts and debugging code for eigenface-based recognition system

 

 

 

 

VI.          Contact Information

 

To obtain a copy of the FERET database the user should see the FERET homepage. Beyond that enquires should be directed to feret@nist.gov.

 

 

VII.          Bibliography

 

The following publications describe the construction and use of the FERET database. These publications. With the exception of the PAMI paper all are on the web or in the doc/feret directory on the CDROMs

 

  1. P. J. Phillips, H. Moon, P. J. Rauss, and S. Rizvi, "The FERET evaluation methodology for face recognition algorithms", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 10, October 2000.

 

  1. P. J. Phillips, P. J. Rauss, and S. Z. Der, " FERET (Face Recognition Technology) Recognition Algorithm Development and Test Results", October 1996. Army Research Lab technical report 995.

 

  1. P. J. Phillips, H. Moon, P. J. Rauss, and S. Rizvi, " The FERET Evaluation Methodology for Face Recognition Algorithms".

 

  1. S. Rizvi, P. J. Phillips and H. Moon, " The FERET Verification Testing Protocol for Face Recognition Algorithms".

 



[1] In the FERET protocol a probe set contains one or more images from a set of individuals. Each person will have exactly one match in the gallery. The gallery may contain images from other individuals who are not in the probe population.

[2] The tables in the published FERET reports in the previous footnote indicate that the gallery for the Duplicate II test held 864 images. This is erroneous; the correct number is 1196.

[3] The Duplicate I probe set holds 722 images whose matches were taken between 0 and 1031 days after the match. The median is 72 days and the mean is 251 days.

[4] The Duplicate II probe set contains 234 images from subjects whose gallery match was taken between 540 and 1031 days beforehand. The median is 569 and the mean is 627 days. Thus the Duplicate II probe images were taken at least 18 months after their gallery match.

[5] The FRVT 2000 tests evaluated leading commercial face recognition vendors. They were sponsored by the DoD Counterdrug Technology Development Program Office, the National Institute of Justice, and the Defense Advanced Research Projects Agency, and were administered between May and June 2000. For the final report see http://www.dodcounterdrug.com/facialrecognition.

[6] To be explicit, non-frontal and azimuthal, mean here that the subject rotated his head and body about a vertical axis facing in a direction other than toward the camera with eyes a horizontal plane. The other degrees of freedom in which the head rotates about horizontal axes: pitch (looking up or down) and roll (tipping head on it's side) are not represented in the FERET database.