ICFR2013

The First ICB Competition on Face Recognition (ICFR2013)

    Automatic face recognition has received increasing attention due to its wide applications in access control, visual surveillance, human-computer interaction, etc. Much progress has been made in face recognition recently. However, the development of a high-performance face recognition algorithm and the transfer of these algorithms from research lab to practical applications is still a challenging task due to unpredictable intra-class variations of facial images on realistic scenario. Therefore the first ICB Competition on Face Recognition (or ICFR2013 shortly) is organized to track the state-of-the-art of face recognition in uncontrolled conditions.

     ICFR2013 is open to both academia and industry. There are two face image databases used in ICFR2013 for training and testing purposes, respectively. The training database (CASIA-FaceV5) contains 2,500 facial images of 500 subjects (i.e. 5 images/subject). And the testing database (CASIA-Face-TestV1) contains 4,000 facial images of 1,000 subjects (i.e. 4 images/subject). All these images were captured by a Logitech USB camera with varying illumination, pose, expression, accessories, and imaging distance. The face images in the training database are 16 bit color bitmap (BMP) files and the image resolution is 640×480 (Fig. 1). Participants of ICFR2013 can download the training database free of charge. The testing database is a sequestered dataset and each of the images is cropped so that only the facial regions remain, which prevents algorithms from performing identification based on non-face regions such as clothes. All face images of the testing database are also 16 bit color BMP files and the image resolution varies with the size of facial region in the 640×480 original image (Fig. 2).

Fig. 1 Example face images in the training database

Fig. 2 Example face images in the testing database

     ICFR2013 has a number of special features compared with some existing face recognition competitions.
    1) A large testing database including face images of one thousand subjects. As we know, most state-of-the-art face recognition competitions are limited to hundreds of subjects. So it is a good opportunity to test the individuality or degree of freedom of face recognition algorithms.
    2) The face images of ICFR2013 are captured on uncontrolled conditions. Large pose, illumination and expression variations are introduced to test the robustness of face recognition methods in practical applications. ICFR2013 is one of the most challenging face recognition competitions.
    3) All subjects in the ICFR2013 face databases are Chinese and as we know most subjects in state-of-the-art face recognition competitions are from Western countries. And these subjects have a wide and even statistical distribution in sex, age and professional. For example, the subjects in testing database include children, graduate students, workers, and waiters, etc.

     All participants should submit two executables "enroll_AuthorName.exe" and "match_AuthorName.exe" in the form of Win32 console applications for third-party performance evaluation. The file "enroll_AuthorName.exe" is used to generate a feature template from a face image. The file "match_AuthorName.exe" is used to match two face feature templates. The syntax of the two executables through command-line is as follows.

    > enroll_AuthorName imagefile templatefile outputfile
where "imagefile" denotes the pathname of a face image (e.g. "0001.bmp"), "templatefile" denotes the pathname of the enrolled feature template (e.g. "0001.dat"), and "outputfile" denotes the pathname of a text file (e.g. "0001.txt"). If the content of "0001.txt" is "1", it denotes a successful enrollment. If the content of "0001.txt" is "0", it denotes a failure to enrollment.

    > match_AuthorName templatefile1 templatefile2 outputfile
where "templatefile1" and "templatefile2" denote the pathname of two face feature templates, and "outputfile" denotes the pathname of a text file indicating the matching score ranging from 0 to 1. If the algorithm fails to match, please output "-1". Example source code of "enroll_AuthorName.exe" and "match_AuthorName.exe" in C language is provided to the participants for their convenience of programming. Download C language source code here In addition, participants can also submit other supporting files along with the two executables. Matlab submissions following a similar format are also allowed. The organizers of ICFR2013 will keep the submitted algorithms confidential. Each participant can protect these submitted files using password or USB key.

     All possible intra-class comparisons are implemented to evaluate the false non-match rate (FNMR) providing a total of 6,000 intra-class match results. One sample is selected from each face class to evaluate the false match rate (FMR) so there are totally 499,500 inter-class match results. If an intra- or inter-class comparison cannot be successfully implemented due to failure to enrollment or failure to match, a random variable ranging from 0 to 1 will be assigned as the matching score. Popular performance metrics of face recognition such as FNMR, FMR, EER and ROC will be reported and the metric F4(FNMR@ FMR=0.0001) will be used to rank the performance of submitted algorithms.

     Each participant can maximally submit three algorithms. We only accept qualified face recognition algorithms which meet the following requirements due to limited competition resources:

     • The equal error rate (EER) must be less than 10% on the training database. • The average processing time for feature encoding must be less than 3 seconds and the average matching time must be less than 0.1 second on a normal personal computer.

     • The average processing time for feature encoding must be less than 3 seconds and the average matching time must be less than 0.1 second on a normal personal computer.

     A public platform, the Biometrics Ideal Test (BIT; http://biometrics.idealtest.org) is used to organize the competition. The competition registration process is as follows.

     1. Register an account in BIT (http://biometrics.idealtest.org ) and select the option "Participant of ICB Competition on Face Recognition".

     2. Download the training database CASIA-FaceV5 after your account is approved and activated.
        Download training database here

     3. Develop a face recognition algorithm using the CASIA-FaceV5 as the training database. Note that the submitted algorithm should work on cropped face regions with variable image resolutions. A cropped version of CASIA-FaceV5 is also provided to the participants for their convenience.

     4. Submit your algorithm to bitadmin@nlpr.ia.ac.cn. If size of the algorithm file is too large, you can also send us a web link for download. To help users quickly master the programming protocols of algorithm evaluation, the source codes of the example files "enroll.c" and "match.c" are provided for download as below. Download C language source code here

     5. Any problem of your algorithm during testing will be reported to you so please respond immediately to the organizers of ICFR2013.

     6. The ICFR2013 Competition Committee will report the performance of all submitted algorithms and the competition results.

     7. Authors of the winner algorithm will be invited to write a paper on their algorithm to be published in the ICB2013 proceedings.

    Any questions on ICFR2013 can be addressed by emailing bitadmin@nlpr.ia.ac.cn.

     Important dates

    Open of the competition                         November 20, 2012

    Deadline of algorithm submission            March 20, 2013

    Release of the testing results                  June 5, 2013