Testing Results of The First ICB Competition on Iris Recognition（ICIR2013）
Number of participants: 8 developers from 6 countries Number of algorithms: 13
*In the competition, there is a little problem about the Nadia Othman’s algorithm. In his original results, smaller matching score indicates more similarity of two images, which does not follow our requirements stated in the demo code. Therefore, the results evaluated in the competition (FNMR when FMR=0.0001) cannot stand for the real performance. After the competition, he corrects the mistake and the new results marked as ‘Nadia Othman * (correction after competition)’ are reported in the website to show the real performance of the submitted algorithm.
The First ICB Competition on Iris Recognition （ICIR2013）
With the pronounced need for reliable personal identification, iris recognition has become an important enabling technology in our society. Although an iris pattern is naturally an ideal identifier, the development of a high-performance iris recognition algorithm and transferring it from research lab to practical applications is still a challenging task. Automatic iris recognition has to face unpredictable variations of iris images in real-world applications. Therefore the first ICB Competition on Iris Recognition (or ICIR2013 shortly) is organized to track the state-of-the-art of iris recognition.
ICIR2013 is open to both academia and industry. There are two iris image databases used in ICIR2013 for training and testing purposes, respectively. The training database (CASIA-Iris-Thousand) contains 20,000 iris images of 1,000 subjects (i.e. 20 images/subject and 10 images/eye). CASIA-Iris-Thousand was collected using the iris camera produced by IrisKing. All iris images of CASIA-Iris-Thousand are 8 bit gray-level BMP files and the image resolution is 640*480. And the testing database (or IR-TestV1) contains 10,000 iris images of 2,000 eyes from 1,000 subjects. The iris images of IR-TestV1 were captured using IrisGuard camera in one session. Each volunteer contributed 10 iris images of both left and right eyes, i.e. 5 images per each class. The main sources of intra-class variations in IR-TestV1 are motion blur, non-linear deformation, eyeglasses and specular reflections. All iris images of IR-TestV1 are 8 bit gray-level BMP files and the image resolution is 640*480.
Fig. 1 Example iris images in the training database
Fig. 2 Example iris images in the testing database
All participants should submit two executables "irisenroll_AuthorName.exe" and "irismatch_AuthorName.exe" in the form of Win32 console applications for third-party performance evaluation. The file "irisenroll_AuthorName.exe" is used to generate a feature template from an iris image. The file "irismatch_AuthorName.exe" is used to match two iris feature templates. The syntax of the two executables through command-line is as follows.
>irisenroll_AuthorName imagefile templatefile outputfile
> irismatch_AuthorName templatefile1 templatefile2 outputfile
All possible intra-class comparisons are implemented to evaluate the false non-match rate (FNMR) providing a total of 20,000 intra-class match results. One sample is selected from each iris class to evaluate the false match rate (FMR) so there are totally 1,999,000 inter-class match results. If an intra- or inter-class comparison cannot be successfully implemented due to failure to enrollment or failure to match, a random variable ranging from 0 to 1 will be assigned as the matching score. Popular performance metrics of iris recognition such as FNMR, FMR, EER and ROC will be reported and the metric F4(FNMR@ FMR=0.0001) will be used to rank the performance of submitted algorithms.
Each participant can maximally submit three algorithms. We only accept qualified iris recognition algorithms which meet the following requirements due to limited competition resources:
• The equal error rate (EER) must be less than 5% on the training database.
• The average processing time for feature encoding must be less than 3 seconds and the average matching time must be less than 0.1 second on a normal personal computer.
A public platform, the Biometrics Ideal Test (BIT; http://biometrics.idealtest.org) is used to organize the competition. The competition registration process is as follows.
1. Register an account in BIT (http://biometrics.idealtest.org ) and select the option "Participant of ICB Competition on Iris Recognition".
2. Download the training database CASIA-Iris-Thousand after your account is approved and activated.
3. Develop an iris recognition algorithm and you can use the CASIA-Iris-Thousand or other iris image databases as the training data.
4. Submit your algorithm to firstname.lastname@example.org. If size of the algorithm file is too large, you can also send us a web link for download. To help users quickly master the programming protocols of algorithm evaluation, the source codes of the example files "enroll.c" and "match.c" are provided for download as below. Download C language source code here
5. Any problem of your algorithm during testing will be reported to you so please respond immediately to the organizers of ICIR2013.
6. The ICIR2013 Competition Committee will report the performance of all submitted algorithms and the competition results.
7. Authors of the winner algorithm will be invited to write a paper on their algorithm to be published in the ICB2013 proceedings.
Any questions on ICIR2013 can be addressed by emailing email@example.com.
Open of the competition November 20, 2012
Deadline of algorithm submission March 20, 2013
Release of the testing results June 5, 2013