Latent Testing Workshop
April 5 & 6, 2006
National Institute of
Standards and Technology
Gaithersburg, Maryland
20899
1 Background and Purpose
Searches of latent fingerprints against rolled impressions, against plain impressions (slaps or flats), or against latent fingerprints repositories are of rapidly increasing importance. Of particular value is the ability to do real-time screening against latent prints. It is projected that present processes and technologies will have difficulty in coping with the anticipated volumes. Accordingly, there is great interest in determining how the latent identification process can be expedited. Effective methods for obtaining system performance parameters such as reliability and search-speed/throughput are required to gauge progress.
The purpose of this workshop is to provide a joint Government-Industry forum for discussing topics which will lead to better understanding of testing of latent algorithms, latent search performance, and related areas. Specific topics include: a) NIST’s proposed approach to latent testing; b) feedback from industry regarding whether the proposed testing approach would place limitations on their products; c) general suggestions for improving the testing methodology and augmenting the testing sets; d) determining the extent to which automation can be used to expedite human-based processing, e) determine whether fully automated feature extraction and matching is practical for some latents, and f) “future direction” of latents and what steps should be taken now to prepare for this future.
It is anticipated that this workshop and its follow-up efforts will result in improved testing methodology, a better understanding of the state of latent matching, and insight into potential future performance improvements. Ultimately this should lead to improved performance. It should be noted that the workshop will focus on the testing of automated latent feature extraction and matching technologies or algorithms, and not on the testing of human latent examiners.
2 SDK Testing
NIST’s SDK (Software Development Kit) testing methodology uses vendor-supplied linkable libraries. These libraries are linked by NIST, and the resulting executable modules are run on NIST-resident computers, using Government data for input. This methodology provides a means of protecting vendor proprietary code and algorithms, avoids privacy issues, while allowing NIST to execute the testing in a standards-compliant manner. The SDK methodology has been used successfully in all recent NIST projects. The workshop will provide a review of these projects, as well as “lessons learned, and ways in which the SDK framework can be used for testing of latent fingerprints.” NIST requests that industry comment on whether such testing impedes or limits their products.
3 Latent Test Sets
NIST has acquired a variety of latent data for use in evaluations. Perhaps best known is NIST’s Special Database 27. This contains 258 latent fingerprints from crime scenes, their matching rolled fingerprint mates, and corresponding minutiae data. We may supplement this data by adding up to 2000 new latent images and rolled mates. In the workshop, we will detail the methods used for ground truthing these probes, and we will describe the selected 1.7M background (gallery) population. Lastly, we will discuss ideas for augmentation of the database.
4 Feature Extraction Methodology
While human fingerprint experts are currently capable of significantly better latent feature extraction than are fully automated systems (resulting in a higher “hit” rate for human extraction), it is anticipated that in the not too distant future the government will be operating in a lights-out (fully automatic) environment for some latent applications. It is therefore desirable that the latent testing simulate a lights-out environment, to the extent feasible. This session provides a forum for any anticipated problems entailed by this modus operandi.
5 Supplemental Latent Test Sets
Sets comprised of actual casework generally result in the most acceptable test sets – after all these provide the “proof of the pudding.” However, for determining specific shortcomings of systems other types of test sets might be superior. Such special test sets consist of graduated “anomalies” which stress a given system in a controlled manner. An example of such a set consists of copies of the same fingerprint but with decreasing number of minutiae in successive prints. A second example is specimens of increasing distortion due to elastic shear.
Test sets formed entirely from casework may contain too few (or no) specimens of these anomalies, precluding statistically valid conclusions. Two methods for creating such supplemental test sets have been identified: a) computer-generated, and b) prints collected under controlled conditions from volunteers. Both methods have their plusses and minuses. Concern is often expressed that computer-generated data may be “too artificial,” and can lead to misleading conclusions. Controlled prints from volunteers, while more realistic, are time consuming and raise human subject issues. Perhaps a combination of the two is desirable. In this session we invite discussions on how this “dual mode” testing might best be implemented.
6 Universal Features/Minutiae
Interoperability between different fingerprint matches has been historically limited to images. At various times it has been proposed that “universal feature set” be specified from which every contractor is able to extract a subset suitable for their system. The term “universal feature set” can be interpreted in two distinct ways: 1) a maximal set of features sufficiently comprehensive such that every vendor may derive their native feature set from it, or 2) a minimal set containing features common to all vendors, though not necessarily providing all features used by a vendor.
No workable solution for the first type has as yet been proposed. Standards have been specified for the minimal feature set type (e.g., x, y, and theta). A commonly held opinion is that the second form of features imposes too great a performance drop, especially for latents. However, thus far this has not been conclusively demonstrated.
The purpose of NIST’s ongoing Minutiae Interoperability Exchange Test 2004 (MINEX04) is to assess the feasibility of using minutiae data (rather than images) as the exchange medium for rolled or plain impressions between different matching systems. This part of the workshop will solicit suggestions for the best approach to testing universal minutiae.
7 Identifying High Probability Latent Matches
Currently all comparisons of latents with candidate matches are performed by human examiners. Because of the interest in lights-out searches, the workshop will address the possibility of using automated processes to identify high-probability latent matches.
8 Latent Image Quality Measures
Image quality measures have been found to be very useful in characterizing both rolled and plain impressions. It has been shown that there is a strong correlation between image quality and “hit” rate for these classes of fingerprints. Transferring the concept to latent images however presents special problems. Latent images are typically of much poorer quality than rolled or plain impressions. For the poorest grade of latents, whether or not a candidate is verifiable (“ident” or “non-ident,” as opposed to “inconclusive”) may depend upon the existence of small sub-regions of greater clarity and high information content. A latent quality measure may therefore need to be more “fine tuned” than the corresponding measure for rolled or plain impressions.
Workshop Scope
This workshop is aimed at providing a forum for discussing topics which will lead to better understanding of testing of latent algorithms, latent search performance, and related areas. This is a two-day workshop. Day one will concentrate on the testing of automated searches of latents. Day two will consider universal features, latent image quality, as well as assessment of "future directions."
Call for Participation
NIST solicits presentations, case studies, papers, and participation from any interested parties including vendors, systems architects, integerators, researchers, and users. Please contact the organizers with any proposed subject matter. We will then develop the program in response to the submissions received. We intend that material finally presented will be posted to the workshop web site. NIST encourages presentations and reports on preliminary work that participants plan to publish elsewhere.
We will produce a preliminary program early in the New Year and finalize it before April.
Who Should Attend
The workshop is aimed at agencies and individuals who use latent fingerprints (or latent-like fingerprints) in automated searches. This includes Federal Government agencies, law enforcement agencies, vendors of AFIS systems. Representatives from academia who are heavily involved in biometrics might also find this workshop useful.