<%@LANGUAGE="JAVASCRIPT" CODEPAGE="65001"%> NIST Speech Group Website
Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology


  • Multimodal Information Group Home
  • Benchmark Tests
  • Tools
  • Test Beds
  • Publications
  • Links
  • Contacts
  • 2001 Speaker Recognition Evaluation

    The 2001 NIST Speaker Recognition evaluation is part of an ongoing series of yearly evaluations conducted by NIST. These evaluations provide an important contribution to the direction of research efforts and the calibration of technical capabilities. They are intended to be of interest to all researchers working on the general problem of text independent speaker recognition. To this end the evaluation was designed to be simple, to focus on core technology issues, to be fully supported, and to be accessible.

    Evaluation Documentation

  • Updated 03/01/01 (version 05.9):
  • Modification to support model identification with the extended data task.
  • The Evaluation Plan for the 2001 NIST Speaker Recognition Evaluation is available in postscript and pdf format.

  • See the paper "Some Experiments on Idiolectal Differences among Speakers (in postscript and pdf format) for motivation behind the extended training task.

  • Supporting documentation for the evaluation test material, NIST Speech Discs r65_1_1 through r65_9_1.

  • Protocol for submitting system results
  •        Supporting Ahumada DevTest Documentation

  • MIT-LL has provided a table documenting handset label type and probability for the Ahumada developtment data Format per line:

    < basefilename > MALE 1 < HS > Pr(HS)

    HS := CARB|ELEC
  • Schedule

    • February 24, 2001 - Deadline for signing up to participate in the evaluation

    • March 12, 2001 - Evaluation data to arrive at participating sites
    • April 9, 2001 - Evaluation results due at NIST
    • April 30, 2001 - Deadline for registration and payment for the Speaker Recognition Evaluation Workshop.

    Evaluation Data

    At stated in the evaluation plan, NIST has created new index files to be used for the one speaker detection task. These index files are a super set of the trials used in the 2000 evaluation and were augmented to help with analysis in areas of interest.

    While the majority of the test segments are still tested against 11 putative models, some test segments will now be tested against up to 48 putative models.

    These index files will be made available below on March 12th, 2001.

    [Now Available] :: One file, all updated one-speaker detection trials (male & female).

    -OR-

    [Now Available] :: Index file for CD #3, Female data
    [Now Available] :: Index file for CD #4, Male data
    [Now Available] :: Index file for CD #5, Female data
    [Now Available] :: Index file for CD #5, Male data

    Resources for the extended data task are now available.

    The development and evaluation data used for this evaluation is provided free of charge. To obtain these data, participants are required to sign limited license agreements which restrict the use of the data for this evaluation:

  • Participants which are not members of the LDC must complete the LDC's License Agreement and return it to the LDC
  • Participants performing the optional one-speaker detection task using the Spanish Ahumada data must complete the Ahumada License Agreement (Word Document, Postscript, or PDF) and return it to:

    Dr. Javier Ortega-Garcia
    EUIT Telecomunicacion - UPM
    Ctra. Valencia km. 7
    28031 Madrid
    SPAIN

  • Scoring Software

    Modified scoring software for the tasks of tracking and segmentation is now available. This version-0 of the scoring software is designed to work with the development test sets. The official software will be a more versatile package, and will be released with the results.

    The generic segmentation scoring script which was used to evaluate the segmentation tasks of this evaluation, is now available.

    The evaluation keys are also available: main key, cell Key


     



    [ SRE Home ]

     

     

    Page Created: Month Day, Year
    Last Updated: November 4, 2008

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
    Privacy Policy | Security Notices|
    Accessibility Statement | Disclaimer | FOIA