<%@LANGUAGE="JAVASCRIPT" CODEPAGE="65001"%> NIST Speech Group Website
Information Technology Lab, Information Access Division NIST: National Institute of Standards and Technology


  • Multimodal Information Group Home

  • Benchmark Tests

  • Tools

  • Test Beds

  • Publications

  • Links

  • Contacts
    •

    1997 Conversational Telephone Recognition Evaluation

    The 1997 Hub-5NE Spring Evaluation
    This evaluation is dedicated to the advancement of speech recognition technology for languages other than English, and specifically this year for Arabic, German, Mandarin, and Spanish. It focuses also on issues related to porting recognition technology to new languages, to system generality, and to language commonalties and universals.

    Evaluation Plan
    Data Files

    The 1997 Hub-5E Spring Evaluation

    The Evaluation Plan
    The 1997 Hub-5E Spring Evaluation Plan

    • Will Define the Task
    • Will Review the Technical Objectives
    • Will Provide all the Information Needed for Participation
    Development Data
    Two common development data subsets of the April '96 Callhome English corpus have been defined. The purpose is to provide a means for common reporting of development results.
  • Subset 1 where speaker variation is important: Selected across all conversations
    Select first 30 seconds from each conversation side
    Segmentation based on the reference answers, defined by NIST

    • View the Down-Loadable Reference File for DevTest Subset 1 now

  • Subset 2 where speaker adaptation is important: Select 7 conversations (selecting both sides of the conversation)
    BBN selected the 7 conversations based on balancing: Gender
    Variance of Word-Error Rate

    • View the Down-Loadable Reference File for DevTest Subset 2 now

    Evaluation Data

    Echo Cancellation Software will be applied to the evaluation data taken from the SwitchBoard-II corpus.

    • The echo cancelling software (ec_v2.5.tar.gz) that is applied to telephone data, may be obtained from Mississippi State University.
    • The LDC has provided a perl script (mu_ec.perl) that will take a sphere-headered, 2-channel mu-law waveform file as input, apply the MSU/ISIP echo cancellation software, and produce a sphere-headered, 2-channel mu-law waveform file as output. In the process, it adds the following to the sphere header of the output file: echo_cancellation ec-v2.5
    • In running the echo canceller on sparcs (ss20, SPARCserver-1000), it takes between 3 and 4 times realtime to operate.

    [ Home ]

     

     

    Page Created: August 16, 2007
    Last Updated: November 4, 2008

    Multimodal Information Group is part of IAD and ITL
    NIST is an agency of the U.S. Department of Commerce
         		Privacy Policy | Security Notices|
    Accessibility Statement | Disclaimer | FOIA