ITL's Information Access Division hosted the ninth workshop in the Text REtrieval Conference (TREC) series at NIST in Gaithersburg on November 13-16, 2000. Seventy groups representing 17 countries participated in the workshop to discuss the results of a yearlong cycle of testing that was conducted by the division's Retrieval Group.

In addition to the breakthrough in question-answer technology, a second emphasis in this year's TREC was an examination of the infrastructure required to evaluate Web search engines. The Web differs from other data collections used in TREC in a variety of ways: size; variety of subject matter, media types, languages, and presentation styles; lack of specific editorial control; explicit links among documents; frequent changes to documents; and generated content. Any of these differences may impact retrieval effectiveness, and the challenge facing the retrieval community is building appropriate test suites that can isolate the effects of these different factors. A highlight of the conference was an invited talk by Andrei Broder, Vice President of Research and Chief Scientist of AltaVista Company, which provided a perspective on evaluation from a commercial Web search engine company. The Web site is http://trec.nist.gov.

CONTACT: Ellen Voorhees, ext. 3761