New participants are required to complete a dry run evaluation using the TDT4 Corpus and last year's (TDT2003) index files. The dry run process is very simple and is actually a milestone that all new participants must go through internally to build a system capable of being run for the TDT evaluation: specifically to process a test corpus and produce scorable system output.

We have required all previous new participants to complete a dry run evaluation to ensure they have built systems that implement the tasks and can produce the specified system output which can be evaluated using the TDT evaluation suite. Dry run system results, (the error rates/detection costs) are not of interest to NIST, so developing a competitive system is not a requirement for the dry run. A passing dry run grade is determined soley by whether or not NIST can score your output.

After NIST has successfully scored your submission, you will be eligible to recieve the TDT4 topic annotations from the LDC and begin system development using the full TDT4 corpus.

To complete the dry run, complete these steps:

1. Obtain the TDT4 text corpus from the LDC
2. Download last year's evaluation index files from the URL ftp://jaguar.ncsl.nist.gov/tdt/tdt2003/eval/tdt2003_indexfiles_v1.20030902.tgz
3. Run a link detection, tracking, topic detection or new event detection system on the data generating data formatted as define in the 2003 evaluation plan, (see the 2003 Evaluation Plan).
4. Submit your system output to NIST for scoring using last years submission instructions found at the URL: ftp://jaguar.ncsl.nist.gov/tdt/tdt2003/eval/submit.htm.
5. After NIST successful scores the submission, NIST will tell LDC to release the TDT4 topic annotations to you.