Web-based Tool Instructions

Once registered to participate, you will be assigned a user ID and a password. Please follow these steps:

1. Re-read the Introduction.
2. Carefully read the Grading instructions.
3. Complete the Training set.
4. Complete the Evaluation set (estimated date of set availability: Feb 07, 2008).

Upon finishing the training section (and as soon as the evaluation data are available), you will be able to proceed to the evaluation sections (one for each of the 2 tasks), which consist of approximately 200 Adequacy judgments and 200 Preference judgments.

You will be able to stop at any time and resume at a later point.

After you make a judgment and proceed to the next page, you will not be able to go back and change any judgments.

If at any time you need to remind yourself of the instructions for making judgments or for using the evaluation tool, you will be able to access them by clicking the Instructions link at the top left of an evaluation page.

On each page, you will make your judgment by selecting the appropriate radio button, then click the Proceed to next segment button.