NIST websiteMT08 Online Evaluation Platform

Adequacy judgment examples

Quick navigation links

Score: All
Score: between Half and All
Score: Half
Score: between None and Half
Score: None

Score: All (Back to top of page)

Example #1:
Reference:The Colombian President Alvaro Uribe had authorized the extradition operation on November 8.
Machine:And Colombian President Uribe allowed complete the process of handing over in the 8th of November.
Score:All
Means essentially the same:Yes
Comments:
  • "hand over" equivalent to "extradition"
  • context "Colombian President Uribe" sufficient, so that missing "Alvaro" not essential
  • inserted "complete" does not interfere significantly
  • note that disfluency is not being penalized

Example #2:
Reference:Extradition Of The Cali Cartel Boss To The United States
Machine:Cali Cartel Leader Handed Over to the United States
Score:All
Means essentially the same:Yes
Comments:
  • perfect paraphrase; changed structure does not change meaning

Example #3:
Reference:Ankara, 12/1 (AFP) - The Turkish Prime Minister Recep Tayyip Erdogan declared today, Wednesday, that Ankara will reject any pressures that the European Union may apply on it to encourage to recognize Cyprus. This comes two weeks before a summit of the heads of countries and governments of the European Union, who will decide on whether or not to start negotiations on joining with Ankara.
Machine:Ankara 1-12 (AFP) - Turkish Prime Minister Recep Tayyip Erdogan announced today, Wednesday, that Ankara will reject any pressure has been exercised by the European Union to urge the recognition of Cyprus, two weeks before the summit of Heads of State and Governments of the European Union who decide to open accession negotiations with Ankara or not.
Score:All
Means essentially the same:Yes
Comments:
  • date format not consistent, but ambiguous already in Reference

Example #4:
Reference:We shall discuss this dossier during the negotiations on joining."
Machine:Will we are discussing this issue during the Accession negotiations ."
Score:All
Means essentially the same:No
Comments:
  • date format not consistent, but ambiguous already in Reference
  • could be interpreted as a question because of word order (punctuation indicates statement though)
  • tense change

Example #5:
Reference:In Florida, the Drug Enforcement Agency refused to comment on Colombia's decision.
Machine:In Florida Drug Control Agency refused to comment on the decision to Colombia.
Score:All
Means essentially the same:No (arguably Yes)
Comments:
  • "to Columbia" unclear




Score: between Half and All (Back to top of page)

Example #1:
Reference:Adding "We wish to start negotiations during the first six months of 2005 or, if this does not work out, during the second part of the year (..) but we cannot set the starting date of negotiations in 2006."
Machine:added, " We are willing to the start of negotiations in the first six months of 2005, in the second part of the year if necessary matter. .. but we cannot identify the date of negotiations in 2006.
Score:between Half and All
Means essentially the same:Yes
Comments:
  • meaning of conditional clause changed, but clear in context
  • "start" missing, but not essential

Example #2:
Reference:Erdogan similarly rejected the possibility of starting negotiations for the establishment of a special partnership between Turkey and the European Union as an alternative to the negotiations on joining.
Machine:Ardogan also rejected the premise to reopen negotiations to set up a special partnership between Turkey and the European Union as an alternative to the Accession negotiations.
Score:between Half and All
Means essentially the same:No
Comments:
  • "premise" vs "possibility"
  • "start" vs "reopen"
  • "Erdogan" vs "Ardogan" (importance arguable)

Example #3:
Reference:Extradition Of The Cali Cartel Boss To The United States
Machine:Leader Kali Handed over to the United States
Score:between Half and All
Means essentially the same:No
Comments:
  • "cartel" missing, "Kali" can be misinterpreted as person

Example #4:
Reference:We shall discuss this dossier during the negotiations on joining."
Machine:will discuss this issue during the accession negotiations. "
Score:between Half and All
Means essentially the same:No (arguably Yes)
Comments:
  • subject missing - may not be essential

Example #5:
Reference:Bogota, 12/4 (AFP) - A reporter for Agence France Presse has reported that the boss of the Cali Cartel (southwest), Gilberto Rodriguez Orejuela, one of the most notorious drug traffickers in the world, was handed over Friday evening to the United States.
Machine:Bogota 4-12 (AFP)- Agence France Presse correspondent reported that the leader of Cali cartel (south-west), Gilberto Rodriguez, one of the largest drug traffickers in the world, peace in the evening of Friday to the United States.
Score:between Half and All
Means essentially the same:No
Comments:
  • part of the name missing, but unambiguous in context
  • main concept "hand over" substituted with misleading "peace"

Example #6:
Reference:Erdogan said on this matter "this proposal, which we cannot accept, is incompatible with a serious organization like the European Union."
Machine:He Erdogan in this topic that "this proposal, which can not accept it, not compatible with the seriousness such as the European Union."
Score:between Half and All
Means essentially the same:Yes
Comments:
  • "organization" missing, but not essential
  • subject of relative clause missing, but clear by default
  • "said" missing, but clear from context




Score: Half (Back to top of page)

Example #1:
Reference:The decree, which was signed by the President and the Minister of the Interior and Justice, Sabas Pretelt, stated that Colombia agreed to the extradition of Rodriguez Orejuela on the basis of charges of "exporting drugs to the United States, possessing and distributing cocaine, and money-laundering through an international network of fictitious institutions."
Machine:A decree signed by the President and Minister of Interior and Justice that Colombia has agreed to hand over based on charges of "the export of drugs to the United States and possessing cocaine and distribution and money laundering through international network of narcissism institutions."
Score:Half
Comments:
  • both person names missing
  • "stated" missing
  • "fictitious" substituted with misleading "narcissism"

Example #2:
Reference:Erdogan told the NTV television station that "It is not possible for the European Union to talk to us by imposing new conditions on us regarding Cyprus.
Machine:He Ardogan station "that since" television that "the European Union could not address by imposing new conditions on the Cyprus.
Score:Half
Comments:
  • "NTV" missing
  • object of "address" missing
  • "told" missing
  • "Erdogan" vs "Ardogan" (importance arguable)

Example #3:
Reference:He then underwent a medical examination by a police physician.
Machine:And then subjected to tests carried out by a police pathologist.
Score:Half
Comments:
  • subject missing
  • "medical" missing
  • "pathologist" vs "physician"

Example #4:
Reference:Orejuela, who is 64 years old and has been under arrest since 1995, was transferred from La Picota Prison and taken in an armored vehicle to a military base located near Bogota International Airport where he was handed over to officials of the American Drug Enforcement Agency.
Machine:and transport (64 years) detained since 1995 from in armored mechanism to neighboring Bogota International Airport, where handed over to military officials in the American Agency for combating drugs.
Score:Half
Comments:
  • person name missing
  • prison name missing
  • unclear that "64 years" refers to a person's age

Example #5:
Reference:Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida.
Machine:When was taken to calm began of the American plane that will carry him to Miami in Florida.
Score:Half
Comments:
  • person name missing
  • "taken to calm" misleading




Score: between None and Half (Back to top of page)

Example #1:
Reference:117 / Urgent: Palestinian leadership condemns operation in Eli Sinai settlement
Machine:117 / an urgent : The leadership <non-translated text>'s settlement
Score:between None and Half

Example #2:
Reference:No humans were infected with the virus.
Machine:He did not hurt anyone with the virus.
Score:between None and Half

Example #3:
Reference:Jeddah (Saudi Arabia) 2/13 (AFP) - The European Union High Representative for Foreign Policy, Javier Solana, conveyed a message of "respect" to the Islamic World, while the Secretary General of the Organization of the Islamic Conference, Ekmaleddin Ihsanoglu, called on the Europeans to adopt legislation to combat "anti-Islamism."
Machine:Jeddah (Saudi Arabia) 2 - 13 (AFP) - pregnancy in the European Union's High Representative for Foreign Policy Javier Solana letter "respect" to the Islamic world and invited the Secretary-General of the Organization of the Islamic Conference to complete debt Ihsan ogly Europeans to adopt legislation to combat "anti-Islam."
Score:between None and Half

Example #4:
Reference:Roberto Calderoli does not intend to apologize. The newspaper Corrieri della Sera quoted him today, Saturday, saying, "I do not feel responsible for these deaths."
Machine:The intended Roberto one apologize. newspaper Corriere della Sera "today, Saturday, as saying" I do not feel responsible for these deaths. "
Score:between None and Half
Comments:
  • last name "Calderoli" missing
  • negation missing on first part; first part is not intelligible overall

Example #5:
Reference:39 / Jordanian monarch meets Syrian Prime Minister
Machine:39 / the king <non-translated text> the president of the ministers <non-translated text>
Score:between None and Half




Score: None (Back to top of page)

Example #1:
Reference:However, no plane has been reported to have been hit.
Machine:A desert end what h e benefits did not reply one affliction this flying .
Score:None

Example #2:
Reference:Syrian President welcomes Moor Atinus
Machine:reactivating the chairman to transmit moratinos
Score:None

Example #3:
Reference:Authorities should establish and enhance emergency mechanisms.
Machine:And a residence and the consolidation of Al Tourae mechanisms are necessary for the authorities .
Score:None

Example #4:
Reference:During the festival the Syrian film producers including Mohammed Malas, Abdul Lattif Abdul Hamid and the Lebanese producer Burhan Alwiah will be honored.
Machine:it will during the carifesta sanjaasuren syrians mohammed abdul hamid tracks abdel latif berhan <non-translated text>.
Score:None

Example #5:
Reference:In a statement to the Egyptian government press, a high-ranking security source denied the report released by the press. According to this report, quoting security sources and Egyptian travellers returning from Libya, the postponement of the visit of Colonel alQuaddafi, the leader of the Libyan Revolution, has been caused by an abortive assassination attempt.
Machine:and the papers are in the statement of his publication <non-translated text> " the origin of my security refuted <non-translated text>'s slender , they moved <non-translated text> away from confiscated desideratum or <non-translated text>'s travelers a two comers from libya regarding the postponement of the colonel visit <non-translated text> is whose revolution is commanding <non-translated text> because of she/you submits his sovereignty to the attempt of a manque assassination ".
Score:None
Comments:
  • parts of meaning present, but overall unintelligible