Adequacy judgment examples
Quick navigation links
Score: All
Score: between Half and All
Score: Half
Score: between None and Half
Score: None
Example #1:
| Reference: | The Colombian President Alvaro Uribe had authorized the extradition operation on November 8. |
|---|
| Machine: | And Colombian President Uribe allowed complete the process of handing over in the 8th of November. |
|---|
| Score: | All |
|---|
| Means essentially the same: | Yes |
|---|
| Comments: | - "hand over" equivalent to "extradition"
- context "Colombian President Uribe" sufficient, so that missing "Alvaro" not essential
- inserted "complete" does not interfere significantly
- note that disfluency is not being penalized
|
|---|
Example #2:
| Reference: | Extradition Of The Cali Cartel Boss To The United States |
|---|
| Machine: | Cali Cartel Leader Handed Over to the United States |
|---|
| Score: | All |
|---|
| Means essentially the same: | Yes |
|---|
| Comments: | - perfect paraphrase; changed structure does not change meaning
|
|---|
Example #3:
| Reference: | Ankara, 12/1 (AFP) - The Turkish Prime Minister Recep Tayyip Erdogan declared today, Wednesday, that Ankara will reject any pressures that the European Union may apply on it to encourage to recognize Cyprus. This comes two weeks before a summit of the heads of countries and governments of the European Union, who will decide on whether or not to start negotiations on joining with Ankara. |
|---|
| Machine: | Ankara 1-12 (AFP) - Turkish Prime Minister Recep Tayyip Erdogan announced today, Wednesday, that Ankara will reject any pressure has been exercised by the European Union to urge the recognition of Cyprus, two weeks before the summit of Heads of State and Governments of the European Union who decide to open accession negotiations with Ankara or not.
|
|---|
| Score: | All |
|---|
| Means essentially the same: | Yes |
|---|
| Comments: | - date format not consistent, but ambiguous already in Reference
|
|---|
Example #4:
| Reference: | We shall discuss this dossier during the negotiations on joining." |
|---|
| Machine: | Will we are discussing this issue during the Accession negotiations ." |
|---|
| Score: | All |
|---|
| Means essentially the same: | No |
|---|
| Comments: | - date format not consistent, but ambiguous already in Reference
- could be interpreted as a question because of word order (punctuation indicates statement though)
- tense change
|
|---|
Example #5:
| Reference: | In Florida, the Drug Enforcement Agency refused to comment on Colombia's decision. |
|---|
| Machine: | In Florida Drug Control Agency refused to comment on the decision to Colombia. |
|---|
| Score: | All |
|---|
| Means essentially the same: | No (arguably Yes) |
|---|
| Comments: | |
|---|
Example #1:
| Reference: | Adding "We wish to start negotiations during the first six months of 2005 or, if this does not work out, during the second part of the year (..) but we cannot set the starting date of negotiations in 2006." |
|---|
| Machine: | added, " We are willing to the start of negotiations in the first six months of 2005, in the second part of the year if necessary matter. .. but we cannot identify the date of negotiations in 2006. |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | Yes |
|---|
| Comments: | - meaning of conditional clause changed, but clear in context
- "start" missing, but not essential
|
|---|
Example #2:
| Reference: | Erdogan similarly rejected the possibility of starting negotiations for the establishment of a special partnership between Turkey and the European Union as an alternative to the negotiations on joining. |
|---|
| Machine: | Ardogan also rejected the premise to reopen negotiations to set up a special partnership between Turkey and the European Union as an alternative to the Accession negotiations. |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | No |
|---|
| Comments: | - "premise" vs "possibility"
- "start" vs "reopen"
- "Erdogan" vs "Ardogan" (importance arguable)
|
|---|
Example #3:
| Reference: | Extradition Of The Cali Cartel Boss To The United States |
|---|
| Machine: | Leader Kali Handed over to the United States |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | No |
|---|
| Comments: | - "cartel" missing, "Kali" can be misinterpreted as person
|
|---|
Example #4:
| Reference: | We shall discuss this dossier during the negotiations on joining." |
|---|
| Machine: | will discuss this issue during the accession negotiations. " |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | No (arguably Yes) |
|---|
| Comments: | - subject missing - may not be essential
|
|---|
Example #5:
| Reference: | Bogota, 12/4 (AFP) - A reporter for Agence France Presse has reported that the boss of the Cali Cartel (southwest), Gilberto Rodriguez Orejuela, one of the most notorious drug traffickers in the world, was handed over Friday evening to the United States. |
|---|
| Machine: | Bogota 4-12 (AFP)- Agence France Presse correspondent reported that the leader of Cali cartel (south-west), Gilberto Rodriguez, one of the largest drug traffickers in the world, peace in the evening of Friday to the United States. |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | No |
|---|
| Comments: | - part of the name missing, but unambiguous in context
- main concept "hand over" substituted with misleading "peace"
|
|---|
Example #6:
| Reference: | Erdogan said on this matter "this proposal, which we cannot accept, is incompatible with a serious organization like the European Union." |
|---|
| Machine: | He Erdogan in this topic that "this proposal, which can not accept it, not compatible with the seriousness such as the European Union." |
|---|
| Score: | between Half and All |
|---|
| Means essentially the same: | Yes |
|---|
| Comments: | - "organization" missing, but not essential
- subject of relative clause missing, but clear by default
- "said" missing, but clear from context
|
|---|
Example #1:
| Reference: | The decree, which was signed by the President and the Minister of the Interior and Justice, Sabas Pretelt, stated that Colombia agreed to the extradition of Rodriguez Orejuela on the basis of charges of "exporting drugs to the United States, possessing and distributing cocaine, and money-laundering through an international network of fictitious institutions." |
|---|
| Machine: | A decree signed by the President and Minister of Interior and Justice that Colombia has agreed to hand over based on charges of "the export of drugs to the United States and possessing cocaine and distribution and money laundering through international network of narcissism institutions." |
|---|
| Score: | Half |
|---|
| Comments: | - both person names missing
- "stated" missing
- "fictitious" substituted with misleading "narcissism"
|
|---|
Example #2:
| Reference: | Erdogan told the NTV television station that "It is not possible for the European Union to talk to us by imposing new conditions on us regarding Cyprus. |
|---|
| Machine: | He Ardogan station "that since" television that "the European Union could not address by imposing new conditions on the Cyprus. |
|---|
| Score: | Half |
|---|
| Comments: | - "NTV" missing
- object of "address" missing
- "told" missing
- "Erdogan" vs "Ardogan" (importance arguable)
|
|---|
Example #3:
| Reference: | He then underwent a medical examination by a police physician. |
|---|
| Machine: | And then subjected to tests carried out by a police pathologist. |
|---|
| Score: | Half |
|---|
| Comments: | - subject missing
- "medical" missing
- "pathologist" vs "physician"
|
|---|
Example #4:
| Reference: | Orejuela, who is 64 years old and has been under arrest since 1995, was transferred from La Picota Prison and taken in an armored vehicle to a military base located near Bogota International Airport where he was handed over to officials of the American Drug Enforcement Agency. |
|---|
| Machine: | and transport (64 years) detained since 1995 from in armored mechanism to neighboring Bogota International Airport, where handed over to military officials in the American Agency for combating drugs. |
|---|
| Score: | Half |
|---|
| Comments: | - person name missing
- prison name missing
- unclear that "64 years" refers to a person's age
|
|---|
Example #5:
| Reference: | Orejuela appeared calm as he was being led to the American plane that was to carry him to Miami in Florida. |
|---|
| Machine: | When was taken to calm began of the American plane that will carry him to Miami in Florida. |
|---|
| Score: | Half |
|---|
| Comments: | - person name missing
- "taken to calm" misleading
|
|---|
Example #1:
| Reference: | 117 / Urgent: Palestinian leadership condemns operation in Eli Sinai settlement |
|---|
| Machine: | 117 / an urgent : The leadership <non-translated text>'s settlement |
|---|
| Score: | between None and Half |
|---|
Example #2:
| Reference: | No humans were infected with the virus. |
|---|
| Machine: | He did not hurt anyone with the virus. |
|---|
| Score: | between None and Half |
|---|
Example #3:
| Reference: | Jeddah (Saudi Arabia) 2/13 (AFP) - The European Union High Representative for Foreign Policy, Javier Solana, conveyed a message of "respect" to the Islamic World, while the Secretary General of the Organization of the Islamic Conference, Ekmaleddin Ihsanoglu, called on the Europeans to adopt legislation to combat "anti-Islamism." |
|---|
| Machine: | Jeddah (Saudi Arabia) 2 - 13 (AFP) - pregnancy in the European Union's High Representative for Foreign Policy Javier Solana letter "respect" to the Islamic world and invited the Secretary-General of the Organization of the Islamic Conference to complete debt Ihsan ogly Europeans to adopt legislation to combat "anti-Islam." |
|---|
| Score: | between None and Half |
|---|
Example #4:
| Reference: | Roberto Calderoli does not intend to apologize. The newspaper Corrieri della Sera quoted him today, Saturday, saying, "I do not feel responsible for these deaths." |
|---|
| Machine: | The intended Roberto one apologize. newspaper Corriere della Sera "today, Saturday, as saying" I do not feel responsible for these deaths. " |
|---|
| Score: | between None and Half |
|---|
| Comments: | - last name "Calderoli" missing
- negation missing on first part; first part is not intelligible overall
|
|---|
Example #5:
| Reference: | 39 / Jordanian monarch meets Syrian Prime Minister |
|---|
| Machine: | 39 / the king <non-translated text> the president of the ministers <non-translated text> |
|---|
| Score: | between None and Half |
|---|
Example #1:
| Reference: | However, no plane has been reported to have been hit. |
|---|
| Machine: | A desert end what h e benefits did not reply one affliction this flying . |
|---|
| Score: | None |
|---|
Example #2:
| Reference: | Syrian President welcomes Moor Atinus |
|---|
| Machine: | reactivating the chairman to transmit moratinos |
|---|
| Score: | None |
|---|
Example #3:
| Reference: | Authorities should establish and enhance emergency mechanisms. |
|---|
| Machine: | And a residence and the consolidation of Al Tourae mechanisms are necessary for the authorities . |
|---|
| Score: | None |
|---|
Example #4:
| Reference: | During the festival the Syrian film producers including Mohammed Malas, Abdul Lattif Abdul Hamid and the Lebanese producer Burhan Alwiah will be honored. |
|---|
| Machine: | it will during the carifesta sanjaasuren syrians mohammed abdul hamid tracks abdel latif berhan <non-translated text>. |
|---|
| Score: | None |
|---|
Example #5:
| Reference: | In a statement to the Egyptian government press, a high-ranking security source denied the report released by the press. According to this report, quoting security sources and Egyptian travellers returning from Libya, the postponement of the visit of Colonel alQuaddafi, the leader of the Libyan Revolution, has been caused by an abortive assassination attempt. |
|---|
| Machine: | and the papers are in the statement of his publication <non-translated text> " the origin of my security refuted <non-translated text>'s slender , they moved <non-translated text> away from confiscated desideratum or <non-translated text>'s travelers a two comers from libya regarding the postponement of the colonel visit <non-translated text> is whose revolution is commanding <non-translated text> because of she/you submits his sovereignty to the attempt of a manque assassination ". |
|---|
| Score: | None |
|---|
| Comments: | - parts of meaning present, but overall unintelligible
|
|---|
|
