1st Version : 2020/12/10

Background Knowledge

BLEU (bilingual evaluation understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another. Native Quality is considered to be the correspondence between a machine’s output and that of a human: “the closer a machine translation is to a professional human translation, the better it is” – this is the central idea behind BLEU. BLEU was one of the first metrics to claim a high correlation with human judgements of quality, and remains one of the most popular automated and inexpensive metrics.
Thus, for translations to serve its’ purpose properly, there is no place for any mistranslations, omissions, additions and other translation issues, and all translations need to be done by native resources.

General Guidelines

BLEU test translation is often provided to evaluate the machine translation’s performance in e-commerce such as Lazada. The product page mainly includes Product Title

Product titles represent a number of phrases that describe products, product shipment methods, sales discount, etc. usually without any punctuation separating them. During translation, it is vital to discriminate those syntactical phrases one by one and translate each of the phrases in the title.

Lazada Title BLEU Evaluation Set - 图1

  1. Determine product nouns
    Open the product link to see the product category bread crumbs and product images to determine the correct product noun. Other words other than product nouns can be viewed as product modifiers.
    e.g.
    Category: Kitchen Tools & Gadgets
    Product link:https://www.aliexpress.com/item/Multifunctional-Kitchen-Tool-Fondue-Cheese-Grater-Lemon-Zester-with-Non-slip-Handle/32720504616.html
    Product title: Multifunctional Kitchen Tool Fondue Cheese Grater Lemon Zester with Non-slip Handle
    Specific operating steps:

According to the category information “Kitchen Tools & Gadgets”, and the picture of the goods page, determine the product name is grater, so product nouns of the title are - “Tool”, “Grater” and “Zester”.
The other words in the title “Multifunctional Kitchen”, “Fondue Cheese”, “Lemon” and “with Non-slip Handle” are a modifiers of the product nouns.
Operating results:
Product nouns: “Tool”, “Grater”, “Zester”
Product modifiers: “Multifunctional Kitchen”, “Fondue Cheese”, “Lemon”, “with Non-slip Handle”

  1. Combining syntactical phrases
    Combine 1 product noun and product modifiers before it into one syntactical phrase. If there are product modifiers after the last product noun in the title, they should be included into one syntactical phrase with the forgoing product name. Product modifiers that start with a preposition should be included into one syntactical phrase with the forgoing product name.
    e.g. On the basis of operating step 1
    Product nouns: “Tool”, “Grater”, “Zester”
    Product modifiers: “Multifunctional Kitchen”, “Fondue Cheese”, “Lemon”, “with Non-slip Handle”
    Specific operating steps:
    Modifier “Multifunctional Kitchen” is before “Tool” and there are other 2 product nouns after it - “Grater”, “Zester”, so the first syntactical phrase could be combined - “Multifunctional Kitchen Tool”.
    Modifier “Fondue Cheese” is before “Grater” and there are other product noun after it - “Zester”, so the second syntactical phrase could be combined - “Fondue Cheese Grater”.
    Modifier “Lemon” is before “Zester” and there are no other product noun after it, so the third syntactical phrase could be combined - “Lemon Zester with Non-slip Handle”.
    Operating results:

    “Multifunctional Kitchen Tool”, “Fondue Cheese Grater”, “Lemon Zester with Non-slip Handle”.

  2. Translating phrases in the title
    Translate syntactical phrases of the source one by one. The number and the order of phrases in the target should be the same as in the source title. Capitalize the first letter of each notional word.
    e.g. On the basis of operating step 2
    Title phrases: “Multifunctional Kitchen Tool”, “Fondue Cheese Grater”, “Lemon Zester with Non-slip Handle”.
    Specific operating steps:
    Translate phrases one by one and combine them together according to their order in the source title.

Special Notes:

  1. Capitalization: capitalize the first letter of each notional word ( ZH-EN, ZH-ID ); follow the local style guide( ZH-VN, ZH-TH ).
  2. Punctuation: follow source as long as it meets the requirements of target language style guide.
    For example,
    (1) If there is no period at the end of the sentence, do NOT add one in the translation.
    (2) If the punctuation is wrongly used at the beginning or the end of the source , do NOT correct it in the translation and keep consistency with the source.
    (3) If one sentence in the source is split into two (or two combined into one) in the translation, you may add (or delete) new punctuation for the sake of fluency.
  3. Minor issue: If the source has minor spelling or grammar issue which is recognizable, please translate it in a correct way. For example,
    (1) It is very goood.-> It is very good.
    (2) I am really interesting in it. -> I am really interested in it.
    But if the source has serious spelling or grammar issue, which makes it unintelligible, please mark “####” in the translation as source error.

  4. Duplication in source should not be fixed in translation:

    ladies womens shoes -> дамские женские туфли / zapatos de mujer de señora
    womens shoes for ladies -> женские туфли для дам / zapatos de mujer para señora
    好好好好好 -> good good good good good
    Huawei/华为 Mate 20 手机壳 -> Huawei/Huawei Mate 20 phone case

  5. Brand name:

    (1) Use the local official brand translation if possible.
    (2) If there’s no local translation for the brand, use the English brand name that is internationally acceptable.
    (3) If the brand is less well-known, please use the English brand name or the PinYin (if the source is in Chinese)
    So the priority should be like: Local official brand translation -> Official brand name in English -> Pinyin （If the Chinese brand is less well-known）

  6. No extra information**:** Translate fluently but do NOT paraphrase or provide explanatory information.

Source: Hi Tiffany, The material is made out of 100% food-grade silicone.
Wrong translation: Chào Tiffany, chất liệu của sản phẩm là 100% silicon đa năng (loại dùng trong quá trình bảo quản và vận chuyển thức ăn)
Explanation: “loại dùng trong quá trình bảo quản và vận chuyển thức ăn” is explanatory information and belongs to Addition issue.

  1. Abbreviation: Please follow the target language style guide. If the abbreviation is widely used in the local, please keep the abbreviation and do not add explanation information unless it is included in source.

    E.g. Source: WTO
    Correct translation 1: WTO
    Correct translation 2: 世界贸易组织
    Wrong translation 1: 世界贸易组织 (WTO)
    Wrong translation 2: WTO（世界贸易组织）
    E.g. Source: WTO (World Trade Organization)
    Correct translation: WTO（世界贸易组织）

  2. Non specified source language: For other language than the assigned source language, you can translate it into target language if you understand what’s the real meaning of that language and add a note to make it clear, otherwise, keep as it is.

  3. Style guide: For more style information, please refer to https://www.yuque.com/alibaba-translate/hgdw9e/gpzad3

QA Notes:

please refer to the guideline in https://www.yuque.com/alibaba-translate/htquality/hxy3cv
Please note that

  1. all issues about space can be ignored, for example, adding double blank and no space between two half-width characters;
  2. In title translation, if you find the translator wrongly added the punctuation in the whole translation, please contact with PM.