Transformer - Attention Is All You Need

Brief introduction to the famous paper about Transformers: Attention Is All You Need Transformer Paper: Attention Is All You Need Github link of Transformer 1. Background encoder-decoder self attention feed-forward network positional encoding Dataset: WMT 2014 English-to-French translation task Evaluation: BLEU (bilingual evaluation understudy) BLEU computes the modified precision metric using n-grams to measure the similarity between the candidate text and reference text. The idea is, if a word in reference is already matched, this word cannot be matched again.