X-Transformer: A Green Self-attention Based Machine Translation Model | Chapter 9 | Research Highlights in Science and Technology Vol. 5

 With the rapid change growth of dossier and the increasing demand for computational possessions, it is crucial to expand energy-effective and environmentally friendly deep knowledge models in AI research. This paper proposes a novel vehicle translation model created to address the growing concern on strength consumption. The projected model named X-Transformer, that refined from united states of america-of-the-art Transformer model in three facets. First, the model parameter of the encoder is compacted. Second, the encoder structure is changed by adopting two layers of the self-consideration mechanism following and reducing the point-intelligent feed forward tier to help the model understand the pertaining to syntax structure of sentences just. Third, we streamline the interpreter model size, while asserting the accuracy. Through experiments, we explain the effectiveness of the green X-Transformer in obtaining significant preparation time conditional and performance improving. The X-Transformer reaches the state-of-the-cunning result of 46.63 and 55.63 points in the BiLingual Evaluation Understudy (BLEU) metric of the World Machine Translation (WMT), from 2014, utilizing the English–German and English–French translation corpora, so outperforming the Transformer model accompanying 19 and 18 BLEU points, respectively. The heat maps of the X-Transformer reach remembrance-level precision (that is, token-to-indication attention), while the Transformer model remnants at the sentence level (i.e., indication-to-sentence attention). In addition, the X-Transformer illustrates significantly smaller training occasion, requiring singular-third of that of the original device that drives a machine.

Author(s) Details:

Huey-Ing Liu,
Department of Electrical Engineering, Fu Jen Catholic University, No. 510 Zhongzheng Rd., Xinzhuang Dist., New Taipei City-242062, Taiwan.

Wei-Lin Chen,
Department of Electrical Engineering, Fu Jen Catholic University, No. 510 Zhongzheng Rd., Xinzhuang Dist., New Taipei City-242062, Taiwan.

Please see the link here: https://stm.bookpi.org/RHST-V5/article/view/11148