International Conference on Information and Knowledge Technology

Home / شانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش

Integrating Wasserstein GANs for High-Speed Transformer-Based Neural Machine Translation

Authors :

Parisa Nekoogol¹ Mostafa Salehi²

1- دانشگاه تهران 2- دانشگاه تهران

Keywords :

Neural Machine Translation،Generative Adversarial Networks،Reinforcement Learning،Transformer

Abstract :

Neural machine translation (NMT), a key achievement in natural language processing (NLP), continues to face challenges such as producing low-quality output for complex sentences and lacking natural fluency. This study aimed to improve machine translation quality by integrating Generative Adversarial Networks (GANs) with an NMT model. Initially, the baseline NMT model, derived from previous research and based on recurrent neural networks (RNNs), was reconstructed and implemented. Subsequently, this architecture was replaced with the advanced Transformer architecture, and the system was developed using a Wasserstein Generative Adversarial Network (WGAN). To overcome the crucial problem of textual data discontinuity (non-differentiability), the Self-Critical Sequence Training (SCST) method, a reinforcement learning (RL) algorithm, was employed. A core objective was to analyze the performance benefits of adversarial training when applied to a robust Transformer-based generator. The research concluded that while adversarial training enhances the model's performance in generating more fluent translations, this particular improvement is more substantial and notable for models based on recurrent neural networks compared to the Transformer architecture.