项目作者: MojammelHossain

项目描述 :
Neural Machine Translation
高级语言: Python
项目地址: git://github.com/MojammelHossain/nmt.git
创建时间: 2020-07-23T05:21:12Z
项目社区:https://github.com/MojammelHossain/nmt

开源协议:MIT License

下载


Neural Machine Translation with RNN variants.

Introduction

Encoder-Decoder architecture for neural machine translation BN->EN.

  • Both LSTM and GRU can be used as Encoder or Decoder.
  • Scratch implementation of vocab language.py and trainer trainer.py class.
  • Implementation of Bahdanau attention decoder.

Setup

Install the following if not installed.

  • python 3.x
  • torch cuda version
  • scarceblue
  • configparser

Training

  • Keep your preprocess data in the data folder check out the sample.txt for data format.
  • Change the appropriate variable inside experiment.ini i.e. lang1, rnn, hidden_size.
  • reverse a bool variable will change the model training i.e. BN->EN to EN->BN
  • Training: python train.py

Resume Training

For resume training change the obj_path from false to logs and checkpoint false to model path i.e. path/to/model.pt.

Train New Language

For train in new language you only need to change the tokenizer in train.py. You can use spacy tokenizer or custom made tokernizer.