This repo contains an example for how to use LightSeq to accerate the training of translation task in Fairseq, together with DeepSpeed for distributed strategies and optimizers. We provide a new trainer for translation task to connect Fairseq and DeepSpeed.
First you should install these requirements.
pip install torch ninja fairseq deepspeedThen you can train a translation task on wmt14 en2de dataset by running the following script:
sh examples/training/deepspeed/ds_fairseq_wmt14en2de.shThis script firstly download the dataset, and then run native Fairseq training script using DeepSpeed launcher without any other parameter modifications.