Easy to raise 'NaN'

[ghtwht]

Its easy to raise 'nan' error when training the translation model with 'transformer_base', have you ever encounter with this problem and how did you deal with it?

[zhaocq-nlp]

Counld you try with --dtype float32 to disable mix-precision training?

[ghtwht]

dtype is default to be float32