Perpetual Pre-training of G and RaGAN

[CristianCerasuoloo]

Pull Request

Description

Add of new 2 features:

1. Optionally pre-train the generator G with content loss for un undefined number of steps by using -1 as g_pretrain_steps configurations
2. Implemented 'Relativistic Average GAN' as the original paper of ESRGAN. Optionally usable by setting config key Training/Losses/relativistic_average_d to True

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Many training tests.

Checklist:

• [X ] I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• [X ] My changes generate no new warnings
• I have added tests that prove that my feature works
• New and existing unit tests pass locally with my changes
• [ X] Any dependent changes have been merged and published in downstream modules

[simon-donike]

Super cool Cristian, thanks for contributing 💪