V0.4 Release Plan #123
Description
Release Manager
Endgame
- Code freeze: Feb. 9th, 2024
- Bug Bash date: Feb. 12th, 2024
- Release date: Feb. 23rd, 2024
Main Features
MS-AMP O3 Optimization
- 1. Support auto scaling factor tuning (ASFT) for FP8 collective communication (Related to Auto scaling factor tuning for FP8 collective communication #41, [Feature] Auto scaling factor tuning for FP8 collective communication #140)
- 2. Support PyTorch FSDP (Related to Support for MS-AMP in FSDP #122)
MS-AMP Improvement
- 1. Move extension installation from post install to setup.py (Related to Moving extension installation from post install to setup.py under project root folder #43)
- 2. Improve FP8 kernel performance in MS-AMP (Optimize performance by fuse adding high precision tensor to fp8 tensor #132)
- 3. MS-AMP support on different devices (Nvidia A100 and AMD MI300X)
MS-AMP Examples
- 1. Release the datapoints (Related to Training curve datapoints or smoothing #115)