[Question] Enabling backward communication in coarse-grained workload

[YutongJau]

Regarding the coarse-grained workload generation, I noticed that the backward communication is currently set to NONE. After reviewing the discussion in #123 , we are considering replacing NONE with ALLREDUCE in the backpropagation phase and adding the corresponding communication size (e.g., tp_comm_size) to facilitate our ns-3 experiments on network congestion and potential solutions.

Is this a recommended approach to capture backward gradient sync traffic? Would this introduce any unintended side effects or logical conflicts within the SimAI simulation framework, such as double-counting issues with the grad_param_comm node?

[tianhao909]

Hello, for reference on where to call allreduce,
you can look at the implementation in the Megatron framework.

Currently, in ns3-ali, each collective communication operator is called serially from mocknccl.

You can also refer to the changes in this PR and continue the discussion under this PR.

https://github.com/aliyun/aicb/pull/45/commits