Problem
MatFormer helper mode includes align_matformer_sparse_grad, but it’s not wired into DisTrO or the network payloads. Helper indices are not transmitted, and the aggregation path only aligns contiguous prefix grads.
Refs:
shared/modeling/src/distro.rs (TODO added above align_matformer_sparse_grad)
Expected
If helper mode uses sparse index selection, the receiver must scatter into full shapes for aggregation.
Possible Approach
- Extend
DistroResult/payloads to include helper indices (or a reproducible seed/epoch for reconstruction).
- Use
align_matformer_sparse_grad in the hetero-aggregation path.
- Add tests for helper-mode sparse aggregation.
Acceptance Criteria
- Helper mode works with sparse gradient transmission.
- Aggregation produces correct full-shape grads.
- Tests validate helper-mode wiring end-to-end.
Problem
MatFormer helper mode includes
align_matformer_sparse_grad, but it’s not wired into DisTrO or the network payloads. Helper indices are not transmitted, and the aggregation path only aligns contiguous prefix grads.Refs:
shared/modeling/src/distro.rs(TODO added abovealign_matformer_sparse_grad)Expected
If helper mode uses sparse index selection, the receiver must scatter into full shapes for aggregation.
Possible Approach
DistroResult/payloads to include helper indices (or a reproducible seed/epoch for reconstruction).align_matformer_sparse_gradin the hetero-aggregation path.Acceptance Criteria