feat: performance improvement and Qwen3 support

[drunkcoding]

Description

Major changes for performance improvement

Motivation

• Support latest QWen3 MoE model
• Overlap hidden states gather with expert copy
• Reduce torch kernel launch overhead

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

Checklist

• I have read the CONTRIBUTION guide.
• I have updated the tests (if applicable).
• I have updated the documentation (if applicable).

[lausannel]

we might not use a tensor item every time, so constructing a tensor just to query its itemsize() might be unnecessarily expensive.

inline size_t torch_dtype_size(int dtype) {
  switch (dtype) {
    case DTYPE_FLOAT32:
      return 4;
    case DTYPE_FLOAT16:
      return 2;
    case DTYPE_BFLOAT16:
      return 2;
    case DTYPE_FP8_E4M3FN:
      return 1;
    default:
      throw std::invalid_argument("Unknown dtype in torch_dtype_size()");
  }
}

[lausannel]

Just wondering—was there a specific reason for removing this?

[Copilot]

Pull Request Overview

This PR adds support for the QWen3 MoE model and implements several performance improvements by overlapping expert copying, introducing fused kernels, CUDA graph support, and refined memory allocators.

• Added Qwen3MoeForCausalLM to model mappings and constants
• Refactored expert modules with a DECLARE_MODULE macro and introduced MoEMLP using CUDA graphs
• Overhauled caching allocators and fused MLP kernels for reduced overhead
• Updated examples, documentation, and CI workflows for Ubuntu 22.04

Reviewed Changes

Copilot reviewed 45 out of 45 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
moe_infinity/common/constants.py	Added QWen3 model to imports and mappings
examples/interface_example.py	Switched to chat template and cleaned dataset loading
core/parallel/expert_module.h	Refactored expert modules with macros and new fields
core/memory/caching_allocator.h	Introduced templated caching allocator
core/model/fused_mlp.{h,cu}	Added fused MLP CUDA kernel and launcher
.github/workflows/*	Upgraded Ubuntu runner from 20.04 to 22.04

Comments suppressed due to low confidence (1)

core/parallel/expert_dispatcher.h:49

• [nitpick] The default num_threads was reduced from 8 to 1, which may degrade parallel throughput. If this is intentional, please document the rationale or expose it as a configurable parameter.

explicit ExpertDispatcher(int num_experts, int num_layers, int dtype, int expert_type, int num_threads = 1);