Skip to content

docs(models): add Qwen3.5-4B page #6

Description

@xiaguan

The models/ section currently only has a Qwen3-4B page (src/content/docs/models/qwen3-4b.md and the matching sidebar entry in astro.config.mjs). Qwen3.5-4B is a supported model line in openinfer (openinfer-qwen35-4b crate, --features qwen35-4b) and should have its own page.

What's missing

  • src/content/docs/models/qwen35-4b.md
  • Sidebar entry under the Models group in astro.config.mjs

Suggested content

Mirror the structure of qwen3-4b.md, but cover the Qwen3.5-4B specifics:

  • Build & launchcargo run --release --features qwen35-4b -- --model-path models/Qwen3.5-4B
  • Build-time Python + Triton AOT — the qwen35-4b feature is the only build step that needs Python (via openinfer-kernels/tools/triton/gen_triton_aot.py); document OPENINFER_TRITON_PYTHON and the fallback chain (.venv/bin/pythonpython3python)
  • Architecture — hybrid layout: 24 linear (SLA) layers + 8 full-attention layers, and how that differs from Qwen3-4B's full attention
  • Accuracy gatescargo test --release -p openinfer-qwen35-4b --features qwen35-4b --test hf_golden_gate (with OPENINFER_TEST_MODEL_PATH=models/Qwen3.5-4B); note that exact-text JSON baselines are retired and e2e_scheduler is the liveness gate
  • Performance — benchmark snapshots when available

Context

  • Repo README confirms Qwen3.5-4B is a first-class model line.
  • AGENTS.md in the main repo has the canonical build/test commands.

Anyone with cycles, feel free to pick this up — happy to review the PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions