Skip to content

Keep model parameters for fuse GGUF export#1321

Open
abnormal749 wants to merge 1 commit into
ml-explore:mainfrom
abnormal749:fix-fuse-gguf-donate-model
Open

Keep model parameters for fuse GGUF export#1321
abnormal749 wants to merge 1 commit into
ml-explore:mainfrom
abnormal749:fix-fuse-gguf-donate-model

Conversation

@abnormal749
Copy link
Copy Markdown

Summary

  • Forward donate_model from utils.save() to save_model() instead of always donating the model.
  • Add regression coverage for both utils.save(..., donate_model=...) behavior and the mlx_lm.fuse --export-gguf path.

Why

utils.save() accepted a donate_model argument but ignored it and always called save_model(..., donate_model=True). That clears the in-memory model parameters even when callers explicitly need them after saving.

mlx_lm.fuse --export-gguf passes donate_model=False because it saves the fused model first, then reads model.parameters() to export GGUF weights. When the flag is ignored, GGUF export receives empty model tensors.

Related to #353, specifically the path where the generated GGUF is missing expected tensors. This does not claim to cover the separate row-major serialization symptom mentioned there.

Validation

  • python -m pre_commit run --files mlx_lm/utils.py tests/test_utils.py tests/test_gguf.py
  • python -m unittest -v tests.test_utils.TestUtils.test_save_respects_donate_model tests.test_gguf
  • Real-world smoke test with mlx-community/Llama-3.2-1B-Instruct-bf16: trained a one-step LoRA adapter, ran mlx_lm fuse --export-gguf, and confirmed the produced GGUF contains token_embd.weight.

@abnormal749 abnormal749 marked this pull request as ready for review May 28, 2026 15:56
@abnormal749 abnormal749 changed the title Respect donate_model in utils.save Keep model parameters for fuse GGUF export May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant