Skip to content

9596; fix M5 performance; check for llama#107

Merged
conda-forge-admin merged 2 commits into
conda-forge:mainfrom
xhochy:fix-m5
Jun 11, 2026
Merged

9596; fix M5 performance; check for llama#107
conda-forge-admin merged 2 commits into
conda-forge:mainfrom
xhochy:fix-m5

Conversation

@xhochy

@xhochy xhochy commented Jun 11, 2026

Copy link
Copy Markdown
Member

We need the newer SDK to support TensorOps on the M5 chips. This makes a massive difference in performance for me. I was getting the following warning:

0.00.051.117 E ggml_metal_library_init_from_source: error compiling source
0.00.051.123 W ggml_metal_device_init: - the tensor API is not supported in this environment - disabling

If we build with a newer SDK, this vanishes and performance increases. E.g. for llama-server -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q4_K_M --spec-type draft-mtp --spec-draft-n-max 2 we go from 220t/s (prefill) and 24t/s (decode) to 550t/s (prefill) and 27t/s (decode).

@xhochy

xhochy commented Jun 11, 2026

Copy link
Copy Markdown
Member Author

I took also the liberty and increased the likelihood of getting new builds as it seemed that recently the bot didn't issue any PR.

@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/recipe.yaml:

  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/27368233897. Examine the logs at this URL for more detail.

@traversaro traversaro added the automerge Merge the PR when CI passes label Jun 11, 2026
@conda-forge-admin conda-forge-admin merged commit 9cbacdc into conda-forge:main Jun 11, 2026
15 checks passed
@conda-forge-admin

Copy link
Copy Markdown
Contributor

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

  • linter: passed
  • azure: passed
  • github-actions: passed

Thus the PR was passing and merged! Have a great day!

@xhochy xhochy deleted the fix-m5 branch June 12, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Merge the PR when CI passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants