feat: optional MLX embedder backend (torch-free, Apple-silicon native)#18
Open
ry-ops wants to merge 1 commit into
Open
feat: optional MLX embedder backend (torch-free, Apple-silicon native)#18ry-ops wants to merge 1 commit into
ry-ops wants to merge 1 commit into
Conversation
…tive) Adds an alternative embedding backend that runs on MLX via mlx-embeddings, selectable with AIANA_EMBEDDER_BACKEND=mlx, so aiana can embed on Apple silicon without PyTorch. Defaults to all-MiniLM-L6-v2 (384-dim) -- the same model family and vector space as the sentence-transformers default, so existing aiana_memories vectors remain searchable. - embeddings/mlx_embedder.py: MLXEmbedder (embed/dimension/similarity), lazy optional import; raises a clear error if mlx-embeddings isn't installed. - embeddings/embedder.py: get_embedder() selects backend via AIANA_EMBEDDER_BACKEND (default 'sentence-transformers', or 'mlx'). - pyproject: new optional extra 'mlx' (= mlx-embeddings); added to 'all'. Sentence-transformers remains the default; no behavior change unless opted in. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional MLX embedding backend so aiana can embed on Apple silicon without PyTorch, selectable via
AIANA_EMBEDDER_BACKEND=mlx. Sentence-transformers stays the default — no behavior change unless opted in.Motivation: running the sentence-transformers/torch embedder alongside a local LLM is heavy (it was the dependency that OOM'd a 16 GB M1 in practice). An MLX embedder shares the same unified-memory runtime as MLX-served models, so the embedding layer becomes lighter and fully Apple-silicon-native.
Changes
embeddings/mlx_embedder.py— newMLXEmbeddermirroring theEmbedderinterface (embed,dimension,similarity), running onmlx-embeddings. Optional import with a clear install hint; defaults toall-MiniLM-L6-v2(384-dim) — the same model family and vector space as the sentence-transformers default, so existingaiana_memoriesvectors stay searchable (no migration).embeddings/embedder.py—get_embedder()selects the backend fromAIANA_EMBEDDER_BACKEND(sentence-transformersdefault, ormlx).pyproject.toml— new optional extramlx(mlx-embeddings), added toall.Usage
Verification
On Python 3.13 / Apple silicon:
Notes
all-MiniLM-L6-v2avoids that for the common case.mainand stacks cleanly with it.🤖 Generated with Claude Code