Skip to content

Add LiteRT-LM LoRA adapter support #173

@leehack

Description

@leehack

Summary

The new LiteRT-LM backend currently rejects LoRA use:

  • ModelParams.loras is rejected during .litertlm model load.
  • setLora(...), removeLora(...), and clearLoras() report UnsupportedError for LiteRT-LM contexts.

This is intentionally out of scope for PR #167, but we should track it as follow-up work instead of leaving it as a permanent limitation.

Current evidence

The pinned litert-lm-native@v0.12.0 C ABI exposes engine/conversation/tokenizer/constrained-decoding helpers, but no stable exported litert_lm_*lora* function for Dart FFI to bind. The macOS artifact does contain internal C++ symbols such as SessionConfig::SetScopedLoraFile(...), so upstream/native support may exist below the C wrapper layer.

We should not bind Dart directly to mangled C++ symbols. This needs a stable native C ABI first.

Proposed work

  1. Investigate upstream LiteRT-LM LoRA semantics: load-time only vs session-scoped vs dynamic adapter switching.
  2. Add stable C wrapper functions in litert-lm-native if upstream exposes a usable API.
  3. Publish updated runtime artifacts for supported platforms.
  4. Add Dart FFI bindings in llamadart.
  5. Map the API to existing high-level LoRA operations where possible:
    • ModelParams.loras
    • setLora(...)
    • removeLora(...)
    • clearLoras()
  6. Add native smoke tests with a real .litertlm model and adapter when a test artifact is available.
  7. Update capability docs and backend-selection guidance.

Acceptance criteria

  • LiteRT-LM either supports the existing LoRA API with documented behavior, or docs explain the exact unsupported operations and native/runtime reason.
  • Unsupported combinations fail early with actionable errors.
  • Runtime artifact version and platform support are documented.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestlitert-lm-parityLiteRT-LM backend parity features

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions