Summary
The new LiteRT-LM backend currently rejects LoRA use:
ModelParams.loras is rejected during .litertlm model load.
setLora(...), removeLora(...), and clearLoras() report UnsupportedError for LiteRT-LM contexts.
This is intentionally out of scope for PR #167, but we should track it as follow-up work instead of leaving it as a permanent limitation.
Current evidence
The pinned litert-lm-native@v0.12.0 C ABI exposes engine/conversation/tokenizer/constrained-decoding helpers, but no stable exported litert_lm_*lora* function for Dart FFI to bind. The macOS artifact does contain internal C++ symbols such as SessionConfig::SetScopedLoraFile(...), so upstream/native support may exist below the C wrapper layer.
We should not bind Dart directly to mangled C++ symbols. This needs a stable native C ABI first.
Proposed work
- Investigate upstream LiteRT-LM LoRA semantics: load-time only vs session-scoped vs dynamic adapter switching.
- Add stable C wrapper functions in
litert-lm-native if upstream exposes a usable API.
- Publish updated runtime artifacts for supported platforms.
- Add Dart FFI bindings in
llamadart.
- Map the API to existing high-level LoRA operations where possible:
ModelParams.loras
setLora(...)
removeLora(...)
clearLoras()
- Add native smoke tests with a real
.litertlm model and adapter when a test artifact is available.
- Update capability docs and backend-selection guidance.
Acceptance criteria
- LiteRT-LM either supports the existing LoRA API with documented behavior, or docs explain the exact unsupported operations and native/runtime reason.
- Unsupported combinations fail early with actionable errors.
- Runtime artifact version and platform support are documented.
Summary
The new LiteRT-LM backend currently rejects LoRA use:
ModelParams.lorasis rejected during.litertlmmodel load.setLora(...),removeLora(...), andclearLoras()reportUnsupportedErrorfor LiteRT-LM contexts.This is intentionally out of scope for PR #167, but we should track it as follow-up work instead of leaving it as a permanent limitation.
Current evidence
The pinned
litert-lm-native@v0.12.0C ABI exposes engine/conversation/tokenizer/constrained-decoding helpers, but no stable exportedlitert_lm_*lora*function for Dart FFI to bind. The macOS artifact does contain internal C++ symbols such asSessionConfig::SetScopedLoraFile(...), so upstream/native support may exist below the C wrapper layer.We should not bind Dart directly to mangled C++ symbols. This needs a stable native C ABI first.
Proposed work
litert-lm-nativeif upstream exposes a usable API.llamadart.ModelParams.lorassetLora(...)removeLora(...)clearLoras().litertlmmodel and adapter when a test artifact is available.Acceptance criteria