Skip to content

Conversation

@derekhiggins
Copy link
Contributor

Implement server-level trusted_model_prefixes configuration that controls which model organizations can execute custom code via trust_remote_code.

Changes:
o Add trusted_model_prefixes to ServerConfig with default trusted orgs o Create TrustedModelConfig mixin with is_trusted_model() method o Update inference providers to use configurable allowlist o Update post_training provider to dynamically set trust_remote_code o Models not in allowlist load with trust_remote_code=False o Add unit tests

Default trusted prefix: nomic-ai/ (default model, requires this)

BREAKING CHANGE:
The default trusted_model_prefixes list has been reduced to only ["nomic-ai/"] for security. Previously, trust_remote_code=True was implicitly allowed for all models.

Impact:
o Post-training: Models not in the trusted list will load with
trust_remote_code=False (previously all loaded with True)
o Inference: Embedding models not in the trusted list will load with
trust_remote_code=False (previously behavior was provider-dependent)

Migration:
Users needing the old behavior can add to their run.yaml:
server:
trusted_model_prefixes:
- "nomic-ai/"
- "your-org/" - "trusted-org/"

This provides centralized security policy while maintaining flexibility for legitimate custom models from trusted sources.

Closes: #4601

--
Note: Code created and iterated over with the assistance of sonnet-4.5-thinking, I've manually and tested the functionality in the inference provider but don't have the setup to verify the code in the post_training providers.

…revent trust_remote_code RCE

Implement server-level trusted_model_prefixes configuration that controls
which model organizations can execute custom code via trust_remote_code.

Changes:
o Add trusted_model_prefixes to ServerConfig with default trusted orgs
o Create TrustedModelConfig mixin with is_trusted_model() method
o Update inference providers to use configurable allowlist
o Update post_training provider to dynamically set trust_remote_code
o Models not in allowlist load with trust_remote_code=False
o Add unit tests

Default trusted prefix: nomic-ai/ (default model, requires this)

BREAKING CHANGE:
The default trusted_model_prefixes list has been reduced to only
["nomic-ai/"] for security. Previously, trust_remote_code=True was
implicitly allowed for all models.

Impact:
o Post-training: Models not in the trusted list will load with
  trust_remote_code=False (previously all loaded with True)
o Inference: Embedding models not in the trusted list will load with
  trust_remote_code=False (previously behavior was provider-dependent)

Migration:
Users needing the old behavior can add to their run.yaml:
  server:
    trusted_model_prefixes:
      - "nomic-ai/"
      - "your-org/"
      - "trusted-org/"

This provides centralized security policy while maintaining flexibility
for legitimate custom models from trusted sources.

Closes: llamastack#4601

Signed-off-by: Derek Higgins <derekh@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unconditional trust_remote_code=True enables arbitrary code execution from untrusted models

1 participant