Skip to content

Restore Hugging Face Integration and Implement from_pretrained() Support#339

Open
Muhammad-Ikhwan-Fathulloh wants to merge 1 commit into
m87-labs:mainfrom
Muhammad-Ikhwan-Fathulloh:main
Open

Restore Hugging Face Integration and Implement from_pretrained() Support#339
Muhammad-Ikhwan-Fathulloh wants to merge 1 commit into
m87-labs:mainfrom
Muhammad-Ikhwan-Fathulloh:main

Conversation

@Muhammad-Ikhwan-Fathulloh

Copy link
Copy Markdown

Related Issue

Closes #338

Summary

This PR restores missing Hugging Face integration components and fixes several regressions that prevented Moondream from working with standard Hugging Face workflows.

Changes Included

Restored Hugging Face package structure

Added missing files:

  • moondream/hf/util.py
  • moondream/hf/moondream.py
  • moondream/hf/__init__.py

Restored:

  • moondream/__init__.py

These files provide the Hugging Face compatibility layer and restore import paths used by existing examples and downstream projects.


Fixed HfMoondream.generate()

Updated the implementation to correctly forward arguments to _generate_answer(), including:

  • pos
  • generation settings

This resolves generation failures caused by missing parameters.


Implemented from_pretrained()

Added a Hugging Face-compatible loading interface:

from moondream.hf import Moondream

model = Moondream.from_pretrained("vikhyatk/moondream2")

The implementation:

  • Downloads model files using huggingface_hub.snapshot_download
  • Automatically locates .safetensors and .pt checkpoints
  • Loads weights through load_weights_into_model
  • Supports standard Hugging Face workflows

Restored backward compatibility

Updated APIs to support previous usage patterns:

caption()
model.caption([image], tokenizer=tokenizer)
encode_image()
model.encode_image([image])

This helps prevent breakages in existing integrations and user code.


Dependencies

Added:

huggingface_hub>=0.20.0

to ensure Hub downloads are available for from_pretrained().

Validation

Verified the following workflows:

Direct loading

from moondream.hf import Moondream

model = Moondream.from_pretrained("vikhyatk/moondream2")

Transformers AutoModel loading

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "vikhyatk/moondream2",
    trust_remote_code=True
)

Legacy API compatibility

model.caption([image], tokenizer=tokenizer)

model.encode_image([image])

Impact

This PR restores:

  • Hugging Face integration modules
  • from_pretrained() support
  • AutoModel compatibility
  • Backward-compatible image APIs

and resolves the regressions reported in #338.

…d, and improve backward compatibility

This commit:
1. Restores missing moondream.hf module structure
2. Adds from_pretrained() class method to HfMoondream that uses huggingface_hub to download models
3. Fixes generate() method to properly call _generate_answer with correct parameters
4. Adds backward-compatible caption() and encode_image() methods that accept list of images
5. Adds missing huggingface_hub dependency
6. Registers HfConfig and HfMoondream with HF Auto classes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Missing Hugging Face Integration Files and Broken from_pretrained() Support

1 participant