CLI basic running error

[IceIce1ce]

Hi author, thannks for your great work. When I run the CLI basic script on an image folder with the Gemma 4 model, the error happened. Can you check your code again?

Traceback (most recent call last):
File "/home/vsw/Desktop/VisionCaptioner/backend.py", line 616, in generate_batch
generated_ids = self.model.generate(
^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/generation/utils.py", line 2543, in generate
result = decoding_method(
^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/generation/utils.py", line 2736, in _sample
outputs = self._prefill(
^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/generation/utils.py", line 3768, in _prefill
return self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/accelerate/hooks.py", line 192, in new_forward
output = module._old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/utils/generic.py", line 876, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/models/gemma4/modeling_gemma4.py", line 2455, in forward
outputs = self.model(
^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/utils/generic.py", line 952, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/utils/generic.py", line 876, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/models/gemma4/modeling_gemma4.py", line 2233, in forward
torch_compilable_check(
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1540, in torch_compilable_check
torch._check_with(error_type, cond, msg_callable)
File "/home/vsw/miniconda3/envs/cap/lib/python3.11/site-packages/torch/init.py", line 1642, in _check_with
raise error_type(message_evaluated)
ValueError: Image features and image tokens do not match, tokens: 264, features: 1

[Brekel]

Thanks for reporting, I think there was an issue with batch sizes higher than 1 and Gemma4 (non-GGUF) models, not 100% sure that fixes your exact problem but I think it does.
Please update to the latest version and try again.

[IceIce1ce]

Hi author, thanks for your information. Also, when I use the GUI version, I found that the processing button will has a problem when my image folder has more than one image. Does the current code support multiple images for captioning?

[Brekel]

Did you select a model and hit "Load Model"? The "Start Processing" button is disabled when no model is loaded

[IceIce1ce]

Yes, I select a downloaded model and hit the "load model" button before using the "start processing". It shows the error that there are 6 text prompts but only one image (my image folder contain 6 images).

[Brekel]

What type of model are you using?
Gemma4 I assume, GGUF or non-GGUF?
Can you copy paste the error please?