Feature Request: Support for smaller text encoders (Gemma 3 4B / quantized Gemma) for <32GB VRAM GPUs

[Jackson3195]

Summary

LTX-2 currently requires Gemma 3 12B as the text encoder, which needs ~24-27GB VRAM to operate. This makes LTX-2 unusable on consumer GPUs with 16GB VRAM (RTX 5080, RTX 4080, etc.), even with FP8 models and all optimizations applied.

Hardware Tested

• GPU: RTX 5080 (16GB VRAM)
• RAM: 32GB
• Model: ltx-2-19b-dev-fp8.safetensors
• Text Encoder: gemma-3-12b-it-qat-q4_0-unquantized

What I Tried

1. ✅ Using FP8 quantized LTX-2 model
2. ✅ Bypassing the LTXVGemmaEnhancePrompt node
3. ✅ Reducing resolution to 512×512 with 41 frames
4. ✅ Using the QAT version of Gemma 3 12B

Errors Encountered

Error 1: Device Mismatch (Partial Offloading)

loaded partially; 13433.80 MB usable, 13096.30 MB loaded, 450.12 MB offloaded
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Error 2: Out of Memory

Allocated memory: 27598 MiB | Peak Usage: 29068 MiB
torch.OutOfMemoryError: Allocation on device

Even basic CLIP text encoding (not enhancement) requires the full Gemma 3 12B model, which exceeds 16GB VRAM.

Feature Request

Please consider adding support for smaller text encoder alternatives:

Option 1: Gemma 3 4B Support

• Google's Gemma 3 4B requires only ~8GB (BF16) or ~2.6GB (int4)
• Would require fine-tuning/distillation but would open LTX-2 to millions of consumer GPU users

Option 2: Proper Int4/GGUF Quantization for Gemma 3 12B

• Google's QAT models reduce Gemma 3 12B from 24GB to ~6.6GB (int4)
• The current loader may not be utilizing quantization efficiently
• Reference: Gemma 3 QAT Models

Option 3: Alternative Text Encoder

• T5-XXL (used in LTX-Video 1.0) works on lower VRAM
• Other efficient encoders like T5Gemma-2-4B-4B

Impact

The current 32GB+ VRAM requirement limits LTX-2 to:

• Datacenter GPUs (A100, H100)
• RTX 5090 / RTX PRO 6000
• ~X% of potential users

Supporting 16GB GPUs would enable:

• RTX 4080, 4080 Super, 5080
• RTX 3090, 4090 Laptop
• ~50%+ of AI enthusiasts

References

• Related issue: [bug] Missing node types #3 (GGUF or int4 request on LTX-2 repo)

• Related issue: Feature request: Memory Management in nodes. #300 (Memory Management request)

  Thank you for the amazing work on LTX-2!

[boyan-orion]

Jackson Jacob (@Jackson3195) I'm currently using it with the --novram argument in Comfy. The text encoder is slow when processing on CPU but the inference part works incredibly fast and it only costs my gpu 3 GB VRAM to make a 720p video. I'm basically streaming all weights from RAM.

On the other hand, using the official Comfy built in workflow will load the text encoder on CPU but process it on GPU, so it's going to be much faster for the text prompting part but it's missing prompt enhancing features which are crucial for the LTX2 model.

I'm running it on RTX 5080 16GB + 64 GB RAM and tried FP16 and FP4 LTX2 weights. All we need is FP8 or FP4 version of the Gemma model indeed.

[Jackson3195]

Update: Workarounds Found

Following up on my original issue - I wanted follow up with some workarounds thanks to Reddit that may help others with 16GB GPUs or less. Sources have been mentioned

Workaround 1: FP8 Gemma Text Encoder

An FP8 quantized Gemma model is available:
https://huggingface.co/GitMylo/LTX-2-comfy_gemma_fp8_e4m3fn/blob/main/gemma_3_12B_it_fp8_e4m3fn.safetensors

Place in ComfyUI/models/text_encoders/ and select in the LTXVGemmaCLIPModelLoader node.

You'll need to restart Comfy.

Source: https://www.reddit.com/r/StableDiffusion/comments/1q5k6al/fix_to_make_ltxv2_work_with_24gb_or_less_of_vram/

Workaround 2: Launch Flags

Configuration	Flag
Low VRAM	--novram

The --novram flag streams weights from system RAM, making LTX-2 usable on 16GB GPUs at the cost of slower text encoding as mentioned by Boyan Yanevsky (@boyan-orion) (Thanks!!!)

Results

✅ LTX-2 now runs on 16GB GPU (RTX 5080)

Closing this thread

Unreleated

Not seeing the same quality that other users seem to be having but that's a different problem unique to me (maybe)

[WeaponZero420]

So I tried this on my 24 GB VRAM 4090. No shot at running Gemma. But I tried the --novram option, seems like it actually passes the enhanced prompt node. but then I get an "mat1 and mat2 shapes cannot be multiplied (264061y1 and 128x3)" error. guess this is becaus of some sage attention problem for not using the gpu?
edit: yes, don't be like me: if you make a start.bat for comfy with --novram, be sure to delete --use-sage-attention --fast fp16_accumulation first...
edit2: reading through the closed comments, the mat1 problem seems to be around previews during sampling. by deleting the sage-attention options, I also deleted my --preview-method latent2rgb option that I used as a workaround for my previews not working properly. By deleting this option it is back to "preview: none" and that solved the problem.

Best option would be imho to just make the node offload gemma on RAM and we are good to go?

[morganavr]

.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --disable-auto-launch --reserve-vram 10

these settings work for RTX 5080

[boyan-orion]

WeaponZero420 I removed the Gemma model loader and replaced it with LTXV Audio Text Encoder Loader, then loaded the single safetensors file instead of the ones you need from repos in 5 parts. I also disabled the prompt enhancing node because it can't be used anymore in this case, however i was able to load everything in RAM and stream everything from RAM with the --novram option, and the benefit is that both text encoder and video model will be processed by GPU, so much faster.

[darkhorizon2020]

"LTXV Audio Text Encoder Loader" for Gemma works 8x times faster then normal loader

[mountaintops]

city96/ComfyUI-GGUF#399
city96/ComfyUI-GGUF#398

[BossLevel]

Does anyone have a solution for this that applies to those not using ComfyUI (I'm running the LTX Pipelines from CLI)?

UPDATE: apologies, just realised this is a ComfyUI repo (came here via google) - leaving for reference, will request feature in appropriate location.