Could not test on 2048x1024 images

[tltsilveira]

Thank you for sharing the source code and pre-trained weights. However, I could not apply this model to 2048x1024 ERP images (as argued as possible on the main page of this repository). In my tests, I skip the resize operation in "check_img" function. The resulting image is, however, of size 1024x512 and does not keep the original image information (but a distorted part of it only). What am I doing wrong? What else should I do to apply this model to ERP images of different sizes? Thank you in advance. Best, Thiago L T da Silveira.

[limuhit]

line 272, the "forward(self,h=32,w=64)" function require the shape of code. For input of 1024x512, it is h=32,w=64. However, if the input changes to 2048x1024, the parameter should change to h=64 and w=128.

[tltsilveira]

Thank you!

[tltsilveira]

I have a 24Gb NVIDIA GPU. Unfortunately, I could not work with 4K (3840x1920) ERP images due to VRAM space restrictions. Am I doing something wrong? Best.

[limuhit]

Since the model requires loading the entire image as input, higher resolutions inevitably demand more GPU memory. Here are some potential solutions to mitigate this issue:

1. Use Inference Mode: Add with torch.no_grad(): before calling the encoder and decoder. This can help reduce GPU memory consumption by disabling gradient computations.
2. Upgrade GPU Hardware: Utilize a GPU with larger memory capacity, such as NVIDIA A100 or H100.
3. Divide and Compress: Split the image into four sections, each with a resolution of 1920×960. Compress each piece individually and then reassemble them. However, this approach may degrade compression performance. For example, if x is the image tensor to be compressed with a shape of (1, 3, h, w), you can create four segments as follows:

   x1, x2, x3, x4 = x[:, :, ::2, ::2], x[:, :, ::2, 1::2], x[:, :, 1::2, ::2], x[:, :, 1::2, 1::2]

[tltsilveira]

Thank you for the answer.
The first option seems to alleviate memory consumption, but I get this error instead:

void sphere_lat_scale_opt::reshape(int, int, int, int): Assertion `height_ % npart_ == 0 && "height must be multiplier of the npart"' failed.

Any clues on how to compress 3840x1920 ERP images using the pre-trained LIC360?

Thank you again.

Best

[limuhit]

try to change the parameter in line 134 in test/model_zoo.py.
now it is 512/16, changing to 16 would help

[tltsilveira]

Hi,

when I follow the points above, I get a strange result. See:

I have changed the following lines in the project:

• line 272 in lic360_demo.py: "forward(self,h=32,w=64)" -> "forward(self,h=120,w=240)"
• line 134 in test/model_zoo.py: "512//16" -> "16"

Plus, I am using torch.no_grad(): for alleviating memory consumption during inference.

I am trying with a 1920x3840 ERP image.

Can you help with that?

Best.

[limuhit]

try to change the parameter in line 134 in test/model_zoo.py. now it is 512/16, changing to 16 would help
I think the pretrained model fit for the parameter 32, change it to 16 would harm the results. Thus try to reshape the testing image to a multiple of 512 to check the results. If it works, my guess should be true.