equation 5 in the open-source code

[chichengfengxue]

Thank you for your excellent work! It seems that there is no implementation of equation 5 in the open-source code? Thank you for your answer!

[Linwei-Chen]

Thank you for your insightful question and for taking the time to review our work and the associated code! This is an excellent point and it touches on an important detail about the practical implementation versus the theoretical concept.

You are correct that you might not find a line-for-line implementation of Equation (5) as it is written. Equation (5) describes the conceptual operation in the frequency domain:

Â_p,q = F⁻¹(I_f - F(A_p,q))

This formula states that the high-pass filter Â is obtained by taking the Fourier transform of the attention map A, subtracting it from an all-pass filter I_f (an array of ones), and then applying the inverse Fourier transform.

However, due to the properties of the Fourier Transform (specifically, its linearity), this operation has a much simpler and more computationally efficient equivalent in the spatial domain.

Let's break it down:

1. F⁻¹(I_f - F(A)) can be rewritten as F⁻¹(I_f) - F⁻¹(F(A)).
2. The term F⁻¹(F(A)) is simply A, the original attention map.
3. The term F⁻¹(I_f) is the inverse Fourier transform of an all-pass filter. An all-pass filter in the frequency domain corresponds to a Dirac delta function in the spatial domain. In a discrete grid, this is an identity matrix (I), which has a 1 at its central element and 0s everywhere else.

Therefore, the entire equation simplifies to its spatial domain equivalent:

 = I - A

This is how it is implemented in the code. We compute the high-pass filter by simply subtracting the original attention map from an identity matrix. This method is mathematically identical to the one described in Equation (5) but avoids two costly FFT operations (one forward, one inverse), making the implementation much more efficient.

You should be able to find this logic in the implementation of the AttInv module.

I hope this clarifies the connection between the theoretical formula in the paper and its practical implementation in the code. Thank you again for your sharp observation and engagement with our research