Question on the Fixed Layer Selection for Text-Only Datasets

[Sunhd123]

Through our experimental analysis, we observed that the most effective layers for detecting text-based datasets (such as XSTest and FigTxt) are not within the fixed range of s=16 to e=29. Since these layers are hard-coded in the implementation, we wonder if this fixed layer selection may cause information loss of early safety signals in text inputs and further affect detection accuracy. We would appreciate the authors’ explanations or discussions on this point.

[snowball521]

Hi, thank you for your interest in our work :)

The layer range s=16 to e=29 is not arbitrarily chosen. It is determined using our safety_aware_layers.py, where we analyze both pure text and bimodal samples.

As discussed in the paper, the presence of visual input tends to delay the activation of refusal-related signals compared to pure text inputs. Therefore, it is expected that the optimal layer range for purely text-based datasets may differ from the range used by us.

Our goal is to find layers that works well across both pure text and multimodal attacks, which may indeed lead to a trade-off, resulting in suboptimal performance for pure text detection.

If your focus is text-only scenarios, you definitely can only use pure text samples in safety_aware_layers.py.