fix: save_pretrained unbound local active_adapters crash(#289)

[umran666]

After merging adapters, the base model retains an internal _hf_peft_config_loaded flag. When you call save_pretrained(), transformers tries to find active adapters that are no longer there, causing an UnboundLocalError crash.
Setting _hf_peft_config_loaded = False on the merged model immediately after the merge prevents the crash and allows the model to save normally.

[gemini-code-assist]

Code Review

This pull request introduces two independent changes: it sets _hf_peft_config_loaded = False on merged models to prevent crashes during save_pretrained, and it adds safety checks to handle cases where self.model is None during model resets (introducing a persistent self.dtype attribute). The reviewer notes that according to the repository style guide, these semantically independent changes must be split into separate pull requests.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

According to the Repository Style Guide (Rules 9 and 10), a pull request must implement only one change, and semantically independent changes must be split into separate PRs.

The changes here to handle self.model is None in reset_model (and the associated introduction of self.dtype) are semantically independent of the fix for the save_pretrained crash (which is resolved by setting _hf_peft_config_loaded = False in get_merged_model). Please split these changes into a separate pull request.

References

1. Pull requests should implement one change, and one change only. PRs containing multiple semantically independent changes must be split into multiple PRs. ^(link)

[p-e-w]

I've never seen such a crash. Under which conditions does this happen?

[umran666]

Check issue #289 you get any idea

[umran666]

I've never seen such a crash. Under which conditions does this happen?

It happens when saving a merged model. merge_and_unload() strips the adapter layers but leaves the internal _hf_peft_config_loaded flag set to True. Because of that, save_pretrained() still expects PEFT layers and crashes with UnboundLocalError. Setting the flag to False prevents the crash.

[p-e-w]

Check issue #289 you get any idea

I already commented in that issue that I don't understand what the issue is. How can I reproduce this?

[umran666]

Check issue #289 you get any idea

I already commented in that issue that I don't understand what the issue is. How can I reproduce this?

To reproduce:
Train any model with LoRA (quantized or not, doesn't matter)
After training, pick "Save the model to a local folder"
Choose "merge" when it asks for the strategy — not "adapter"
That's it. The crash happens on save_pretrained() right after the merge.

What's going on: merge_and_unload() removes all the LoRA layers from the model, but it doesn't clear the _hf_peft_config_loaded flag. So when save_pretrained() runs, transformers still thinks this is a PEFT model and tries to grab active_adapters — which don't exist anymore since we just unloaded them. Boom, UnboundLocalError.

The fix just sets that flag to False right after merging so save_pretrained() takes the normal save path instead of the PEFT one. Same error reported in #289.

[p-e-w]

Train any model with LoRA (quantized or not, doesn't matter)
After training, pick "Save the model to a local folder"
Choose "merge" when it asks for the strategy — not "adapter"
That's it. The crash happens on save_pretrained() right after the merge.

I've done that dozens if not hundreds of times, and I've never seen a crash. I almost always choose "merge". And all models use LoRAs in Heretic, it's the only ablation method we support.

This is not it.

[umran666]

Ah, figured it out. It only happens if the model path you load is already a LoRA folder (with adapter_config.json) instead of a raw base model.

When you pass a LoRA folder to from_pretrained, transformers wraps it and sets _hf_peft_config_loaded = True.

Then merge_and_unload() strips the LoRA layers but leaves the flag True. So when save_pretrained() runs, it tries to find the active adapter, finds nothing, and crashes with UnboundLocalError.

If you always load raw base models, the flag is False so you never see it.

[p-e-w]

That doesn't make sense. Heretic doesn't and cannot support loading LoRAs, how would that work?

We need to do inference on the model we load in order to abliterate it. We can't do that with a LoRA, the bulk of the model is missing.

[kabachuha]

When you do from_pretrained on a lora directory, it pulls the "base model" mentioned in the lora peft configuration (config.json)

[p-e-w]

Really? But we're using AutoModelForCausalLM.from_pretrained from Transformers. That doesn't involve PEFT at all. How does that work? Does Transformers check whether PEFT is available and then execute alternate logic?

[umran666]

That doesn't make sense. Heretic doesn't and cannot support loading LoRAs, how would that work?

It actually works because Hugging Face from_pretrained automatically loads the base model from base_model_name_or_path in the adapter's config.
But loading it that way sets _hf_peft_config_loaded = True on the base model.
When Heretic calls merge_and_unload(), the LoRA layers get stripped but that flag stays True. So save_pretrained() thinks it's still a PEFT model, tries to look up the active adapter, and crashes with UnboundLocalError.

[p-e-w]

Yeah, I still don't get how this works when we load the model through Transformers, even though LoRA functionality is provided by PEFT.

Which LoRA model did you test this with?

[umran666]

Yeah, I still don't get how this works when we load the model through Transformers, even though LoRA functionality is provided by PEFT.

This problem is a general one in the PEFT and Transformers integration and is unrelated to the model that was used. As soon as you load any directory with adapter_config.json through AutoModelForCausalLM.from_pretrained, the problem is solved, and _hf_peft_config_loaded becomes True.

Since Heretic invokes merge_and_unload(), it merges and removes the adapter; however, _hf_peft_config_loaded is still equal to True. That's why when save_pretrained() is invoked, an attempt is made to find the adapters. Since they do not exist anymore, _hf_peft_config_loaded should be set to False.

[p-e-w]

Ok, the explanation makes sense. I will merge this after the 1.4 release because I can't test this comprehensively right now.