fix(nodes): restore non-quantized backups locally and skip super's mmap walk#445
Open
Booyaka101 wants to merge 1 commit into
Open
fix(nodes): restore non-quantized backups locally and skip super's mmap walk#445Booyaka101 wants to merge 1 commit into
Booyaka101 wants to merge 1 commit into
Conversation
…ap walk The override forwards unpatch_weights=True to comfy core's unpatch_model, which then calls self.model.to(device_to). Walking that .to() over the GGUF-quantized tensors that are still mmap-backed crashes with a Windows access violation (city96#444), surfacing whenever ComfyUI actually invokes unpatch_model — easiest repro is wrapping UnetLoaderGGUF in a node that returns multiple outputs, which flushes cache and triggers the unpatch path. The maintainer mitigated this in Sep 2024 (commit 6dbb4ba) by passing device_to=None to super, then reverted in 717a0e1 because that broke VRAM estimation. Reporter's proposed fix (passing unpatch_weights=False to super) avoids the crash but skips comfy core's non-quantized backup restoration — patch_weight_to_device's `if key not in self.backup` guard then never re-saves the original weight, and the next run's temp_weight = weight.to(...) reads the still-patched value, so LoRA deltas compound across runs. Restore the non-quantized backups in the override mirroring comfy core's loop (copy_to_param / set_attr_param + backup.clear() + comfy_patched_weights cleanup), then forward unpatch_weights=False so super skips its weight-restore + model.to(device_to) block. Keeps the crash fix without regressing LoRA correctness or VRAM accounting. Verified with a stub-based reproducer (verify_unpatch.py) covering three scenarios: pre-fix crashes on simulated mmap .to() — matches city96#444 reporter fix avoids crash, but non-quantized weight stays patched this fix avoids crash AND restores non-quantized backup correctly Closes city96#444 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Confirmed working on real CUDA. RTX 3060 (12GB), Windows 10, ComfyUI 0.22.2, PyTorch 2.8.0+cu129. Running Flux 2 Klein 9b, with both TE and diffusion model as GGUFs. Was facing an issue where any attempt to unload GGUF model weights (either manually or automatically, in my case triggered by normal between-run model caching) led to the same "Windows fatal exception: access violation" fault described above. After applying this patch, both manual and automatic unloads work and repeated runs are stable. LoRAs produce identical output across multiple runs with the same seed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #444.
Symptom
UnetLoaderGGUFwired into a multi-output node (e.g. `(MODEL, STRING)`) causes ComfyUI to flush the cached patcher and call `unpatch_model`. The forwarded `unpatch_weights=True` reaches comfy core's `ModelPatcher.unpatch_model`, which ultimately runs `self.model.to(device_to)`. Walking `.to()` over the still-mmap-backed GGUF quantized tensors faults with a Windows `EXCEPTION_ACCESS_VIOLATION`.Background
This is a known regression. The September 2024 mitigation in 6dbb4ba gated the device_to forward on `DISABLE_SMART_MEMORY`, sidestepping the `.to()` walk in normal use. It was reverted in 717a0e1 ("Seems to break VRAM estimation and needs more involved fix with upstream ComfyUI repo").
@stonerabit's proposed fix (forward `unpatch_weights=False` to super) avoids the crash, but it also skips comfy core's non-quantized backup-restore loop. Combined with `patch_weight_to_device:51` — `if key not in self.backup` only re-saves once — the next `patch_model` reads the still-patched live weight as `temp_weight` and `calculate_weight` re-applies the delta on top, so LoRA deltas compound across runs.
Fix
Restore non-quantized backups inside the GGUF override (mirroring comfy core's loop verbatim — `copy_to_param` / `set_attr_param`, `backup.clear()`, `current_weight_patches_uuid = None`, drop `comfy_patched_weights` attr), then forward `unpatch_weights=False` to super so it skips both the weight-restore block and the `self.model.to(device_to)` walk.
Net effect:
Verification
Cannot run real CUDA in the dev environment, so I built a stub-based reproducer (`verify_unpatch.py`) that synthesizes the relevant interfaces: an `mmap_backed` flag on a stub tensor that raises `OSError("EXCEPTION_ACCESS_VIOLATION")` from `.to(device)` (mirroring the Windows symptom), a minimal base `unpatch_model` mirroring the comfy core sequence, and three override variants applied to the same fixture:
```
Scenario 1: pre-fix override (forwards unpatch_weights=True)
PASS crashes with simulated Windows access violation
PASS matches reporter's symptom (#444)
Scenario 2: reporter's fix (super(unpatch_weights=False))
PASS avoids the access-violation crash
PASS LoRA-correctness regression: non-quantized backup NOT restored
PASS quantized patches still cleared
Scenario 3: proposed fix (manual backup restore + super(False))
PASS avoids the access-violation crash
PASS non-quantized backup restored to original
PASS quantized patches still cleared
PASS backup dict cleared post-restore
```
Scenario 1 reproduces the exact failure mode of the live unpatch path. Scenario 2 demonstrates the LoRA-correctness gap in the simpler patch. Scenario 3 covers what this PR ships.
Scope notes
Credit @stonerabit for the clean repro + first-pass diagnosis pinning the mmap-touching base patcher as the trigger.