Skip to content

[Windows] Search for the closest free page for allocate jit memory in x64#122

Closed
lucianopa-msft wants to merge 1 commit intomicrosoft:mainfrom
lucianopa-msft:lucianopa/jit-memory-allocation
Closed

[Windows] Search for the closest free page for allocate jit memory in x64#122
lucianopa-msft wants to merge 1 commit intomicrosoft:mainfrom
lucianopa-msft:lucianopa/jit-memory-allocation

Conversation

@lucianopa-msft
Copy link
Copy Markdown
Member

@lucianopa-msft lucianopa-msft commented Mar 17, 2026

Previous algorithm would take the base address compute 2GB up and start searching for a free page there (starting from furtherst from base address). Depending on the base address this could cause VirtualAlloc to try access an invalid memory range (stack memory) causing a crash. This changes the implementation to look instead for the closest free page.
This also makes it consistent with other implementations such as allocate_jit_memory_unix

@mazong1123
Copy link
Copy Markdown
Collaborator

Hi @lucianopa-msft , thanks for providing the fix! Would you mind adding some tests to cover the change. Ideally, those tests should fail on main branch but pass with your changes. Thanks!

@lucianopa-msft
Copy link
Copy Markdown
Member Author

I really tried to but couldn't get a repro testing repro that mimics the exact crash. So there may be special conditions for the repro I am still not sure.

mazong1123 added a commit that referenced this pull request Mar 25, 2026
The old x86_64 Windows implementation of allocate_jit_memory scanned linearly
from func_addr - 2GB upward, which often allocated memory ~2GB away from the
function. This could land in/near the stack region, disrupting the stack guard
page and causing STATUS_STACK_OVERFLOW (0xc00000fd) during parallel test
execution.

This change replaces the linear scan with a bidirectional closest-first search
(matching the existing aarch64 Windows and Unix implementations). The allocator
now searches outward from the function address at +offset and -offset, finding
the closest available page first.

This is an improved version of PR #122 that fixes two additional issues:
- Searches both directions (not just downward) for robustness
- Avoids an infinite loop when checked_sub fails by keeping offset increment
  outside the inner direction loop

Also adds tests:
- Unit test verifying JIT allocation is close to source (<128MB, not ~2GB)
- Unit test verifying JIT allocation is not near the stack region
- Integration test for stack growth after patching
- Integration test for concurrent patching with deep stack usage (8 threads)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mazong1123
Copy link
Copy Markdown
Collaborator

Thanks @lucianopa-msft ! I have created a separate PR and added tests: #124 based on this PR.

mazong1123 added a commit that referenced this pull request Mar 25, 2026
…124)

The old x86_64 Windows implementation of allocate_jit_memory scanned linearly
from func_addr - 2GB upward, which often allocated memory ~2GB away from the
function. This could land in/near the stack region, disrupting the stack guard
page and causing STATUS_STACK_OVERFLOW (0xc00000fd) during parallel test
execution.

This change replaces the linear scan with a bidirectional closest-first search
(matching the existing aarch64 Windows and Unix implementations). The allocator
now searches outward from the function address at +offset and -offset, finding
the closest available page first.

This is an improved version of PR #122 that fixes two additional issues:
- Searches both directions (not just downward) for robustness
- Avoids an infinite loop when checked_sub fails by keeping offset increment
  outside the inner direction loop

Also adds tests:
- Unit test verifying JIT allocation is close to source (<128MB, not ~2GB)
- Unit test verifying JIT allocation is not near the stack region
- Integration test for stack growth after patching
- Integration test for concurrent patching with deep stack usage (8 threads)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lucianopa-msft lucianopa-msft deleted the lucianopa/jit-memory-allocation branch March 25, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants