Skip to content

Add optional memory locking support#27

Merged
julman99 merged 8 commits into
mainfrom
issue-12-lock-memory
May 14, 2026
Merged

Add optional memory locking support#27
julman99 merged 8 commits into
mainfrom
issue-12-lock-memory

Conversation

@julman99
Copy link
Copy Markdown
Owner

@julman99 julman99 commented May 14, 2026

Summary

  • add backend capability initialization for memory stats and memory locking
  • add --lock-memory / -l, shown only when the active backend supports page locking
  • implement POSIX mlock / munlock support and Windows VirtualLock / VirtualUnlock support
  • keep memory summary output shared between normal execution and help output
  • extend malloc-hook and CLI tests for chunk sizing and lock-memory behavior

Tests

  • git diff --check
  • bash test/ci/all.test.sh

Closes #12

Summary by CodeRabbit

  • New Features

    • Added a --lock-memory option to lock allocated memory.
    • Terminal progress display during allocation/verification.
    • Help now shows current memory summary when available.
  • Enhancements

    • Improved percentage-based size handling (clarified to mean percent of available memory).
  • Bug Fixes

    • New exit codes for memory-lock scenarios: unsupported (22) and lock failure (23).
  • Documentation

    • Updated usage/options to document --lock-memory and percentage behavior.

Review Change Stack

Add backend capability initialization for memory stats and memory locking, with POSIX shared lock helpers and Windows VirtualLock/VirtualUnlock support.

Expose --lock-memory/-l only when supported, keep help output aligned with backend capabilities, and reuse the memory summary printer for help and normal execution.

Extend malloc-hook and CLI tests to cover chunk sizing and lock-memory behavior.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Warning

Rate limit exceeded

@julman99 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 12 minutes and 22 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b475e6bc-9eb0-4d34-9f87-cd805b8a904f

📥 Commits

Reviewing files that changed from the base of the PR and between 5517991 and 60f704d.

📒 Files selected for processing (4)
  • src/eatmemory.linux.c
  • src/eatmemory.posix-common.c
  • test/ci/posix/eatmemory_posix_lock_range.test.c
  • test/ci/posix/posix-lock-range.test.sh
📝 Walkthrough

Walkthrough

This PR adds memory locking capability to eatmemory with backend-aware feature detection. The refactored API introduces a eatmemory_backend struct for platform capability flags, updates all platform backends to initialize and export stats via renamed functions, implements POSIX region locking via mlock/munlock, integrates locking into the core allocation flow with error recovery, and conditionally exposes the lock option in the CLI based on platform support.

Changes

Memory locking feature

Layer / File(s) Summary
API contracts and type definitions
src/eatmemory.h
New eatmemory_backend and progress types; system_memory_stats drops supported; allocation adds total, chunk_size, locked_count; new eatmemory_lock_result and eatmemory_stats_result enums; get_chunk_size, updated eat() and digest() signatures.
Platform backends: initialization and stats
src/eatmemory.linux.c, src/eatmemory.windows.c, src/eatmemory.apple.c, src/eatmemory.aix.c, src/eatmemory.solaris.c, src/eatmemory.unknown.c
Each platform adds eatmemory_get_backend_capabilities() and replaces get_system_memory_stats() with eatmemory_get_system_memory_stats() returning an enum; POSIX platforms include shared locking implementation or provide stubs where unsupported.
Core allocation refactor with chunk sizing & locking
src/eatmemory.c
Adds get_chunk_size() helper; eat() accepts lock_memory and progress, locks chunks conditionally, tracks locked_count, performs cleanup on failure via updated digest(), and uses get_chunk_size() in read/write/verify loops.
CLI option handling and output
src/main.c
configure_cmd() & print_help() accept backend to conditionally expose -l/--lock-memory; print_memory_summary() shows stats when available; main() initializes backend, reads lock flag only when supported, invokes eat_with_options() with progress, and maps new lock errors to exit codes.
Test harness and test cases
test/ci/malloc/eatmemory_malloc_hook.test.c, test/ci/simple/allocation.test.sh
Test hooks track lock/unlock calls and support forced outcomes, rename stats hook, add get_chunk_size unit test, update many eat(...) calls to new signature, add lock-specific tests (no-lock default, successful lock/unlock, lock-failure cleanup, unsupported-lock handling), and CI verifies help shows lock option.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • julman99/eatmemory#21: Solaris backend work refactored and aligned with the new stats/locking API.
  • julman99/eatmemory#17: Earlier PR that introduced core allocation and stats APIs which this change renames and extends.
  • julman99/eatmemory#24: Related changes to percent/help behavior and stats gating now integrated with backend capability API.

Poem

🐰 I hopped through code to bind and lock,

nibbling bytes 'til clocks did knock,
A gentle mlock keeps memories near,
I unlock with care — no swap or fear,
Hooray, the heap stays snug as a rock!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 1.37% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and clearly summarizes the primary change: adding optional memory locking support as a new feature.
Linked Issues check ✅ Passed The PR comprehensively addresses issue #12 by implementing memory locking (mlock/VirtualLock) to keep allocated memory resident in RAM and prevent OS swapping.
Out of Scope Changes check ✅ Passed All changes are in-scope: backend capability APIs, memory locking implementation, CLI flag exposure, progress reporting, and test coverage all directly support the memory locking feature.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-12-lock-memory

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kluster-ai
Copy link
Copy Markdown

kluster-ai Bot commented May 14, 2026

kluster.ai PR Review Summary

This PR introduces a memory locking feature to prevent the operating system from swapping allocated pages to disk. A new --lock-memory (or -l) flag has been added to the CLI, supported by backend implementations for Linux, macOS (Apple), AIX, Solaris, and Windows using mlock or VirtualLock.

Key changes include:

  • Architecture Refactoring: Introduced an eatmemory_backend abstraction to manage platform-specific capabilities like memory stats and locking support.
  • Memory Locking Logic: The eat function has been extended via eat_with_options to attempt page locking after allocation and verification.
  • Error Handling: Added specific exit codes (22, 23) for cases where memory locking is unsupported by the OS or fails due to insufficient permissions/limits.
  • Robust Cleanup: Enhanced the digest function to ensure locked memory is properly unlocked before being freed.
  • Test Coverage: Added unit tests to verify locking behavior, success paths, and proper cleanup during partial failures.

The UI has also been updated to conditionally display help text and available memory stats based on what the current backend supports.

Warning: We haven't found any reviews performed on this branch from the IDE. Make sure to install the IDE and CLI integrations to catch issues earlier and accelerate your development workflow.

Powered by kluster.ai - Real-Time AI code review in your IDE

Unfold

Comment thread src/eatmemory.linux.c Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@README.md`:
- Around line 92-97: Update the CLI flags table to include the short alias -l
alongside --lock-memory: modify the table row that currently shows
`--lock-memory` to list both `-l`, `--lock-memory` (or add a separate row for
`-l`) and keep the same description "Lock allocated pages in physical memory,
when supported." so the README flag table matches the CLI help output and
documents the short alias used by the program.

In `@src/main.c`:
- Around line 27-32: Always register the "lock-memory l" option regardless of
backend support (call ap_add_flag for "lock-memory l" in configure_cmd), but
keep the help text conditional; after argument parsing, check whether the
lock-memory flag was requested and if backend->memory_lock_supported is false
emit the EM_ERROR_MEMORY_LOCK_UNSUPPORTED error/exit code 22 (reject explicitly)
— update the post-parse validation logic (where you inspect parsed flags) to
detect the lock flag and fail when unsupported. Ensure you make the same change
in the other configure/parse site referenced (lines ~108-111).

In `@test/ci/simple/allocation.test.sh`:
- Around line 56-58: The "Lock small allocation" CI test currently asserts
successful locking by calling run_success with "-l 1K" and the "Done,
sleeping..." message which is flaky on systems with restrictive RLIMIT_MEMLOCK;
instead, change the test in allocation.test.sh to not assume locking succeeds:
either assert that the CLI exposes the lock flag/help (e.g., run_success or
run_cmd on the program with --help or -h and verify the -l/--lock option
appears) or gate the success-path assertion behind a runtime probe (use a quick
runtime check such as querying the current memlock limit via ulimit -l or a
small probe program and only expect the "Done, sleeping..." success when that
probe shows sufficient limit), leaving the actual lock-success behavior to the
malloc-hook tests; update references to the "Lock small allocation" test and
calls to run_success accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f13434dd-469d-4df8-89c4-adb90df493a2

📥 Commits

Reviewing files that changed from the base of the PR and between 8829797 and 0298ffe.

📒 Files selected for processing (14)
  • README.md
  • src/eatmemory.aix.c
  • src/eatmemory.apple.c
  • src/eatmemory.c
  • src/eatmemory.h
  • src/eatmemory.linux.c
  • src/eatmemory.posix-common.c
  • src/eatmemory.solaris.c
  • src/eatmemory.unknown.c
  • src/eatmemory.windows.c
  • src/errors.h
  • src/main.c
  • test/ci/malloc/eatmemory_malloc_hook.test.c
  • test/ci/simple/allocation.test.sh

Comment thread README.md
Comment thread src/main.c
Comment thread test/ci/simple/allocation.test.sh Outdated
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 14, 2026

Greptile Summary

This PR adds optional memory locking (--lock-memory / -l) via POSIX mlock/munlock and Windows VirtualLock/VirtualUnlock, introduces a new eatmemory_backend capability struct to gate the flag's visibility at runtime, and refactors the eat core into eat_with_options with unlock-on-free support in digest.

  • New eatmemory_backend init pattern replaces the old per-call stats->supported field with static flags set at startup; --lock-memory is only registered and shown when memory_lock_supported is true.
  • get_chunk_size helper centralises the chunk-boundary calculation that was previously inlined with a running allocated counter, removing the tracked variable from both the write and verify loops.
  • POSIX common file (eatmemory.posix-common.c) is #include-d by all POSIX backends, avoiding duplicated mlock/munlock wrappers across Linux, macOS, AIX, and Solaris.

Confidence Score: 3/5

The memory locking mechanics and cleanup paths are correct, but the migration away from per-call stats success tracking introduces a silent failure on every platform — a regression worth fixing before merge.

The old design propagated runtime stat-call failure through a per-call supported field; the new design only checks a static init-time flag. When a backend's stat call fails at runtime, the function returns zeroed stats with no error indicator. Any %-based size argument silently resolves to 0 bytes instead of producing a parse error.

src/eatmemory.c (string_to_bytes % path), src/eatmemory.aix.c, src/eatmemory.apple.c, src/eatmemory.linux.c, and src/eatmemory.windows.c all have the same runtime-failure-not-surfaced pattern.

Important Files Changed

Filename Overview
src/eatmemory.c Core logic: adds get_chunk_size helper, eat_with_options with lock-memory support, and unlock loop in digest(). The % allocation path in string_to_bytes silently returns 0 bytes when the runtime stats call fails — a behavioral regression.
src/eatmemory.h Adds eatmemory_backend struct, eatmemory_lock_result enum, and extended allocation struct fields. Clean structural changes with no issues.
src/eatmemory.posix-common.c New shared POSIX mlock/munlock implementation included by all POSIX backends. Straightforward and correct.
src/eatmemory.linux.c Adds eatmemory_init and renames get_system_memory_stats. Removed #if defined() guards around sysconf constants that previously protected against non-standard toolchains.
src/eatmemory.aix.c Adds eatmemory_init, renames stats function, includes posix-common.c. perfstat failure path returns silently with zero stats, not surfaced to callers.
src/eatmemory.apple.c Two early-return failure paths in the Mach host stats query return zero stats silently; memory_stats_supported remains true.
src/eatmemory.windows.c Adds VirtualLock/VirtualUnlock support. GlobalMemoryStatusEx failure leaves stats at zero — same pattern as other backends.
src/main.c Adds --lock-memory/-l CLI flag gated on backend capability, refactors memory summary output, routes through eat_with_options. Clean and correct.
src/errors.h Adds two new error codes (22 and 23) for lock-unsupported and lock-failed. Consistent with README.
test/ci/malloc/eatmemory_malloc_hook.test.c Adds lock/unlock intercept hooks and four new test cases. Well-structured and thorough.

Reviews (1): Last reviewed commit: "Add optional memory locking support" | Re-trigger Greptile

Comment thread src/eatmemory.c
Comment thread src/eatmemory.linux.c Outdated
julman99 added 3 commits May 14, 2026 15:16
Document the -l alias in the README and make the cross-host CLI suite verify lock option exposure instead of requiring real page-lock success.
Rename backend initialization to eatmemory_get_backend_capabilities and make eatmemory_get_system_memory_stats return an explicit result enum.

Reject percentage sizes when the runtime stats query fails, while keeping backend capabilities responsible for help rendering.
Keep the core allocation and cleanup functions named eat and digest while passing lock/progress options explicitly.

Move lock attempts into the allocation loop and show progress for eating, verification, and freeing only on interactive terminals.
Comment thread src/eatmemory.c Outdated
Comment thread src/eatmemory.c Outdated
Comment thread src/eatmemory.posix-common.c Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/eatmemory.linux.c`:
- Around line 15-29: In eatmemory_get_system_memory_stats, the sysconf check
treats free_pages > 0 as required, which wrongly marks a valid zero free-pages
state as failure; change the condition so pages and page_size remain checked for
> 0 but allow free_pages >= 0 (i.e., treat -1 as the only failure), then keep
computing stats->total and stats->free with pages_to_bytes_clamped and return
EM_STATS_OK when free_pages >= 0.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1e1d13ff-dd73-4284-9bcb-fdf743c8b990

📥 Commits

Reviewing files that changed from the base of the PR and between 0298ffe and 5517991.

📒 Files selected for processing (13)
  • AGENTS.md
  • README.md
  • src/eatmemory.aix.c
  • src/eatmemory.apple.c
  • src/eatmemory.c
  • src/eatmemory.h
  • src/eatmemory.linux.c
  • src/eatmemory.solaris.c
  • src/eatmemory.unknown.c
  • src/eatmemory.windows.c
  • src/main.c
  • test/ci/malloc/eatmemory_malloc_hook.test.c
  • test/ci/simple/allocation.test.sh
✅ Files skipped from review due to trivial changes (1)
  • AGENTS.md

Comment thread src/eatmemory.linux.c
@julman99 julman99 merged commit 744c2ac into main May 14, 2026
96 of 100 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query about swapping etc

1 participant