Skip to content

Fix tracer pkey re-protection during init for libc compartment support#666

Merged
oinoom merged 2 commits intomainfrom
fix/tracer-pkey-reprotect-init
Feb 11, 2026
Merged

Fix tracer pkey re-protection during init for libc compartment support#666
oinoom merged 2 commits intomainfrom
fix/tracer-pkey-reprotect-init

Conversation

@oinoom
Copy link
Copy Markdown
Contributor

@oinoom oinoom commented Feb 10, 2026

Background

When IA2_LIBC_COMPARTMENT is enabled, the custom glibc's __minimal_malloc protects its entire heap (including TLS blocks) with pkey 1. During IA2 initialization, protect_tls_pages re-protects each compartment's TLS with the correct per-compartment pkey. The kernel handles this correctly via VMA splitting, but the tracer's Rust memory map tracking rejected the operation with overly strict monotonicity checks.

  • C policy (is_op_permitted in track_memory_map.c): already allows compartment 0 to pkey_mprotect any memory during init
  • Rust map tracking (memory_map_pkey_mprotect_region in lib.rs): rejected it because the region was already owned by another pkey or already pkey_mprotect'd

Changes

Relax the Rust monotonicity checks before init_finished, matching the existing pattern for regular mprotect. During init:

  • Allow re-protecting memory owned by another compartment (needed because the loader's pkey 1 heap contains TLS for all compartments)
  • Allow repeated pkey_mprotect on the same region
  • Update owner_pkey to the new pkey during init
  • Remove the commented-out cmake workaround that disabled the sandbox for all libc compartment tests
  • Run loader_minimal_malloc without the sandbox since it reads /proc/self/smaps, which is not allowed by landlock (the sandbox's filesystem walk skips /proc/self because it is a symlink)

David Anekstein added 2 commits February 10, 2026 10:20
When IA2_LIBC_COMPARTMENT is enabled, the custom glibc's __minimal_malloc
protects its entire heap (including TLS blocks) with IA2_LDSO_PKEY (pkey 1).
During IA2 initialization, protect_tls_pages must re-protect each
compartment's TLS with the correct per-compartment pkey. The kernel
handles this correctly via VMA splitting.

The C policy check (is_op_permitted) already allows compartment 0 to
pkey_mprotect any memory during init, but the Rust memory map tracking
rejected it with overly strict monotonicity checks. This aligns the Rust
checks with the C policy by relaxing them before init_finished, matching
the existing pattern for regular mprotect.

Also re-enable the sandbox for libc compartment tests now that the tracer
supports the re-protection pattern.
The tracer now handles pkey re-protection during init, so there is no
need to disable the sandbox for libc compartment tests. Also run
loader_minimal_malloc without the sandbox since it reads
/proc/self/smaps, which is blocked by landlock.
@oinoom oinoom requested a review from fw-immunant February 10, 2026 21:17
Copy link
Copy Markdown
Contributor

@fw-immunant fw-immunant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CMake change needs cleanup, but otherwise looks good.

Comment thread cmake/define-test.cmake Outdated
@oinoom oinoom merged commit a17f658 into main Feb 11, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants