Skip to content

memcmp of two initialized strings in str::eq in Iterator::find leads to ud2 in core::ptr::read::precondition_check #151095

@PatchMixolydic

Description

@PatchMixolydic

I don't really understand what's happening here, so I'm afraid I don't even know where to begin with minimizing this.

This operating system consistently reproduces the bug locally. Further information, including build instructions for an x86-64 Linux host, is included in the readme. Be warned that the code is really bad, so it's entirely possible this is just the result of UB affecting the UB checks. I just wanted to get everything to work before worrying about tidying up too much, but it seems fate had other plans ^^;

Background

Upon running this operating system, the following things happen:

  • Memory management, interrupts, etc. are set up (successfully)
  • The scheduler is set up (successfully)
  • The hard drive is set up (successfully)
  • The filesystem is set up (successfully, tested by loading and printing a file right after initializing the VFS; see commit history for more)
  • I create a Process struct for a usermode process (successfully)

Things promptly go sideways:

  • I call Process::exec on the process with a path to an ELF executable
  • Process::exec tries to read the executable through the VFS
  • The VFS, in all of its hacked together prototype glory, just defers to the root filesystem, a tarball (filesystem::Tar)
  • Tar does a simple linear search to find the executable
    • Using an impl Iterator (TarIterator), it reads in each file's metadata. This is done through a shim that does pseudo-IPC with the IDE driver (to prepare for the IDE driver being moved out of the kernel)
    • By using Iterator::find, the search attempts to find a file with a filename that matches the requested filename
    • During what specifically seems to be the comparison between the requested filename and the filename of the correct file, Iterator::read calls memcmp, which calls ptr::read_unaligned::<u128>, which calls ptr::read::<u128>(correct_address.cast())(?), which calls... something, which calls core::ptr::read:precondition_check(corrupted_arguments).
    • precondition_check calls Arguments::from_str, then promptly ud2s

When building in --release (eg. with ub-checks disabled), things seem to work fine (with the code hitting a todo!() in Process::exec), though that doesn't completely rule out UB.

Bug

Start the operating system and wait for a bit. Eventually, you will see this:

...
find Programs/ == Programs/System/startup
find Programs/System/ == Programs/System/startup
find Programs/System/startup == Programs/System/startup

panicked at kernel/src/arch/amd64/interrupts.rs:113:5:
invalid opcode
InterruptStackFrame {
    instruction_pointer: VirtAddr(
        0xffffffff800668ea,
    ),
    code_segment: SegmentSelector {
        index: 1,
        rpl: Ring0,
    },
    cpu_flags: RFlags(
        RESUME_FLAG | INTERRUPT_FLAG | SIGN_FLAG | AUXILIARY_CARRY_FLAG | 0x2,
    ),
    stack_pointer: VirtAddr(
        0xffff80000001cb48,
    ),
    stack_segment: SegmentSelector {
        index: 2,
        rpl: Ring0,
    },
}

In other words, it encountered an illegal instruction at address 0xffffffff800668ea. Using the disassemble_kernel.sh utility (which just calls objdump), it becomes apparent that this is in core::ptr::read::precondition_check:

ffffffff800668a0 <core::ptr::read::precondition_check>:
ffffffff800668a0:       sub    rsp,0x28
ffffffff800668a4:       mov    al,dl
ffffffff800668a6:       mov    QWORD PTR [rsp],rdi
ffffffff800668aa:       mov    QWORD PTR [rsp+0x8],rsi
ffffffff800668af:       mov    cl,al
ffffffff800668b1:       and    cl,0x1
ffffffff800668b4:       mov    BYTE PTR [rsp+0x17],cl
ffffffff800668b8:       mov    QWORD PTR [rsp+0x18],0xffffffff8007ae1b
ffffffff800668c1:       mov    QWORD PTR [rsp+0x20],0xda
ffffffff800668ca:       movzx  edx,al
ffffffff800668cd:       and    edx,0x1
ffffffff800668d0:       call   ffffffff80066d10 <core::ub_checks::maybe_is_aligned_and_not_null>
ffffffff800668d5:       test   al,0x1
ffffffff800668d7:       jne    ffffffff800668ec <core::ptr::read::precondition_check+0x4c>
ffffffff800668d9:       mov    rdi,0xffffffff8007ae1b
ffffffff800668e0:       mov    esi,0xda
ffffffff800668e5:       call   ffffffff800666b0 <<core::fmt::Arguments>::from_str>
ffffffff800668ea:       ud2
ffffffff800668ec:       add    rsp,0x28
ffffffff800668f0:       ret

Curiously, it seems like this function is on a panicking path, but instead of actually panicking, it executes an invalid opcode.

Attaching GDB and dumping a backtrace (plus a bit of experimentation) seems to indicate that this occurred during the comparison of the correct file's filename and the requested filename.

Backtrace
#0  core::core_arch::x86::sse2::_mm_pause ()
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/x86/sse2.rs:26
#1  0xffffffff80032277 in core::hint::spin_loop ()
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/hint.rs:287
#2  snowball::panic (info=0xffff80000001ca80) at kernel/src/main.rs:51
#3  0xffffffff800592e6 in core::panicking::panic_fmt (fmt=...)
    at src/panicking.rs:80
#4  0xffffffff8002430d in snowball::arch::amd64::interrupts::invalid_opcode_handler (stack_frame=...) at kernel/src/arch/amd64/interrupts.rs:113
#5  0xffffffff800668ea in core::ptr::read::precondition_check (addr=0x8, 
    align=66194, is_zst=255)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ub_checks.rs:73
#6  0xffff80000001cbf8 in ?? ()
#7  0x0000000000000010 in ?? ()
#8  0xffffffff80065ab3 in core::ptr::read (src=0xffff80000001cbf8)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ub_checks.rs:78
#9  0xffffffff800669da in core::ptr::const_ptr::{impl#0}::read (
    self=0xffff80000001cbf8)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/const_ptr.rs:1171
#10 0xffffffff800658a0 in core::mem::maybe_uninit::MaybeUninit::assume_init (self=...)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/maybe_uninit.rs:710
#11 core::ptr::read_unaligned (src=0xffff80000002d200)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:1799
#12 0xffffffff800669ca in core::ptr::const_ptr::{impl#0}::read_unaligned
    (self=0xffff80000002d200)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/const_ptr.rs:1212
#13 0xffffffff800662ff in compiler_builtins::mem::impls::compare_bytes::cmp (
    a=0xffff80000002d200, b=0xffffffff80071e38, n=23, f=...)
    at src/mem/x86_64.rs:142
#14 compiler_builtins::mem::impls::compare_bytes::{closure#4} (
    a=0xffff80000002d200, b=0xffffffff80071e38, n=23) at src/mem/x86_64.rs:167
#15 0xffffffff800652e2 in compiler_builtins::mem::impls::compare_bytes (
    a=0xffff80000002d200, b=0xffffffff80071e38, n=23) at src/mem/x86_64.rs:168
#16 compiler_builtins::mem::memcmp (s1=0xffff80000002d200, 
    s2=0xffffffff80071e38, n=23) at src/mem/mod.rs:41
#17 0xffffffff80065757 in compiler_builtins::mem::memcmp::memcmp (
    s1=0xffff80000002d200, s2=0xffffffff80071e38, n=23) at src/macros.rs:416
#18 0xffffffff8005c603 in core::slice::cmp::{impl#5}::equal (self=..., 
    other=...) at src/slice/cmp.rs:160
#19 0xffffffff8005c57d in core::slice::cmp::{impl#0}::eq (self=..., 
    other=...) at src/slice/cmp.rs:18
#20 0xffffffff80052bb7 in core::cmp::impls::{impl#9}::eq<[u8], [u8]> (
    self=0xffff80000001cee0, other=0xffff80000001cef0) at src/cmp.rs:2115
#21 0xffffffff80008e2f in core::str::traits::{impl#1}::eq (self=..., other=...)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/traits.rs:30
#22 0xffffffff80023aad in alloc::string::{impl#93}::eq (
    self=0xffff80000001d3b8, other=0xffff80000001d6b8)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/string.rs:2668
#23 0xffffffff800388c7 in snowball::filesystem::tar::{impl#3}::find_file::{closure#0} (metadata=0xffff80000001d3b8) at kernel/src/filesystem/tar.rs:201
#24 0xffffffff80005118 in core::iter::traits::iterator::Iterator::find::check::{closure#0} (
    x=)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2903
#25 0xffffffff80038761 in core::iter::traits::iterator::Iterator::try_fold, core::ops::control_flow::ControlFlow> (
    self=0xffff80000001d6c8, init=(), f=...)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2434
#26 0xffffffff80038698 in core::iter::traits::iterator::Iterator::find (self=0xffff80000001d6c8, predicate=...)
    at /home/patchmixolydic/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2907
#27 0xffffffff80035279 in snowball::filesystem::tar::Tar::find_file (
    self=0xffff80000002c7e0, filename=...) at kernel/src/filesystem/tar.rs:199
#28 0xffffffff80036194 in snowball::filesystem::tar::{impl#4}::open_file (
    self=0xffff80000002c7e0, filename=..., 
    access_mode=snowball::filesystem::AccessMode::ReadOnly)
    at kernel/src/filesystem/tar.rs:213
#29 0xffffffff8001ad10 in snowball::filesystem::Vfs::open (
    self=0xffffffff8007c050 , filename=..., 
    access_mode=snowball::filesystem::AccessMode::ReadOnly)
    at kernel/src/filesystem/mod.rs:105
#30 0xffffffff80031599 in snowball::process::Process::exec (
    self=0xffff80000001e0b8, filename=...) at kernel/src/process.rs:41
#31 0xffffffff8003209e in snowball::post_scheduler_main ()
    at kernel/src/main.rs:102
#32 0xffff80000001e170 in ?? ()
#33 0x0000000000000010 in ?? ()
#34 0x0000000000000000 in ?? ()

Particularly concerning are entries 6 and 7, which seem to be invalid (perhaps an interrupt arrived?), as well as 24, which appears to be the first call with an invalid argument (though I don't understand where this argument could've come from).

Meta

rustc --version --verbose:

rustc 1.94.0-nightly (2f1bd3f37 2026-01-12)
binary: rustc
commit-hash: 2f1bd3f3781c90a8447e37d65a898442b8618895
commit-date: 2026-01-12
host: x86_64-unknown-linux-gnu
release: 1.94.0-nightly
LLVM version: 21.1.8

.cargo/config.toml:

[build]
target = "x86_64-snowkernel.json"

[target."x86_64-snowkernel"]
runner = "bootimage runner"

[unstable]
build-std = ["core", "compiler_builtins", "alloc"]
build-std-features = ["compiler-builtins-mem"]

./x86_64-snowkernel.json:

{
    "arch": "x86_64",
    "cpu": "x86-64-v2",
    "code-model": "kernel",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "disable-redzone": true,
    "executables": true,
    "features": "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2,+soft-float",
    "linker": "rust-lld",
    "linker-flavor": "ld.lld",
    "llvm-target": "x86_64-unknown-none-elf",
    "max-atomic-width": 64,
    "os": "none",
    "panic-strategy": "abort",
    "plt-by-default": false,
    "position-independent-executables": false,
    "post-link-args": {"ld.lld": ["-Tkernel.ld"]},
    "relocation-model": "static",
    "rustc-abi": "x86-softfloat",
    "target-c-int-width": 32,
    "target-endian": "little",
    "target-pointer-width": 64
}

@rustbot labels +E-needs-mcve +O-bare-metal

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-compiler-builtinsArea: compiler-builtins (https://github.com/rust-lang/compiler-builtins)C-gubCategory: the reverse of a compiler bug is generally UBO-bare-metalTarget: Rust without an operating system

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions