mmap-io
^{_{MEMORY-MAPPED FILE I/O FOR RUST}}

Zero-copy reads. Lock-free atomic views. Safe concurrent access.
Built for databases, log structures, caches, game runtimes, and shared-memory IPC.

What you get

Zero-copy reads on every mode. as_slice returns a MappedSlice<'_> borrowed directly from the mapping. No allocation. No memcpy. Works on read-only, read-write, and copy-on-write mappings uniformly.
Zero-allocation iteration. mmap.chunks(N) and mmap.pages() walk the file in fixed strides without ever heap-allocating. A 1 GiB scan at 4 KiB chunks skips 262,144 allocations and half the memory bandwidth of the naive approach.
Aligned atomic views. atomic_u32 / atomic_u64 return a wrapper that derefs to &AtomicU64. Multi-thread fetch_add over a memory-mapped counter is one cache-line ping; no cross-process locking required.
Configurable durability. FlushPolicy::EveryBytes(N), EveryWrites(N), EveryMillis(N), Always, or Manual. The accumulator is correctly debited on partial flushes (audit C1) and the millis policy actually runs a background flusher (audit C2).
Thread-safe. Interior mutability via parking_lot::RwLock. Multiple concurrent readers, one writer at a time. Live atomic views block resize() until released so memory under your reference cannot move (audit C3).
Cross-platform. Linux, macOS, Windows. Per-platform fast paths (MS_ASYNC flush on Linux, MADV_HUGEPAGE on huge-page hints, posix_fadvise for OS-level prefetch).
Opt-in surface. Default features are advise + iterator. Everything else (async, atomic, cow, locking, watch, hugepages) is off by default to keep compile time tight.
MSRV: 1.75. Pinned and verified in CI.

Quick start

[dependencies]
mmap-io = "0.9"

use mmap_io::MemoryMappedFile;

fn main() -> Result<(), mmap_io::MmapIoError> {
    // Open an existing file in read-only mode.
    let mmap = MemoryMappedFile::open_ro("data.bin")?;

    // Zero-copy read of the first 16 bytes. `slice` derefs to &[u8].
    let slice = mmap.as_slice(0, 16)?;
    println!("First bytes: {:?}", &*slice);

    Ok(())
}

Open-or-create with one call:

use mmap_io::MemoryMappedFile;

fn main() -> Result<(), mmap_io::MmapIoError> {
    // Opens "data.bin" if it exists; creates it at 1 MiB otherwise.
    let mmap = MemoryMappedFile::open_or_create("data.bin", 1024 * 1024)?;

    mmap.update_region(100, b"Hello, mmap!")?;
    mmap.flush()?;
    Ok(())
}

Migrating from 0.9.6

Versions 0.9.7-0.9.10 changed three method signatures without a major version bump (a semver violation we should have caught; see CHANGELOG.md for the full apology). Since 0.9.11 we ship compat shims so 0.9.6 code recovers with a one-line rename per call site:

0.9.6 signature	Use this in 0.9.11+ for a drop-in fix
`mmap.as_slice(off, len)?` → `&[u8]`	`mmap.as_slice_bytes(off, len)?`
`mmap.chunks(N)` yields `Result<Vec<u8>>`	`mmap.chunks_owned(N)`
`for_each_mut(..)` with `Result<Result<(), E>>`	`for_each_mut_legacy(..)`

The new (unflagged) replacements (as_slice → MappedSlice<'_>, chunks → MappedSlice<'a>, flattened for_each_mut → Result<()>) are the recommended path for new code: they're zero-copy and faster. The compat shims preserve the 0.9.6 ergonomics for code that doesn't want to migrate yet.

Optional features

Feature	Description
`async`	Runtime-agnostic async helpers via the `blocking` crate. Works on tokio, smol, async-std, or any executor. Since 0.9.11.
`bytes`	`bytes::Bytes` conversion for plugging into the hyper/tower/tonic/axum/reqwest ecosystem. Since 0.9.11.
`advise`	Memory hinting via `madvise`/`posix_madvise` (Unix) or `PrefetchVirtualMemory` (Windows).
`iterator`	Iterator-based access to memory chunks or pages with zero-copy reads.
`hugepages`	Huge Pages via MAP_HUGETLB (Linux) or FILE_ATTRIBUTE_LARGE_PAGES (Windows); falls back to regular pages.
`cow`	Copy-on-Write mapping mode using private per-process memory views.
`locking`	Page-level memory locking via `mlock`/`munlock` (Unix) or `VirtualLock` (Windows).
`atomic`	Atomic views into memory as aligned `u32` / `u64` with strict alignment checks.
`watch`	Native file-change notifications: `inotify` (Linux), FSEvents (macOS), `ReadDirectoryChangesW` (Windows).

⚠️ Features are opt-in. Enable only those relevant to your use case to reduce compile time and dependency footprint.

Default features

By default, the following features are enabled:

advise — memory access hinting for performance.
iterator — iterator-based chunk/page access.

Installation patterns

Default features:

[dependencies]
mmap-io = "0.9"

Enable async helpers:

[dependencies]
mmap-io = { version = "0.9", features = ["async"] }

Multiple features:

[dependencies]
mmap-io = { version = "0.9", features = ["cow", "locking"] }

Minimal — disable defaults, opt into only what you need:

[dependencies]
mmap-io = { version = "0.9", default-features = false, features = ["locking"] }

Flush Policy

mmap-io supports configurable flush behavior for ReadWrite mappings via FlushPolicy, letting you trade off durability and throughput.

Policy variants:

FlushPolicy::Never / FlushPolicy::Manual — no automatic flushes. Call mmap.flush() when you want durability.
FlushPolicy::Always — flush after every write; slowest but most durable.
FlushPolicy::EveryBytes(n) — accumulate bytes written across update_region() calls; flush when at least n bytes have been written.
FlushPolicy::EveryWrites(n) — flush after every n writes.
FlushPolicy::EveryMillis(ms) — automatically flushes pending writes at the specified interval using a background thread.

Builder usage:

use mmap_io::{MemoryMappedFile, MmapMode};
use mmap_io::flush::FlushPolicy;

let mmap = MemoryMappedFile::builder("file.bin")
    .mode(MmapMode::ReadWrite)
    .size(1_000_000)
    .flush_policy(FlushPolicy::EveryBytes(256 * 1024)) // flush every 256KB written
    .create()?;

Manual flush:

use mmap_io::{create_mmap, update_region, flush};

let mmap = create_mmap("data.bin", 1024 * 1024)?;
update_region(&mmap, 0, b"batch1")?;
// ... more batched writes ...
flush(&mmap)?; // ensure durability now

Note

On some platforms, visibility of writes without explicit flush may still occur due to OS behavior, but durability timing is best-effort without flush.

Round-trip example

Create a file, write to it, and read back:

use mmap_io::{create_mmap, update_region, flush, load_mmap, MmapMode};

fn main() -> Result<(), mmap_io::MmapIoError> {
    // Create a 1MB memory-mapped file
    let mmap = create_mmap("data.bin", 1024 * 1024)?;

    // Write data at offset 100
    update_region(&mmap, 100, b"Hello, mmap!")?;

    // Persist to disk
    flush(&mmap)?;

    // Open read-only and verify
    let ro = load_mmap("data.bin", MmapMode::ReadOnly)?;
    let slice = ro.as_slice(100, 12)?;
    assert_eq!(slice, b"Hello, mmap!");

    Ok(())
}

Memory Advise (`feature = "advise"`)

Optimize OS-level memory access patterns:

#[cfg(feature = "advise")]
use mmap_io::{create_mmap, MmapAdvice};

fn main() -> Result<(), mmap_io::MmapIoError> {
    let mmap = create_mmap("data.bin", 1024 * 1024)?;

    // Advise sequential access for better prefetching
    mmap.advise(0, 1024 * 1024, MmapAdvice::Sequential)?;

    // Process file sequentially...

    // Advise that we won't need this region soon
    mmap.advise(0, 512 * 1024, MmapAdvice::DontNeed)?;

    Ok(())
}

Iterator-Based Access (`feature = "iterator"`)

Process files in chunks or pages:

#[cfg(feature = "iterator")]
use mmap_io::create_mmap;

fn main() -> Result<(), mmap_io::MmapIoError> {
    let mmap = create_mmap("large_file.bin", 10 * 1024 * 1024)?;

    // Process file in 1MB chunks
    for (i, chunk) in mmap.chunks(1024 * 1024).enumerate() {
        let data = chunk?;
        println!("Processing chunk {i} with {} bytes", data.len());
    }

    // Process file page by page (OS-optimal)
    for page in mmap.pages() {
        let _page_data = page?;
        // Process page...
    }

    Ok(())
}

Page Pre-warming

Eliminate page-fault latency by pre-warming pages into memory before a critical section:

use mmap_io::{MemoryMappedFile, MmapMode, TouchHint};

fn main() -> Result<(), mmap_io::MmapIoError> {
    // Eagerly pre-warm all pages on creation for benchmarks
    let mmap = MemoryMappedFile::builder("benchmark.bin")
        .mode(MmapMode::ReadWrite)
        .size(1024 * 1024)
        .touch_hint(TouchHint::Eager)
        .create()?;

    // Manually pre-warm a specific range before a critical operation
    mmap.touch_pages_range(0, 512 * 1024)?;

    Ok(())
}

Atomic Operations (`feature = "atomic"`)

Lock-free concurrent access at aligned offsets:

#[cfg(feature = "atomic")]
use mmap_io::create_mmap;
use std::sync::atomic::Ordering;

fn main() -> Result<(), mmap_io::MmapIoError> {
    let mmap = create_mmap("counters.bin", 64)?;

    // Get atomic view of u64 at offset 0
    let counter = mmap.atomic_u64(0)?;
    counter.store(0, Ordering::SeqCst);

    // Increment atomically from multiple threads
    let old = counter.fetch_add(1, Ordering::SeqCst);
    println!("Counter was: {old}");

    Ok(())
}

Memory Locking (`feature = "locking"`)

Prevent pages from being swapped (requires elevated privileges):

#[cfg(feature = "locking")]
use mmap_io::create_mmap;

fn main() -> Result<(), mmap_io::MmapIoError> {
    let mmap = create_mmap("critical.bin", 4096)?;

    // Lock pages in memory
    mmap.lock(0, 4096)?;

    // Critical operations that need guaranteed memory residence...

    // Unlock when done
    mmap.unlock(0, 4096)?;

    Ok(())
}

File Watching (`feature = "watch"`)

Native OS event sources (inotify on Linux, FSEvents on macOS, ReadDirectoryChangesW on Windows). Drop the returned handle to stop the watch and release the OS subscription.

#[cfg(feature = "watch")]
use mmap_io::{create_mmap, ChangeEvent};

fn main() -> Result<(), mmap_io::MmapIoError> {
    let mmap = create_mmap("watched.bin", 1024)?;

    let _handle = mmap.watch(|event: ChangeEvent| {
        println!("File changed: {:?}", event.kind);
    })?;

    // File is being watched... handle is dropped when out of scope.

    Ok(())
}

Note: mmap-side writes (update_region + flush) are not a reliable trigger for FS watchers; they reach the watcher only at OS-decided writeback time. Reliable detection comes from std::fs API writes (another process, another file handle) — which is the real-world use case for watch.

Copy-on-Write Mode (`feature = "cow"`)

Private per-process memory views:

#[cfg(feature = "cow")]
use mmap_io::MemoryMappedFile;

fn main() -> Result<(), mmap_io::MmapIoError> {
    let cow_mmap = MemoryMappedFile::open_cow("shared.bin")?;

    // Reads see the original file content
    let _data = cow_mmap.as_slice(0, 100)?;

    // Writes affect this process only; underlying file remains unchanged.
    Ok(())
}

Async Operations (`feature = "async"`)

Tokio-based async helpers:

#[cfg(feature = "async")]
#[tokio::main]
async fn main() -> Result<(), mmap_io::MmapIoError> {
    use mmap_io::manager::r#async::{create_mmap_async, copy_mmap_async};

    let mmap = create_mmap_async("async.bin", 4096).await?;
    mmap.update_region(0, b"async data")?;
    mmap.flush()?;

    copy_mmap_async("async.bin", "copy.bin").await?;

    Ok(())
}

Async-Only Flushing

When using async write helpers, mmap-io enforces durability by flushing after each async write. This avoids visibility inconsistencies across platforms when awaiting async tasks.

#[cfg(feature = "async")]
#[tokio::main(flavor = "multi_thread")]
async fn main() -> Result<(), mmap_io::MmapIoError> {
    use mmap_io::MemoryMappedFile;

    let mmap = MemoryMappedFile::create_rw("data.bin", 4096)?;
    // Async write that auto-flushes under the hood
    mmap.update_region_async(128, b"ASYNC-FLUSH").await?;
    // Optional explicit async flush
    mmap.flush_async().await?;
    Ok(())
}

Contract: after awaiting update_region_async or flush_async, opening a fresh RO mapping observes the persisted data.

Platform Parity

Flush visibility is guaranteed across operating systems: after calling flush() or flush_range(), a newly opened read-only mapping will observe the persisted bytes on all supported platforms.

Full-file flush: both written regions are visible after flush().
Range flush: only the flushed range is guaranteed visible; a later flush() persists remaining regions.

See parity tests in the repository that validate this contract on each platform.

Huge Pages (`feature = "hugepages"`)

Best-effort huge page support to reduce TLB misses and improve performance for large mappings.

Linux — multi-tier approach for huge page allocation:

Tier 1: Optimized mapping with immediate MADV_HUGEPAGE to encourage kernel huge page allocation.
Tier 2: Standard mapping with MADV_HUGEPAGE hint for Transparent Huge Pages (THP).
Tier 3: Silent fallback to regular pages if huge pages are unavailable.

Windows — attempts FILE_ATTRIBUTE_LARGE_PAGES. Requires the "Lock Pages in Memory" privilege and system configuration. Falls back to normal pages if unavailable.

Other platforms — no-op.

⚠️ .huge_pages(true) does NOT guarantee huge pages will be used. Actual allocation depends on system configuration, available memory, kernel heuristics, and process privileges. The mapping functions correctly regardless of whether huge pages are actually used.

Builder usage:

#[cfg(feature = "hugepages")]
use mmap_io::{MemoryMappedFile, MmapMode};

let mmap = MemoryMappedFile::builder("hp.bin")
    .mode(MmapMode::ReadWrite)
    .size(2 * 1024 * 1024) // 2MB - typical huge page size
    .huge_pages(true) // best-effort optimization
    .create()?;

Safety Notes

All operations perform bounds checks.
Unsafe blocks are limited to mapping calls and documented with SAFETY comments.
Interior mutability uses parking_lot::RwLock for high performance.
Avoid flushing while holding a write guard to prevent deadlocks — drop the guard first.

⚠️ Unsafe Code Disclaimer

This crate uses unsafe internally to manage raw memory mappings (mmap, VirtualAlloc, etc.) across platforms. Public APIs are designed to be memory-safe when used correctly. However:

You must not modify the file concurrently outside of this process.
Mapped slices are only valid as long as the underlying file and mapping stay valid.
Behavior is undefined if you access a truncated or deleted file via a stale mapping.

All unsafe logic is documented in the source and footguns are marked with caution.

Minimum supported Rust version

1.75 — pinned in Cargo.toml and verified by CI.

License

Licensed under the Apache License, Version 2.0. See LICENSE for the full text.

You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for specific language governing permissions and limitations.

Name		Name	Last commit message	Last commit date
Latest commit History 91 Commits
.github/workflows		.github/workflows
benches		benches
docs		docs
examples		examples
fuzz		fuzz
src		src
tests		tests
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
REPS.md		REPS.md
clippy.toml		clippy.toml
deny.toml		deny.toml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

mmap-io
^{_{MEMORY-MAPPED FILE I/O FOR RUST}}

What you get

Quick start

Migrating from 0.9.6

Optional features

Default features

Installation patterns

Flush Policy

Round-trip example

Memory Advise (`feature = "advise"`)

Iterator-Based Access (`feature = "iterator"`)

Page Pre-warming

Atomic Operations (`feature = "atomic"`)

Memory Locking (`feature = "locking"`)

File Watching (`feature = "watch"`)

Copy-on-Write Mode (`feature = "cow"`)

Async Operations (`feature = "async"`)

Async-Only Flushing

Platform Parity

Huge Pages (`feature = "hugepages"`)

Safety Notes

⚠️ Unsafe Code Disclaimer

Minimum supported Rust version

Further reading

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

mmap-io MEMORY-MAPPED FILE I/O FOR RUST

What you get

Quick start

Migrating from 0.9.6

Optional features

Default features

Installation patterns

Flush Policy

Round-trip example

Memory Advise (feature = "advise")

Iterator-Based Access (feature = "iterator")

Page Pre-warming

Atomic Operations (feature = "atomic")

Memory Locking (feature = "locking")

File Watching (feature = "watch")

Copy-on-Write Mode (feature = "cow")

Async Operations (feature = "async")

Async-Only Flushing

Platform Parity

Huge Pages (feature = "hugepages")

Safety Notes

⚠️ Unsafe Code Disclaimer

Minimum supported Rust version

Further reading

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

mmap-io
^{_{MEMORY-MAPPED FILE I/O FOR RUST}}

Memory Advise (`feature = "advise"`)

Iterator-Based Access (`feature = "iterator"`)

Atomic Operations (`feature = "atomic"`)

Memory Locking (`feature = "locking"`)

File Watching (`feature = "watch"`)

Copy-on-Write Mode (`feature = "cow"`)

Async Operations (`feature = "async"`)

Huge Pages (`feature = "hugepages"`)

Packages