Linux file inventory and process tools

A semester-long Linux systems programming project implemented in C and Bash. The project evolves from basic Linux command-line auditing into a multiprocess file inventory system with versioned binary databases, mmap-based inter-process communication, fork/exec workers, pipes, signals, and reproducible command-line orchestration.

The code is organized as five incremental homework stages, each preserving the same project structure while adding a new operating-system concept. Together, the stages form a coherent toolkit around two main directions:

• FileOps — file-tree traversal, metadata extraction, file inventory databases, checksums, and snapshot comparison.
• ProcOps — process observation through /proc, process snapshots, worker processes, status reporting, and signal-driven control.

Project highlights

• Linux-focused systems programming in C and Bash.
• Reproducible workflow through a single orchestration script: tools/fileops.sh.
• Recursive source discovery and incremental compilation.
• Versioned binary database formats with fixed headers and fixed-size records.
• Controlled database updates without uncontrolled append-only growth.
• fcntl-based synchronization for concurrent snapshot writers.
• Process snapshots collected from /proc.
• Recursive file indexing with symlink-aware traversal.
• Multiprocess architecture using a manager and multiple worker processes.
• Worker startup through fork() followed by exec().
• Shared-memory IPC through mmap(..., MAP_SHARED, ...).
• Ring-buffer job queues and result queues protected by process-shared semaphores.
• Atomic final database publication through temporary-file writing and rename().
• SHA-256 hashing for regular files in the inventory builder.
• Pipe-based control plane for worker-to-manager messages.
• Signal handling for on-demand status and graceful shutdown.
• Structurally valid incomplete databases using complete=0 after controlled termination.

Repository layout

Each homework keeps the same standard structure:

HWx/
├── bin/        # generated executables, ignored by Git except .gitkeep
├── data/       # runtime databases and IPC files, ignored by Git except .gitkeep
├── doc/        # technical documentation
├── include/    # C header files
├── logs/       # generated execution logs, ignored by Git except .gitkeep
├── reports/    # generated test/audit reports, ignored by Git except .gitkeep
├── src/        # C source files
├── tests/      # non-interactive shell tests
├── tmp/        # temporary build/test data, ignored by Git except .gitkeep
└── tools/      # Bash orchestration scripts

Generated binaries, databases, logs, reports and temporary files are intentionally not versioned. They are reproducible through the provided scripts.

Homework evolution

Stage	Focus	Main contribution
HW1	Linux audit and project organization	Standard project layout, documented Linux commands, audit script and generated reports.
HW2	Bash orchestration	Single entry point for init, build, run, clean and test.
HW3	File and process snapshots	C utilities for file indexing, process snapshots and binary database diffing.
HW4	Multiprocess inventory builder	Manager/worker architecture with fork, exec, mmap, semaphores and atomic DB output.
HW5	Control plane and signals	Pipe-based progress messages, SIGUSR1 status, graceful SIGINT/SIGTERM shutdown and valid incomplete DBs.

Requirements

This project targets a Linux environment.

Required tools:

• bash
• gcc
• standard Linux command-line utilities such as find, grep, sort, head, cut, wc, ps, pstree, du
• POSIX/Linux APIs available through the system C library

The C code uses Linux/POSIX mechanisms such as:

• open, read, write, lseek, close
• stat, lstat, readlink
• fcntl file locks
• fork, exec, wait
• mmap, munmap, ftruncate
• process-shared POSIX semaphores
• pipe
• signal/sigaction-style signal handling

No third-party C libraries are required.

Build and test workflow

Run commands from inside the homework directory you want to test.

Example:

cd HW5
chmod +x tools/fileops.sh
./tools/fileops.sh init
./tools/fileops.sh build
./tools/fileops.sh test

For a warning-enabled GCC build, the scripts can be used with CFLAGS:

CFLAGS="-std=gnu11 -Wall -Wextra" ./tools/fileops.sh build

gnu11 is used here because the project relies on POSIX/Linux APIs such as lstat, readlink, mmap, fork, exec, pipes and semaphores.

HW1 — Linux audit and project structure

HW1 establishes the base project organization and introduces reproducible command-line auditing.

Implemented elements:

• standard directory layout;

• doc/README.md describing the project structure;

• doc/T1_comenzi.md documenting twelve Linux commands grouped into:

  • filesystem commands;
  • process commands;
  • /proc-based commands;
  • pipeline commands;

• tools/t1_audit.sh, a Bash script that creates report directories and generates audit outputs.

Run:

cd HW1
bash tools/t1_audit.sh

The script generates reports under reports/, grouped by command category.

HW2 — Bash project orchestrator

HW2 introduces the reusable project entry point used by the later assignments:

./tools/fileops.sh {init|build|run|clean|test}

Implemented features:

• init creates the standard project structure and checks for gcc;
• build recursively discovers C source files;
• auxiliary .c files are compiled into object files under tmp/obj/;
• files named main_<name>.c generate executables named bin/<name>;
• compilation is incremental: source files are recompiled only when needed;
• run executes binaries from bin/ using -- as an argument separator;
• clean removes generated build artifacts;
• test recursively discovers shell tests and writes a PASS/FAIL report.

Run:

cd HW2
./tools/fileops.sh init
./tools/fileops.sh build
./tools/fileops.sh test

HW3 — Versioned binary databases and snapshots

HW3 adds C-based FileOps and ProcOps utilities.

Generated executables:

bin/fileops_indexer
bin/proc_snapshot
bin/db_diff

fileops_indexer

Recursively indexes a directory tree and stores the result in a binary database.

Captured file information includes:

• absolute path;
• entry type: regular file, directory, symlink or FIFO;
• file size for regular files;
• modification time;
• deterministic checksum for regular file contents;
• st_dev and st_ino values;
• symlink target when applicable.

Symlinks are handled with lstat() and are recorded as symlinks without being dereferenced or traversed recursively.

Example:

cd HW3
./tools/fileops.sh build
mkdir -p tmp/demo-root
printf "hello\n" > tmp/demo-root/file.txt
./tools/fileops.sh run -- fileops_indexer --root tmp/demo-root --db data/index.db

proc_snapshot

Captures a snapshot of current processes using /proc and writes it to a binary database.

Stored process information includes:

• PID and PPID;
• process state;
• comm;
• truncated command line;
• RSS value;
• CPU time information.

Processes that disappear while the snapshot is being collected are handled without invalidating the database.

Example:

./tools/fileops.sh run -- proc_snapshot --db data/proc.db

db_diff

Compares two databases of the same type and writes a stable text report.

Example:

./tools/fileops.sh run -- db_diff \
  --old data/index_old.db \
  --new data/index_new.db \
  --out reports/T3_filediff.txt

HW3 database design

Both databases use a common versioned header containing:

• magic — database type identifier, such as IDX1 or PRC1;
• format_version;
• snapshot_id;
• snapshot_state;
• active_writers;
• record_count.

The implementation uses fixed-size records and direct offset calculation. This simplifies validation, in-place updates and record-level synchronization. Concurrent writers are coordinated using fcntl locks and the snapshot lifecycle transitions from OPEN to SEALED when the last writer detaches.

HW4 — Multiprocess file inventory builder

HW4 evolves the project into a multiprocess file inventory application.

Generated executables:

bin/fileops_manager
bin/fileops_worker

The manager is responsible for:

• parsing and validating CLI arguments;
• initializing the shared IPC file;
• starting workers using fork() and exec();
• collecting file records from shared memory;
• waiting for worker termination;
• writing the final binary database atomically;
• verifying and dumping the final database.

Workers are responsible for:

• attaching to the shared IPC region;
• retrieving directory jobs from the shared job queue;
• scanning directories;
• publishing file records to the result queue;
• discovering subdirectories and creating new jobs;
• publishing per-worker statistics.

HW4 IPC architecture

The main data flow uses a file mapped with:

mmap(..., MAP_SHARED, ...)

The shared IPC region contains:

• IPC header and global state;
• fixed-capacity job queue;
• fixed-capacity result queue;
• process-shared semaphores;
• per-worker statistics.

The queues are implemented as ring buffers. Synchronization prevents corruption of jobs, results and statistics. Backpressure is handled through semaphores so producers do not overwrite unread data.

HW4 final database

The final inventory database uses magic INV4 and includes:

• a versioned header;
• a complete flag;
• file records;
• per-worker statistics;
• stable offsets for verification and dump mode.

The manager writes the final database atomically by writing to a temporary file first and then publishing it with rename().

Example:

cd HW4
./tools/fileops.sh build
mkdir -p tmp/inventory-root/subdir
printf "sample\n" > tmp/inventory-root/subdir/file.txt

./tools/fileops.sh run -- fileops_manager \
  --root tmp/inventory-root \
  --workers 4 \
  --ipc data/ipc.mmap \
  --db data/inventory.db

./tools/fileops.sh run -- fileops_manager --db data/inventory.db --verify
./tools/fileops.sh run -- fileops_manager --db data/inventory.db --dump

HW5 — Control plane, pipes and signals

HW5 extends the HW4 manager/worker architecture without replacing the shared-memory data plane.

Added mechanisms:

• anonymous pipe for short worker-to-manager control messages;
• stable T5MSG message format;
• SIGUSR1 for on-demand status;
• SIGINT and SIGTERM for graceful shutdown;
• manager PID file for deterministic external signalling;
• complete=0 for structurally valid incomplete databases.

Data plane vs control plane

HW5 separates the architecture into three planes:

• Data plane — file records, jobs, result queues and statistics remain in mmap shared memory.
• Control plane — short progress and error messages are sent through an anonymous pipe.
• Signal plane — the manager reacts to status and shutdown signals.

The pipe is not used to transmit inventory records. It is used only for short control messages such as:

T5MSG type=JOB_DONE worker_id=<id> jobs=<n> files=<n> bytes=<n>
T5MSG type=WORKER_EXITING worker_id=<id> reason=<reason>
T5MSG type=ERROR worker_id=<id> errno=<errno> where=<context>

Workers receive the inherited pipe descriptor through the --control-fd argument after fork() and exec().

Signal behavior

SIGUSR1 requests a stable status line from the manager:

STATUS queued_jobs=<n> active_jobs=<n> files=<n> bytes=<n> workers_alive=<n> complete=0

SIGINT and SIGTERM request a controlled shutdown:

• the manager sets shutdown flags;
• workers stop in a controlled manner;
• remaining results are drained;
• worker statistics are preserved;
• the manager waits up to the configured graceful timeout;
• the final database is still written in a structurally valid format;
• the database uses complete=0 if the inventory did not finish normally.

Example:

cd HW5
./tools/fileops.sh build
mkdir -p tmp/inventory-root/subdir
printf "sample\n" > tmp/inventory-root/subdir/file.txt

./tools/fileops.sh run -- fileops_manager \
  --root tmp/inventory-root \
  --workers 4 \
  --simulate-work-ms 20 \
  --pid-file tmp/manager.pid \
  --db data/inventory.db \
  --ipc data/ipc.mmap &

wrapper_pid=$!
sleep 1
manager_pid=$(cat tmp/manager.pid)
kill -USR1 "$manager_pid"
kill -TERM "$manager_pid"
wait "$wrapper_pid"

./tools/fileops.sh run -- fileops_manager --db data/inventory.db --verify
./tools/fileops.sh run -- fileops_manager --db data/inventory.db --dump

Running all available tests

From the repository root:

for hw in HW2 HW3 HW4 HW5; do
  echo "== $hw =="
  (
    cd "$hw" || exit 1
    chmod +x tools/fileops.sh
    ./tools/fileops.sh init
    ./tools/fileops.sh test
  )
done

HW1 is an audit/documentation stage and can be run separately:

cd HW1
bash tools/t1_audit.sh

Cleaning generated files

Inside any homework directory:

./tools/fileops.sh clean

This removes build outputs generated by the orchestrator. Runtime databases, logs and reports are ignored by Git and can be regenerated through tests and program runs.

Version-control policy

The repository is intended to track:

• C source files;
• header files;
• Bash scripts;
• tests;
• Markdown documentation;
• .gitkeep placeholders for required empty directories.

The repository intentionally ignores:

• generated executables in bin/;
• object files in tmp/obj/;
• generated .db databases;
• .mmap IPC files;
• PID and lock files;
• generated logs;
• generated reports;
• temporary test data.

This keeps the repository clean while preserving reproducibility.

Technical learning outcomes

This project demonstrates practical understanding of:

• Linux filesystem organization;
• shell scripting and command orchestration;
• recursive file discovery;
• POSIX file I/O;
• binary file formats;
• database validation and stable dump output;
• concurrency control using file locks;
• /proc process introspection;
• process creation and replacement;
• shared-memory IPC;
• process-shared semaphores;
• producer-consumer ring buffers;
• atomic file publication;
• pipe-based message passing;
• asynchronous process notification through signals;
• graceful shutdown design.

Notes

This repository is an academic Operating Systems coursework project. It is designed to demonstrate Linux systems programming concepts through a progressive, testable and reproducible implementation.