diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a9b052..a6996b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,27 +2,39 @@ All notable changes to `xtafkit` will be documented in this file. -## [1.3.0] - 2026-05-17 +## [1.3.0] - 2026-05-18 -Adds read-extraction for Xbox 360 STFS container packages — Arcade (XBLA), XBLIG, Title Updates, Marketplace DLC. New library surface under `fatxlib::stfs::extract` and a new `xtafkit extract-stfs` CLI subcommand. TUI integration for STFS is intentionally deferred to a later release. +Adds read-extraction for Xbox 360 STFS container packages — Arcade (XBLA), XBLIG, Title Updates, Marketplace DLC. Covers both the CLI (`xtafkit extract-stfs`) and the TUI upload-sniff path (`u` on a local STFS file → `e(X)tract / (R)aw / Esc`). New library surface under `fatxlib::stfs::extract` with a `StfsSink` trait abstraction shared between host-filesystem and FATX-volume destinations. -### STFS extraction +### STFS extraction (CLI) - Added `xtafkit extract-stfs ` for streaming Xbox 360 STFS containers (CON / LIVE / PIRS) to a local directory. `--to ` overrides the destination; `--dry-run` lists the entries with sizes and totals; `--json` emits machine-readable output for both dry-run and post-extract modes. - Default destination (no `--to`) is `.//` taken straight from the STFS header — no catalog lookup, no `[TitleID]` suffix. The per-package `display_name` is consistently more specific than the title-id catalog mapping (notably for XBLIG, where every indie game shares the same system title-id and the catalog can only return a generic bucket name). - Read-only / type-1 STFS only. Type-0 (read-write, used by save games / on-drive system files / CON packages) surfaces an explicit `"STFS type 0 (read-write) not supported yet"` error rather than producing wrong output. - Block-index → byte-offset translator handles all interleaved hash levels: L0 every `0xAA` blocks, L1 every `0x70E4`, L2 every `0x4AF768`. Inline boundary tests pin offsets at `0xA9`, `0xAA`, `0x70E3`, `0x70E4` against literal hex values and assert strict monotonicity across boundaries. - File chain follower covers both the consecutive fast path (no hash-block reads) and the fragmented case (next-block pointer threaded through the L0 hash block at offset `(N % 0xAA) * 24 + 0x15`). Walk is capped at `used_blocks` iterations to reject malformed cyclic chains, mirroring the existing FAT cycle rejection in `volume.rs`. -- Defensive parent-chain resolution in `extract_to_host`: rejects cyclic `parent_index` references and out-of-range parent pointers; refuses to overwrite existing output files. +- Defensive parent-chain resolution: rejects cyclic `parent_index` references and out-of-range parent pointers; refuses to overwrite existing output files. + +### STFS extraction (TUI) +- The upload prompt (`u`) now sniffs every local file. On a CON / LIVE / PIRS package that contains a depth-0 `default.xex`, the prompt becomes `Detected STFS '' (Arcade). e(X)tract / (R)aw / Esc:`. Default action is `(X)tract`. +- `(X)tract` streams the package's inner files directly into a new subfolder under the current FATX directory, named from the STFS `display_name` (sanitised for FATX). No intermediate host-FS staging. +- Gating rule: the (X) option only appears when the package has a depth-0 `default.xex`. Title Updates (which contain `default.xexp` only) and streaming-DLC Marketplace packages (no executable) bypass the sniff and fall through to plain raw upload — they aren't useful as loose files on the drive. +- The CLI `extract-stfs` remains unrestricted; the TUI gating is a UX guard rail, not a library-level restriction. +- Mid-extraction `Esc` cancels cleanly. The library function honours an `Option<&AtomicBool>` cancel flag checked at the top of every entry iteration. ### Library API additions - New `fatxlib::stfs` submodules: `volume_descriptor`, `block_translator`, `file_entry`, `extract`. Existing header parsing (`StfsHeader`, `parse_header`, `MIN_HEADER_BYTES`) moved into `fatxlib::stfs::header` and re-exported at the namespace root — no breaking changes for existing callers. -- `fatxlib::stfs::StfsPackage::{open, header, volume, entries, read_block_chain, read_file}` — read API for STFS packages. `read_file` streams through a writer; no full-file buffering even for multi-hundred-MiB packages. -- `fatxlib::stfs::extract::extract_to_host(&mut StfsPackage, &Path, Option) -> Result` — top-level walk + write, with progress callback shape matching the existing XISO extract (`(rel_path, file_size, total_bytes_so_far)`). -- `fatxlib::stfs::extract::ExtractReport` — `{ files, directories, bytes }` returned on success. +- `fatxlib::stfs::StfsPackage::{open, header, volume, entries, read_block_chain, read_file, has_default_xex}` — read API for STFS packages. `read_file` streams through a writer; no full-file buffering even for multi-hundred-MiB packages. `has_default_xex` is the depth-0 root-only gating check used by the TUI sniff. +- `fatxlib::stfs::extract::extract_to_host(&mut StfsPackage, &Path, Option) -> Result` — top-level walk + write to a host filesystem, with progress callback shape matching the existing XISO extract (`(rel_path, file_size, total_bytes_so_far)`). +- `fatxlib::stfs::extract::extract_to_fatx(&mut StfsPackage, &mut FatxVolume, &str, Option, Option<&AtomicBool>) -> Result` — top-level walk + write directly into a FATX volume. Used by the TUI worker; available to library consumers. +- `fatxlib::stfs::extract::ExtractReport` — `{ files, directories, bytes }` returned on success. Unified semantics: `directories` counts STFS-package directory entries only (does not count an implicitly-created destination root). + +### Internal refactor +- `fatxlib/src/stfs/extract/` split from a single file into a directory module with four parts, mirroring the existing `fatxlib/src/iso/god/` pattern: `core.rs` (the `StfsSink` trait + shared `run_extract` engine), `sink_host.rs` (`HostSink` impl + `extract_to_host` wrapper), `sink_fatx.rs` (`FatxSink` impl + `extract_to_fatx` wrapper + the streaming `StfsFileReader` adapter), and `mod.rs` (`StfsPackage` + its tests). Eliminates the ~95% duplication that existed between the two `extract_to_*` functions and gives any future extract destination a single trait to implement. ### Testing / maintenance -- 27 inline synthetic tests across the four new STFS submodules: volume-descriptor parsing (type-0/1 detection, wrong-size rejection, truncation), type-1 block translator (boundary indices at every hash level plus monotonicity), file entry parsing (consecutive flag, directory flag, parent-index, non-ASCII tolerance), and end-to-end synthetic package extraction with a nested directory tree. -- v2 TUI extract-gating rule documented in the design spec: the future TUI sniff prompt will offer `(X)tract` only when the package contains a `default.xex` file (the only reliable signal that loose extraction produces something useful for alt-dashboards). Fallback: gate on `content_type == 0x000D0000`. The CLI `extract-stfs` is unrestricted. +- 32 inline synthetic tests across the STFS submodules: volume-descriptor parsing (type-0/1 detection, wrong-size rejection, truncation), type-1 block translator (boundary indices at every hash level plus monotonicity), file entry parsing (consecutive flag, directory flag, parent-index, non-ASCII tolerance), `has_default_xex` (positive, case-insensitive, subfolder-only rejection, TU `default.xexp` rejection), and end-to-end synthetic package extraction with a nested directory tree. +- New `fatxlib/tests/stfs_extract_to_fatx.rs` integration test exercises a round-trip through `extract_to_fatx` against a real FATX volume, including the cancel-flag path. +- TUI sniff helper (`stfs_extract_target`) covered by three new unit tests in `src/tui.rs`: non-STFS file rejected, STFS without `default.xex` rejected, STFS with `default.xex` returns the catalog-derived destination name. ## [1.2.1] - 2026-05-17 diff --git a/fatxlib/src/stfs/extract/core.rs b/fatxlib/src/stfs/extract/core.rs new file mode 100644 index 0000000..fb518ad --- /dev/null +++ b/fatxlib/src/stfs/extract/core.rs @@ -0,0 +1,150 @@ +//! `StfsSink` trait + `run_extract` engine. + +use std::io::{Read, Seek}; +use std::path::{Path, PathBuf}; + +use crate::error::{FatxError, Result}; +use crate::stfs::file_entry::StfsEntry; + +use super::{ExtractReport, ProgressFn, StfsPackage}; + +// ── StfsSink ──────────────────────────────────────────────────────────────── + +/// Destination abstraction used by [`run_extract`]. +/// +/// Implementors handle the sink-specific work (host filesystem vs FATX volume) +/// while the engine drives the common traversal loop. +pub(crate) trait StfsSink { + /// Create a directory at `rel` relative to the destination root. + /// Treats "already exists as directory" as success. + fn ensure_dir(&mut self, rel: &Path) -> Result<()>; + + /// Return an error if a file already exists at `rel`. + /// + /// Default: `Ok(())` — FATX's `create_file_from_reader` enforces this + /// internally so the FATX sink can rely on the default. + fn refuse_if_file_exists(&mut self, _rel: &Path) -> Result<()> { + Ok(()) + } + + /// Stream `size` bytes from `reader` into the destination at `rel`. + fn write_file(&mut self, rel: &Path, size: u64, reader: &mut dyn Read) -> Result<()>; + + /// Called once per entry before any work, enabling early cancellation. + /// + /// Default: `Ok(())` — the host sink has no cancel signal. + fn check_cancelled(&self) -> Result<()> { + Ok(()) + } +} + +// ── engine ────────────────────────────────────────────────────────────────── + +/// Walk every entry in `package` and stream it through `sink`. +/// +/// `directories` in the returned report counts only STFS package directory +/// entries — NOT the destination root created by the sink itself. +pub(crate) fn run_extract( + package: &mut StfsPackage, + sink: &mut S, + progress: Option>, +) -> Result +where + R: Read + Seek, + S: StfsSink, +{ + let entries = package.entries()?; + let paths = build_relative_paths(&entries)?; + + let mut files = 0usize; + let mut directories = 0usize; + let mut bytes = 0u64; + + for (idx, entry) in entries.iter().enumerate() { + sink.check_cancelled()?; + let rel = &paths[idx]; + + if entry.is_directory { + sink.ensure_dir(rel)?; + directories += 1; + continue; + } + + // Ensure the file's parent directory exists before writing. + if let Some(parent) = rel.parent() + && !parent.as_os_str().is_empty() + { + sink.ensure_dir(parent)?; + } + + sink.refuse_if_file_exists(rel)?; + + if let Some(cb) = progress { + cb(&rel.to_string_lossy(), entry.size, bytes); + } + + let mut reader = super::sink_fatx::StfsFileReader::new(package, entry)?; + sink.write_file(rel, entry.size, &mut reader)?; + files += 1; + bytes += entry.size; + } + + Ok(ExtractReport { + files, + directories, + bytes, + }) +} + +// ── build_relative_paths ──────────────────────────────────────────────────── + +/// Build a `Vec` parallel to `entries`, resolving `parent_index` +/// chains. Orphan entries (out-of-range parent) get a `` prefix. +pub(crate) fn build_relative_paths(entries: &[StfsEntry]) -> Result> { + let mut out: Vec> = vec![None; entries.len()]; + + fn resolve( + idx: usize, + entries: &[StfsEntry], + cache: &mut Vec>, + guard: &mut Vec, + ) -> Result { + if let Some(p) = &cache[idx] { + return Ok(p.clone()); + } + if guard[idx] { + return Err(FatxError::Other(format!( + "STFS entry parent chain cycles at index {}", + idx, + ))); + } + guard[idx] = true; + let entry = &entries[idx]; + let path = if entry.parent_index == -1 { + PathBuf::from(&entry.name) + } else { + let parent_idx = entry.parent_index as usize; + if parent_idx >= entries.len() { + return Err(FatxError::Other(format!( + "STFS entry {} references out-of-range parent {}", + idx, entry.parent_index, + ))); + } + let parent_path = resolve(parent_idx, entries, cache, guard)?; + parent_path.join(&entry.name) + }; + cache[idx] = Some(path.clone()); + Ok(path) + } + + let mut guard = vec![false; entries.len()]; + for i in 0..entries.len() { + resolve(i, entries, &mut out, &mut guard)?; + // Reset guard for the next traversal — the recursive helper marks + // visited nodes but does not reset them between top-level calls. + for slot in guard.iter_mut() { + *slot = false; + } + } + Ok(out.into_iter().map(|p| p.unwrap()).collect()) +} diff --git a/fatxlib/src/stfs/extract.rs b/fatxlib/src/stfs/extract/mod.rs similarity index 70% rename from fatxlib/src/stfs/extract.rs rename to fatxlib/src/stfs/extract/mod.rs index ad9d101..b4cd223 100644 --- a/fatxlib/src/stfs/extract.rs +++ b/fatxlib/src/stfs/extract/mod.rs @@ -1,7 +1,22 @@ //! High-level STFS read + extract API. +//! +//! # Module layout +//! +//! | File | Contents | +//! |---|---| +//! | `mod.rs` | Public types, `StfsPackage`, submodule declarations | +//! | `core.rs` | `StfsSink` trait, `run_extract` engine, `build_relative_paths` | +//! | `sink_host.rs` | `HostSink` + `extract_to_host` | +//! | `sink_fatx.rs` | `FatxSink` + `extract_to_fatx` + `StfsFileReader` | + +pub(crate) mod core; +pub(crate) mod sink_fatx; +pub(crate) mod sink_host; + +pub use sink_fatx::extract_to_fatx; +pub use sink_host::extract_to_host; use std::io::{Read, Seek, SeekFrom, Write}; -use std::path::{Path, PathBuf}; use crate::error::{FatxError, Result}; use crate::stfs::block_translator::{BLOCK_SIZE, BLOCKS_PER_L0, block_to_byte_offset}; @@ -9,6 +24,8 @@ use crate::stfs::file_entry::{self, ENTRIES_PER_BLOCK, FILE_ENTRY_SIZE, StfsEntr use crate::stfs::header::{MIN_HEADER_BYTES, StfsHeader, parse_header}; use crate::stfs::volume_descriptor::{self, VolumeDescriptor}; +// ── Public types ───────────────────────────────────────────────────────────── + /// Summary of a completed extraction. #[derive(Debug, Clone, PartialEq, Eq)] pub struct ExtractReport { @@ -20,107 +37,7 @@ pub struct ExtractReport { /// Progress callback: `(relative_path, file_size, total_bytes_so_far)`. pub type ProgressFn<'a> = &'a dyn Fn(&str, u64, u64); -/// Walk `package` and extract every file under `dest_root`. Creates -/// directories as needed. Returns counts on success. -/// -/// `progress` is invoked once per file just before its write begins. -pub fn extract_to_host( - package: &mut StfsPackage, - dest_root: &Path, - progress: Option>, -) -> Result { - let entries = package.entries()?; - let paths = build_relative_paths(&entries)?; - - let mut files = 0usize; - let mut directories = 0usize; - let mut bytes = 0u64; - - for (idx, entry) in entries.iter().enumerate() { - let rel = &paths[idx]; - let target = dest_root.join(rel); - if entry.is_directory { - std::fs::create_dir_all(&target).map_err(FatxError::Io)?; - directories += 1; - continue; - } - if let Some(parent) = target.parent() { - std::fs::create_dir_all(parent).map_err(FatxError::Io)?; - } - if target.exists() { - return Err(FatxError::Other(format!( - "refusing to overwrite existing file: {}", - target.display(), - ))); - } - if let Some(cb) = progress { - cb(&rel.to_string_lossy(), entry.size, bytes); - } - let file = std::fs::File::create(&target).map_err(FatxError::Io)?; - let mut writer = std::io::BufWriter::new(file); - package.read_file(entry, &mut writer)?; - writer.flush().map_err(FatxError::Io)?; - files += 1; - bytes += entry.size; - } - - Ok(ExtractReport { - files, - directories, - bytes, - }) -} - -/// Build a vec of relative paths parallel to `entries`. Walks -/// `parent_index` chains; tolerates orphan entries by attaching them to -/// the root with a `` prefix. -fn build_relative_paths(entries: &[StfsEntry]) -> Result> { - let mut out: Vec> = vec![None; entries.len()]; - - fn resolve( - idx: usize, - entries: &[StfsEntry], - cache: &mut Vec>, - guard: &mut Vec, - ) -> Result { - if let Some(p) = &cache[idx] { - return Ok(p.clone()); - } - if guard[idx] { - return Err(FatxError::Other(format!( - "STFS entry parent chain cycles at index {}", - idx, - ))); - } - guard[idx] = true; - let entry = &entries[idx]; - let path = if entry.parent_index == -1 { - PathBuf::from(&entry.name) - } else { - let parent_idx = entry.parent_index as usize; - if parent_idx >= entries.len() { - return Err(FatxError::Other(format!( - "STFS entry {} references out-of-range parent {}", - idx, entry.parent_index, - ))); - } - let parent_path = resolve(parent_idx, entries, cache, guard)?; - parent_path.join(&entry.name) - }; - cache[idx] = Some(path.clone()); - Ok(path) - } - - let mut guard = vec![false; entries.len()]; - for i in 0..entries.len() { - resolve(i, entries, &mut out, &mut guard)?; - // reset guard for next traversal - for slot in guard.iter_mut() { - *slot = false; - } - } - Ok(out.into_iter().map(|p| p.unwrap()).collect()) -} +// ── StfsPackage ────────────────────────────────────────────────────────────── pub struct StfsPackage { reader: R, @@ -172,7 +89,7 @@ impl StfsPackage { } /// Read one data block (4 KiB) into a fresh buffer. - fn read_data_block(&mut self, block_index: u32) -> Result> { + pub(crate) fn read_data_block(&mut self, block_index: u32) -> Result> { let total = self.volume.total_alloc_blocks; if block_index >= total { return Err(FatxError::Other(format!( @@ -192,15 +109,12 @@ impl StfsPackage { /// L0 hash block byte offset for the group containing `data_block`. fn l0_hash_block_offset(&self, data_block: u32) -> u64 { let group_start = (data_block / BLOCKS_PER_L0) * BLOCKS_PER_L0; - // The L0 hash block sits AFTER the group's data blocks. Equivalently - // it occupies the position one block before block_to_byte_offset - // would place data block (group_start + BLOCKS_PER_L0). + // The L0 hash block sits AFTER the group's data blocks. block_to_byte_offset(group_start + BLOCKS_PER_L0) - BLOCK_SIZE } /// Read the next-block pointer for `data_block` from its L0 hash entry. - /// Returns `0xFFFFFF` (sentinel end-of-chain) if the data block is the - /// last in its chain. + /// Returns `0xFFFFFF` (end-of-chain sentinel) for the last block. fn read_next_block(&mut self, data_block: u32) -> Result { let hash_offset = self.l0_hash_block_offset(data_block); let entry_offset = hash_offset + (data_block as u64 % BLOCKS_PER_L0 as u64) * 24 + 0x15; @@ -252,6 +166,17 @@ impl StfsPackage { Ok(entry.size) } + /// True if the package contains a depth-0 (root-level) file named + /// `default.xex` (case-insensitive). Used by the TUI to decide + /// whether an Arcade-style "extract loose to drive" prompt makes + /// sense. + pub fn has_default_xex(&mut self) -> Result { + let entries = self.entries()?; + Ok(entries.iter().any(|e| { + !e.is_directory && e.parent_index == -1 && e.name.eq_ignore_ascii_case("default.xex") + })) + } + /// Walk the file table block chain (v1: consecutive only). pub fn entries(&mut self) -> Result> { let count = self.volume.file_table_block_count as u32; @@ -272,15 +197,14 @@ impl StfsPackage { } } +// ── Tests ───────────────────────────────────────────────────────────────────── + #[cfg(test)] mod tests { use super::*; use std::io::Cursor; /// Build the minimum bytes needed for `StfsPackage::open` to succeed. - /// Magic + title_id at 0x360 + display_name + title_name (already - /// covered by parse_header's MIN_HEADER_BYTES) + volume descriptor at - /// 0x379 (which overlaps the header region — we just patch those bytes). fn synthetic_package(read_only_format: bool) -> Vec { let mut buf = vec![0u8; MIN_HEADER_BYTES]; // LIVE magic @@ -321,26 +245,22 @@ mod tests { } /// Build a more complete synthetic package: header + one file-table - /// block at block 0 containing two entries (one dir, one file). + /// block at block 0 containing entries. fn synthetic_package_with_one_file_table_block(entries: &[Vec]) -> Vec { use crate::stfs::block_translator::{BLOCK_SIZE, FIRST_DATA_BLOCK_OFFSET}; - // Header region: MIN_HEADER_BYTES. let mut buf = vec![0u8; MIN_HEADER_BYTES]; buf[0..4].copy_from_slice(b"LIVE"); buf[0x360..0x364].copy_from_slice(&0x4D5307E6u32.to_be_bytes()); buf[0x379] = 0x24; buf[0x37B] = 0x01; buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); - // file_table_block_number stays 0 buf[0x395..0x399].copy_from_slice(&2u32.to_be_bytes()); // total_alloc = 2 - // Grow to the start of block 0. let block_zero_offset = FIRST_DATA_BLOCK_OFFSET as usize; if buf.len() < block_zero_offset { buf.resize(block_zero_offset, 0); } - // File table block 0: pack entries. let mut ft_block = vec![0u8; BLOCK_SIZE as usize]; for (i, e) in entries.iter().enumerate() { ft_block[i * 0x40..(i + 1) * 0x40].copy_from_slice(e); @@ -359,7 +279,6 @@ mod tests { flags |= 0x80; } buf[0x28] = flags; - // used_blocks = 1 for files, 0 for dirs if !is_dir { buf[0x2C] = 1; } @@ -398,59 +317,37 @@ mod tests { assert_eq!(listed[0].name, "only.bin"); } - /// Build a synthetic package with one fragmented file whose blocks are - /// 5 → 3 → 1 (non-consecutive). Requires planting next-block pointers - /// inside the L0 hash block. fn synthetic_package_with_fragmented_file() -> Vec { use crate::stfs::block_translator::{BLOCK_SIZE, BLOCKS_PER_L0, FIRST_DATA_BLOCK_OFFSET}; - // File table at block 0; data blocks 1..=5. - // total_alloc_blocks = 6 (covers blocks 0..=5). let mut buf = vec![0u8; MIN_HEADER_BYTES]; buf[0..4].copy_from_slice(b"LIVE"); buf[0x360..0x364].copy_from_slice(&0x4D5307E6u32.to_be_bytes()); buf[0x379] = 0x24; buf[0x37B] = 0x01; buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); - buf[0x395..0x399].copy_from_slice(&6u32.to_be_bytes()); // total_alloc = 6 + buf[0x395..0x399].copy_from_slice(&6u32.to_be_bytes()); let block_zero_offset = FIRST_DATA_BLOCK_OFFSET as usize; buf.resize(block_zero_offset, 0); - // Block 0: file table with one fragmented file - // "frag.bin", consecutive=false, used_blocks=3, start_block=5 let mut ft_block = vec![0u8; BLOCK_SIZE as usize]; let mut entry = vec![0u8; 0x40]; let name = b"frag.bin"; entry[..name.len()].copy_from_slice(name); - // flags: name length 8, consecutive OFF, dir OFF entry[0x28] = name.len() as u8 & 0x3F; - entry[0x2C] = 3; // used_blocks - entry[0x2F] = 5; // start_block = 5 + entry[0x2C] = 3; + entry[0x2F] = 5; entry[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); - // file_size = 3 * BLOCK_SIZE = 0x3000 entry[0x34..0x38].copy_from_slice(&0x3000u32.to_be_bytes()); ft_block[..0x40].copy_from_slice(&entry); buf.extend_from_slice(&ft_block); - // Blocks 1..=5: each filled with a single byte equal to the block index. for b in 1u8..=5 { let block_data = vec![b; BLOCK_SIZE as usize]; buf.extend_from_slice(&block_data); } - // L0 hash block sits AT block_index = 0xAA per the type-1 layout — - // but our package has only 6 data blocks, so the hash block lives - // beyond the file proper. We need to plant next-block pointers - // somewhere the reader will look for them. - // - // For data block N, the next-block pointer lives in the L0 hash - // block at index (N / BLOCKS_PER_L0) * BLOCKS_PER_L0 + (BLOCKS_PER_L0 - 1) + 1 - // = the position after the group. With BLOCKS_PER_L0 = 0xAA and all - // our blocks in group 0, that's the hash block at block_to_byte_offset(0xAA) - BLOCK_SIZE. - // - // Compute that offset and pad the buffer to it, then plant the - // hash entries for blocks 5 → 3 → 1. let hash_block_offset = (FIRST_DATA_BLOCK_OFFSET + (BLOCKS_PER_L0 as u64) * BLOCK_SIZE) as usize; if buf.len() < hash_block_offset + BLOCK_SIZE as usize { @@ -464,7 +361,6 @@ mod tests { }; plant_next(&mut buf, 5, 3); plant_next(&mut buf, 3, 1); - // block 1 has no successor; plant 0xFFFFFF (end-of-chain) plant_next(&mut buf, 1, 0xFFFFFF); buf @@ -484,11 +380,9 @@ mod tests { #[test] fn read_chain_uses_fast_path_for_consecutive_files() { let entries = vec![fe("file.bin", false, -1, 3 * 0x1000, 1)]; - // Mark consecutive (fe sets it true) let bytes = synthetic_package_with_one_file_table_block(&entries); let mut pkg = StfsPackage::open(Cursor::new(bytes)).expect("open"); let listed = pkg.entries().expect("entries"); - // fe() sets used_blocks=1, override for this test let mut entry = listed[0].clone(); entry.used_blocks = 3; entry.consecutive = true; @@ -500,28 +394,21 @@ mod tests { fn read_chain_caps_at_used_blocks_to_reject_cycles() { use crate::stfs::block_translator::{BLOCK_SIZE, BLOCKS_PER_L0, FIRST_DATA_BLOCK_OFFSET}; - // Plant a cycle: 5 → 3 → 5 → 3 → ... - // Start from the unpatched bytes and rewrite the next-pointer for block 3. let mut raw = synthetic_package_with_fragmented_file(); let hash_block_offset = (FIRST_DATA_BLOCK_OFFSET + (BLOCKS_PER_L0 as u64) * BLOCK_SIZE) as usize; - // Rewrite next-pointer for block 3 to point back at block 5 let entry_off = hash_block_offset + 3 * 24 + 0x15; raw[entry_off] = 5; raw[entry_off + 1] = 0; raw[entry_off + 2] = 0; let mut pkg = StfsPackage::open(Cursor::new(raw)).expect("open"); let entry = pkg.entries().expect("entries")[0].clone(); - // used_blocks=3; walk should return exactly 3 blocks even though - // the chain loops 5 → 3 → 5 → 3 → ... let chain = pkg.read_block_chain(&entry).expect("chain"); assert_eq!(chain.len(), 3); } #[test] fn read_file_streams_consecutive_file_contents() { - // Build a package with one consecutive 2-block file containing - // [0xAA; BLOCK_SIZE] then [0xBB; BLOCK_SIZE - 1] (last block 1 byte short). use crate::stfs::block_translator::{BLOCK_SIZE, FIRST_DATA_BLOCK_OFFSET}; let mut buf = vec![0u8; MIN_HEADER_BYTES]; @@ -529,7 +416,7 @@ mod tests { buf[0x379] = 0x24; buf[0x37B] = 0x01; buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); - buf[0x395..0x399].copy_from_slice(&3u32.to_be_bytes()); // total_alloc = 3 + buf[0x395..0x399].copy_from_slice(&3u32.to_be_bytes()); let block_zero = FIRST_DATA_BLOCK_OFFSET as usize; buf.resize(block_zero, 0); @@ -537,17 +424,17 @@ mod tests { let mut ft = vec![0u8; BLOCK_SIZE as usize]; let name = b"data.bin"; ft[..name.len()].copy_from_slice(name); - ft[0x28] = (name.len() as u8 & 0x3F) | 0x40; // consecutive - ft[0x2C] = 2; // used_blocks - ft[0x2F] = 1; // start_block + ft[0x28] = (name.len() as u8 & 0x3F) | 0x40; + ft[0x2C] = 2; + ft[0x2F] = 1; ft[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); let file_size: u32 = (BLOCK_SIZE as u32) + (BLOCK_SIZE as u32) - 1; ft[0x34..0x38].copy_from_slice(&file_size.to_be_bytes()); buf.extend_from_slice(&ft); - buf.extend_from_slice(&[0xAA; 0x1000]); // block 1 + buf.extend_from_slice(&[0xAA; 0x1000]); let mut last_block = vec![0xBBu8; 0x1000]; - last_block[BLOCK_SIZE as usize - 1] = 0; // last byte unused + last_block[BLOCK_SIZE as usize - 1] = 0; buf.extend_from_slice(&last_block); let mut pkg = StfsPackage::open(Cursor::new(buf)).expect("open"); @@ -565,8 +452,6 @@ mod tests { #[test] fn extract_to_host_writes_nested_tree() { - // Build: /Media/ -> /Media/cover.png (4 bytes "ABCD") - // /default.xex (4 bytes "MZRX") use crate::stfs::block_translator::{BLOCK_SIZE, FIRST_DATA_BLOCK_OFFSET}; let mut buf = vec![0u8; MIN_HEADER_BYTES]; @@ -574,28 +459,25 @@ mod tests { buf[0x379] = 0x24; buf[0x37B] = 0x01; buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); - buf[0x395..0x399].copy_from_slice(&3u32.to_be_bytes()); // total_alloc = 3 + buf[0x395..0x399].copy_from_slice(&3u32.to_be_bytes()); let block_zero = FIRST_DATA_BLOCK_OFFSET as usize; buf.resize(block_zero, 0); let mut ft = vec![0u8; BLOCK_SIZE as usize]; - // Entry 0: dir "Media", parent -1 (root) let mut e0 = vec![0u8; 0x40]; e0[..5].copy_from_slice(b"Media"); - e0[0x28] = 0x05 | 0x40 | 0x80; // dir + consecutive (irrelevant for dir) + e0[0x28] = 0x05 | 0x40 | 0x80; e0[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); ft[..0x40].copy_from_slice(&e0); - // Entry 1: file "cover.png", parent 0 (Media), start_block 1 let mut e1 = vec![0u8; 0x40]; e1[..9].copy_from_slice(b"cover.png"); e1[0x28] = 0x09 | 0x40; - e1[0x2C] = 1; // used_blocks - e1[0x2F] = 1; // start_block + e1[0x2C] = 1; + e1[0x2F] = 1; e1[0x32..0x34].copy_from_slice(&0i16.to_be_bytes()); - e1[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); // size + e1[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); ft[0x40..0x80].copy_from_slice(&e1); - // Entry 2: file "default.xex", parent -1 (root), start_block 2 let mut e2 = vec![0u8; 0x40]; e2[..11].copy_from_slice(b"default.xex"); e2[0x28] = 0x0B | 0x40; @@ -606,11 +488,9 @@ mod tests { ft[0x80..0xC0].copy_from_slice(&e2); buf.extend_from_slice(&ft); - // Block 1: cover.png contents let mut b1 = vec![0u8; BLOCK_SIZE as usize]; b1[..4].copy_from_slice(b"ABCD"); buf.extend_from_slice(&b1); - // Block 2: default.xex contents let mut b2 = vec![0u8; BLOCK_SIZE as usize]; b2[..4].copy_from_slice(b"MZRX"); buf.extend_from_slice(&b2); @@ -627,4 +507,42 @@ mod tests { let xex = std::fs::read(tmp.path().join("default.xex")).expect("read xex"); assert_eq!(xex, b"MZRX"); } + + #[test] + fn has_default_xex_true_when_root_default_xex_present() { + let entries = vec![ + fe("Media", true, -1, 0, 0), + fe("default.xex", false, -1, 0x100, 1), + ]; + let bytes = synthetic_package_with_one_file_table_block(&entries); + let mut pkg = StfsPackage::open(Cursor::new(bytes)).expect("open"); + assert!(pkg.has_default_xex().expect("walk")); + } + + #[test] + fn has_default_xex_case_insensitive() { + let entries = vec![fe("DEFAULT.XEX", false, -1, 0x100, 1)]; + let bytes = synthetic_package_with_one_file_table_block(&entries); + let mut pkg = StfsPackage::open(Cursor::new(bytes)).expect("open"); + assert!(pkg.has_default_xex().expect("walk")); + } + + #[test] + fn has_default_xex_false_when_only_in_subfolder() { + let entries = vec![ + fe("Media", true, -1, 0, 0), + fe("default.xex", false, 0, 0x100, 1), + ]; + let bytes = synthetic_package_with_one_file_table_block(&entries); + let mut pkg = StfsPackage::open(Cursor::new(bytes)).expect("open"); + assert!(!pkg.has_default_xex().expect("walk")); + } + + #[test] + fn has_default_xex_false_for_title_update_with_xexp_only() { + let entries = vec![fe("default.xexp", false, -1, 0x100, 1)]; + let bytes = synthetic_package_with_one_file_table_block(&entries); + let mut pkg = StfsPackage::open(Cursor::new(bytes)).expect("open"); + assert!(!pkg.has_default_xex().expect("walk")); + } } diff --git a/fatxlib/src/stfs/extract/sink_fatx.rs b/fatxlib/src/stfs/extract/sink_fatx.rs new file mode 100644 index 0000000..695c87f --- /dev/null +++ b/fatxlib/src/stfs/extract/sink_fatx.rs @@ -0,0 +1,184 @@ +//! FATX-volume sink for STFS extraction + `StfsFileReader`. + +use std::io::{Read, Seek, Write}; +use std::path::Path; +use std::sync::atomic::{AtomicBool, Ordering}; + +use crate::error::{FatxError, Result}; +use crate::stfs::block_translator::BLOCK_SIZE; +use crate::stfs::file_entry::StfsEntry; +use crate::volume::FatxVolume; + +use super::core::{StfsSink, run_extract}; +use super::{ExtractReport, ProgressFn, StfsPackage}; + +// ── FatxSink ──────────────────────────────────────────────────────────────── + +pub(crate) struct FatxSink<'a, S: Read + Write + Seek> { + vol: &'a mut FatxVolume, + root: String, + cancel: Option<&'a AtomicBool>, +} + +impl<'a, S: Read + Write + Seek> FatxSink<'a, S> { + pub(crate) fn new( + vol: &'a mut FatxVolume, + dest_root: &str, + cancel: Option<&'a AtomicBool>, + ) -> Result { + let root = dest_root.trim_end_matches('/').to_string(); + ensure_fatx_dir_chain(vol, &root)?; + Ok(Self { vol, root, cancel }) + } + + /// Convert a relative `Path` (from the engine) to an absolute FATX path. + fn fatx_path(&self, rel: &Path) -> String { + let rel_str = rel.to_string_lossy(); + let rel_fwd = rel_str.replace('\\', "/"); + format!("{}/{}", self.root, rel_fwd) + } +} + +impl StfsSink for FatxSink<'_, S> { + fn ensure_dir(&mut self, rel: &Path) -> Result<()> { + ensure_fatx_dir_chain(self.vol, &self.fatx_path(rel)) + } + + fn write_file(&mut self, rel: &Path, size: u64, reader: &mut dyn Read) -> Result<()> { + let path = self.fatx_path(rel); + self.vol.create_file_from_reader(&path, size, reader, None) + } + + fn check_cancelled(&self) -> Result<()> { + if let Some(flag) = self.cancel + && flag.load(Ordering::Relaxed) + { + return Err(FatxError::Other("cancelled".to_string())); + } + Ok(()) + } +} + +// ── public wrapper ─────────────────────────────────────────────────────────── + +/// Walk `package` and stream every file into `dest_root` on the given FATX +/// volume. Creates directories as needed; refuses to overwrite existing +/// files. Returns counts on success. +/// +/// `progress` is invoked once per file just before its write begins: +/// `(relative_path, file_size, total_bytes_so_far)`. +pub fn extract_to_fatx( + package: &mut StfsPackage, + vol: &mut FatxVolume, + dest_root: &str, + progress: Option>, + cancel: Option<&AtomicBool>, +) -> Result +where + R: Read + Seek, + S: Read + Write + Seek, +{ + let mut sink = FatxSink::new(vol, dest_root, cancel)?; + run_extract(package, &mut sink, progress) +} + +// ── StfsFileReader ─────────────────────────────────────────────────────────── + +/// Adapter that exposes a single STFS file entry as a streaming [`Read`]. +/// +/// `create_file_from_reader` (and the engine's `sink.write_file`) require +/// `Read`. We pre-walk the block chain once and read blocks on demand to avoid +/// buffering entire files in memory. +pub(crate) struct StfsFileReader<'a, R: Read + Seek> { + package: &'a mut StfsPackage, + chain: Vec, + chain_idx: usize, + block_buf: Vec, + block_pos: usize, + block_len: usize, + remaining: u64, +} + +impl<'a, R: Read + Seek> StfsFileReader<'a, R> { + pub(crate) fn new(package: &'a mut StfsPackage, entry: &StfsEntry) -> Result { + let chain = package.read_block_chain(entry)?; + Ok(Self { + package, + chain, + chain_idx: 0, + block_buf: Vec::new(), + block_pos: 0, + block_len: 0, + remaining: entry.size, + }) + } +} + +impl Read for StfsFileReader<'_, R> { + fn read(&mut self, buf: &mut [u8]) -> std::io::Result { + if self.remaining == 0 { + return Ok(0); + } + // Refill block buffer when exhausted. + if self.block_pos == self.block_len { + if self.chain_idx == self.chain.len() { + return Ok(0); + } + let block_idx = self.chain[self.chain_idx]; + self.chain_idx += 1; + self.block_buf = self + .package + .read_data_block(block_idx) + .map_err(|e| std::io::Error::other(format!("STFS block read: {e}")))?; + let take = self.remaining.min(BLOCK_SIZE) as usize; + self.block_len = take; + self.block_pos = 0; + } + let want = buf.len().min(self.block_len - self.block_pos); + buf[..want].copy_from_slice(&self.block_buf[self.block_pos..self.block_pos + want]); + self.block_pos += want; + self.remaining -= want as u64; + Ok(want) + } +} + +// ── ensure_fatx_dir_chain ──────────────────────────────────────────────────── + +/// Create the FATX directory `path` and every missing ancestor. +pub(crate) fn ensure_fatx_dir_chain( + vol: &mut FatxVolume, + path: &str, +) -> Result<()> { + let trimmed = path.trim_start_matches('/').trim_end_matches('/'); + if trimmed.is_empty() { + return Ok(()); + } + let mut current = String::new(); + for part in trimmed.split('/') { + if part.is_empty() { + continue; + } + current.push('/'); + current.push_str(part); + match vol.create_directory(¤t) { + Ok(()) => {} + Err(FatxError::FileExists(_)) => { + // Tolerate FileExists only if the existing entry is actually a + // directory. A stale file would cause a confusing NotADirectory + // error later inside create_file_from_reader. + match vol.resolve_path(¤t) { + Ok(entry) if entry.is_directory() => {} + Ok(_) => { + return Err(FatxError::Other(format!( + "STFS extract: '{}' exists but is not a directory", + current + ))); + } + Err(e) => return Err(e), + } + } + Err(e) => return Err(e), + } + } + Ok(()) +} diff --git a/fatxlib/src/stfs/extract/sink_host.rs b/fatxlib/src/stfs/extract/sink_host.rs new file mode 100644 index 0000000..e13e33a --- /dev/null +++ b/fatxlib/src/stfs/extract/sink_host.rs @@ -0,0 +1,64 @@ +//! Host-filesystem sink for STFS extraction. + +use std::io::{BufWriter, Read, Seek, Write}; +use std::path::Path; + +use crate::error::{FatxError, Result}; + +use super::core::{StfsSink, run_extract}; +use super::{ExtractReport, ProgressFn, StfsPackage}; + +// ── HostSink ───────────────────────────────────────────────────────────────── + +pub(crate) struct HostSink<'a> { + dest_root: &'a Path, +} + +impl<'a> HostSink<'a> { + pub(crate) fn new(dest_root: &'a Path) -> Result { + std::fs::create_dir_all(dest_root).map_err(FatxError::Io)?; + Ok(Self { dest_root }) + } +} + +impl StfsSink for HostSink<'_> { + fn ensure_dir(&mut self, rel: &Path) -> Result<()> { + let target = self.dest_root.join(rel); + std::fs::create_dir_all(&target).map_err(FatxError::Io) + } + + fn refuse_if_file_exists(&mut self, rel: &Path) -> Result<()> { + let target = self.dest_root.join(rel); + if target.exists() { + return Err(FatxError::Other(format!( + "refusing to overwrite existing file: {}", + target.display(), + ))); + } + Ok(()) + } + + fn write_file(&mut self, rel: &Path, _size: u64, reader: &mut dyn Read) -> Result<()> { + let target = self.dest_root.join(rel); + let file = std::fs::File::create(&target).map_err(FatxError::Io)?; + let mut writer = BufWriter::new(file); + std::io::copy(reader, &mut writer).map_err(FatxError::Io)?; + writer.flush().map_err(FatxError::Io)?; + Ok(()) + } +} + +// ── public wrapper ──────────────────────────────────────────────────────────── + +/// Walk `package` and extract every file under `dest_root`. Creates +/// directories as needed. Returns counts on success. +/// +/// `progress` is invoked once per file just before its write begins. +pub fn extract_to_host( + package: &mut StfsPackage, + dest_root: &Path, + progress: Option>, +) -> Result { + let mut sink = HostSink::new(dest_root)?; + run_extract(package, &mut sink, progress) +} diff --git a/fatxlib/tests/stfs_extract_to_fatx.rs b/fatxlib/tests/stfs_extract_to_fatx.rs new file mode 100644 index 0000000..1aaabb2 --- /dev/null +++ b/fatxlib/tests/stfs_extract_to_fatx.rs @@ -0,0 +1,124 @@ +//! Integration test for `extract_to_fatx` — streams STFS package contents +//! into a FATX volume in memory and verifies the resulting tree byte-for-byte. + +mod common; + +use std::io::Cursor; + +use fatxlib::stfs::StfsPackage; +use fatxlib::stfs::block_translator::{BLOCK_SIZE, FIRST_DATA_BLOCK_OFFSET}; +use fatxlib::stfs::extract::{ExtractReport, extract_to_fatx}; +use fatxlib::stfs::header::MIN_HEADER_BYTES; + +/// Build a synthetic STFS package containing: +/// /Media/cover.png — 4 bytes "ABCD" (block 1) +/// /default.xex — 4 bytes "MZRX" (block 2) +/// +/// File-table block is at block 0; total_alloc_blocks = 3. +fn make_two_file_package() -> Vec { + let mut buf = vec![0u8; MIN_HEADER_BYTES]; + buf[0..4].copy_from_slice(b"LIVE"); + // Volume descriptor at 0x379 + buf[0x379] = 0x24; // descriptor_size + buf[0x37B] = 0x01; // read_only_format + buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); // file_table_block_count = 1 + // file_table_block_number = 0 (already zero) + buf[0x395..0x399].copy_from_slice(&3u32.to_be_bytes()); // total_alloc_blocks = 3 + + // Grow to the start of block 0. + let block_zero = FIRST_DATA_BLOCK_OFFSET as usize; + buf.resize(block_zero, 0); + + // Block 0: file table + let mut ft = vec![0u8; BLOCK_SIZE as usize]; + + // Entry 0: dir "Media", parent -1 (root), consecutive + dir flags + let mut e0 = vec![0u8; 0x40]; + e0[..5].copy_from_slice(b"Media"); + e0[0x28] = 0x05 | 0x40 | 0x80; // name_len=5 | consecutive | dir + e0[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); // parent = root + ft[..0x40].copy_from_slice(&e0); + + // Entry 1: file "cover.png", parent 0 (Media), block 1, size 4 + let mut e1 = vec![0u8; 0x40]; + e1[..9].copy_from_slice(b"cover.png"); + e1[0x28] = 0x09 | 0x40; // name_len=9 | consecutive + e1[0x2C] = 1; // used_blocks + e1[0x2F] = 1; // start_block + e1[0x32..0x34].copy_from_slice(&0i16.to_be_bytes()); // parent = entry 0 + e1[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); // size + ft[0x40..0x80].copy_from_slice(&e1); + + // Entry 2: file "default.xex", parent -1 (root), block 2, size 4 + let mut e2 = vec![0u8; 0x40]; + e2[..11].copy_from_slice(b"default.xex"); + e2[0x28] = 0x0B | 0x40; // name_len=11 | consecutive + e2[0x2C] = 1; + e2[0x2F] = 2; // start_block + e2[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); // parent = root + e2[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); + ft[0x80..0xC0].copy_from_slice(&e2); + + buf.extend_from_slice(&ft); + + // Block 1: cover.png payload + let mut b1 = vec![0u8; BLOCK_SIZE as usize]; + b1[..4].copy_from_slice(b"ABCD"); + buf.extend_from_slice(&b1); + + // Block 2: default.xex payload + let mut b2 = vec![0u8; BLOCK_SIZE as usize]; + b2[..4].copy_from_slice(b"MZRX"); + buf.extend_from_slice(&b2); + + buf +} + +#[test] +fn extract_to_fatx_writes_nested_tree() { + // Spin up a fresh FATX volume backed by a temp file. + let (_tmp, mut vol) = common::create_fatx_image(4); + + let raw = make_two_file_package(); + let mut pkg = StfsPackage::open(Cursor::new(raw)).expect("open stfs package"); + + let report: ExtractReport = + extract_to_fatx(&mut pkg, &mut vol, "/Halo Wars", None, None).expect("extract to fatx"); + + // Two files extracted. directories counts only STFS package dir entries + // (just /Media here); the dest_root (/Halo Wars) is NOT counted, + // matching the unified host-semantics behaviour. + assert_eq!(report.files, 2, "files"); + assert_eq!(report.directories, 1, "directories"); + assert_eq!(report.bytes, 8, "bytes"); + + let cover = vol + .read_file_by_path("/Halo Wars/Media/cover.png") + .expect("read cover.png"); + assert_eq!(cover, b"ABCD", "cover.png content"); + + let xex = vol + .read_file_by_path("/Halo Wars/default.xex") + .expect("read default.xex"); + assert_eq!(xex, b"MZRX", "default.xex content"); +} + +#[test] +fn extract_to_fatx_honors_cancel_flag() { + use std::sync::atomic::AtomicBool; + + let (_tmp, mut vol) = common::create_fatx_image(4); + + let raw = make_two_file_package(); + let mut pkg = StfsPackage::open(Cursor::new(raw)).expect("open stfs package"); + + // Pre-cancelled flag — should abort before writing any file. + let cancel = AtomicBool::new(true); + let err = extract_to_fatx(&mut pkg, &mut vol, "/cancelled", None, Some(&cancel)) + .expect_err("should cancel"); + assert!( + format!("{}", err).contains("cancelled"), + "error message should contain 'cancelled', got: {}", + err + ); +} diff --git a/src/tui.rs b/src/tui.rs index 5ad9b73..e2bc9b6 100644 --- a/src/tui.rs +++ b/src/tui.rs @@ -63,6 +63,7 @@ use ratatui::{ use fatxlib::iso::image::XisoImage; use fatxlib::partition::format_size; +use fatxlib::stfs::StfsPackage; use fatxlib::types::FileAttributes; use fatxlib::volume::FatxVolume; @@ -129,6 +130,18 @@ enum XisoUploadAction { Raw, } +/// What to do with a local STFS package that's being uploaded. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum StfsUploadAction { + /// Walk the package and stream each file into `//`. + /// Only offered when `default.xex` is present at the package root — + /// the only reliable signal that loose extraction produces something + /// alt-dashboards can launch. + Extract, + /// Copy the source bytes byte-for-byte to `/`. + Raw, +} + /// How to order the directory listing. #[derive(Debug, Clone, Copy, PartialEq, Eq)] enum SortMode { @@ -167,6 +180,13 @@ enum IoCmd { source: PathBuf, dest_dir: String, }, + /// Open `source` as an STFS package and stream every inner file into + /// `` on the FATX volume. The dest directory is created by + /// the worker; it must not already exist. + ExtractStfsToFatx { + source: PathBuf, + dest_dir: String, + }, /// Convert `source` (an XDVDFS image) to a Games-on-Demand package /// rooted at `dest_dir` on the FATX volume. Writes /// `//00007000/{,.data/Data0000..N}`. @@ -260,6 +280,10 @@ enum InputMode { /// `r` falls back to a raw byte copy of the source file. /// The default action on bare Enter depends on cwd context. ConfirmXisoUpload, + /// Two-way prompt after detecting an extractable STFS package during + /// upload: `x` extracts the contents into `//`, + /// `r` falls back to a raw byte copy of the source file. + ConfirmStfsUpload, } struct App { @@ -286,6 +310,9 @@ struct App { /// Local XISO path + default action stashed between the upload prompt /// and the three-way confirmation prompt (extract / GoD / raw). pending_xiso_upload: Option<(PathBuf, XisoUploadAction)>, + /// Local STFS path + destination subfolder name stashed between the + /// upload prompt and the extract/raw confirmation prompt. + pending_stfs_upload: Option<(PathBuf, StfsExtractTarget)>, /// Current listing sort order. Toggleable with `s`. sort_mode: SortMode, } @@ -318,6 +345,48 @@ fn is_xiso(path: &std::path::Path) -> bool { } } +/// Information needed to extract an STFS package to the FATX volume. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct StfsExtractTarget { + /// Sanitised subfolder name to create under cwd. Derived from the + /// package's STFS `display_name` (falling back to `title_name`, then + /// the local filename stem). + pub dest_name: String, +} + +/// Returns `Some(target)` if `path` is a type-1 STFS package containing a +/// depth-0 `default.xex`. The sniff opens the package, validates the +/// header, walks the file table, and closes the file before returning. +/// +/// Returns `None` for: not an STFS file, type-0 (read-write) packages, +/// truncated/corrupt headers, packages without a root `default.xex`, +/// and any I/O error during the walk. Errors are swallowed deliberately — +/// the sniff is best-effort; if anything goes wrong the user falls +/// through to the raw-upload path. +fn stfs_extract_target(path: &std::path::Path) -> Option { + let file = std::fs::File::open(path).ok()?; + let mut pkg = StfsPackage::open(file).ok()?; + if !pkg.has_default_xex().ok()? { + return None; + } + let display = pkg.header().display_name.trim().to_string(); + let title = pkg.header().title_name.trim().to_string(); + let raw = if !display.is_empty() { + display + } else if !title.is_empty() { + title + } else { + path.file_stem()?.to_string_lossy().into_owned() + }; + let sanitized = sanitize_fatx_filename(&raw); + if sanitized.is_empty() { + return None; + } + Some(StfsExtractTarget { + dest_name: sanitized, + }) +} + /// Resolve the destination folder name for an XISO extract by reading the /// embedded `Default.xex` / `default.xbe` and looking the TitleID up in /// [`fatxlib::titles`]. Returns the catalog-known game name, sanitized for @@ -394,6 +463,7 @@ impl App { pending_cleanup: Vec::new(), pending_delete: None, pending_xiso_upload: None, + pending_stfs_upload: None, sort_mode: SortMode::ByName, } } @@ -1178,6 +1248,99 @@ fn io_worker( let _ = resp_tx.send(resp); } + IoCmd::ExtractStfsToFatx { source, dest_dir } => { + cancel_flag.store(false, Ordering::Relaxed); + + let display_source = source + .file_name() + .map(|n| n.to_string_lossy().to_string()) + .unwrap_or_else(|| source.display().to_string()); + + let file = match fs::File::open(&source) { + Ok(f) => f, + Err(e) => { + let _ = resp_tx.send(IoResp::Error { + message: format!("Open {}: {}", source.display(), e), + }); + continue; + } + }; + let mut pkg = match fatxlib::stfs::StfsPackage::open(file) { + Ok(p) => p, + Err(e) => { + let _ = resp_tx.send(IoResp::Error { + message: format!("Parse {}: {}", source.display(), e), + }); + continue; + } + }; + + // Compute total bytes (for progress denominator). If the + // entries() walk fails here, fall back to 0 — the actual + // extraction will surface the same error properly. + let bytes_total: u64 = match pkg.entries() { + Ok(es) => es.iter().filter(|e| !e.is_directory).map(|e| e.size).sum(), + Err(_) => 0, + }; + + // Throttled progress: send IoResp::Progress at most every 200ms. + let last_progress = std::cell::Cell::new(std::time::Instant::now()); + let resp_tx_for_cb = resp_tx.clone(); + let cancel_for_cb = Arc::clone(&cancel_flag); + let cb = move |rel: &str, _size: u64, bytes_done: u64| { + if cancel_for_cb.load(Ordering::Relaxed) { + return; + } + if last_progress.get().elapsed().as_millis() > 200 { + let _ = resp_tx_for_cb.send(IoResp::Progress { + message: format!( + "{} ({}/{})", + rel, + format_size(bytes_done), + format_size(bytes_total), + ), + }); + last_progress.set(std::time::Instant::now()); + } + }; + + match fatxlib::stfs::extract::extract_to_fatx( + &mut pkg, + &mut vol, + &dest_dir, + Some(&cb), + Some(&cancel_flag), + ) { + Ok(report) => { + if flush_or_error(&mut vol, &resp_tx, "STFS extract flush failed") { + continue; + } + let _ = resp_tx.send(IoResp::Done { + message: format!( + "Extracted {} → {} ({} files, {})", + display_source, + dest_dir, + report.files, + format_size(report.bytes), + ), + }); + } + Err(e) => { + let _ = flush_or_error(&mut vol, &resp_tx, "STFS extract flush failed"); + let msg = format!("{}", e); + if msg.contains("cancelled") { + let _ = resp_tx.send(IoResp::Cancelled { + message: format!("STFS extract cancelled: {}", display_source), + }); + } else { + let _ = resp_tx.send(IoResp::Error { + message: format!("Extract {}: {}", source.display(), msg), + }); + } + } + } + } + IoCmd::Flush => { if flush_or_error(&mut vol, &resp_tx, "Flush failed") { continue; @@ -1703,6 +1866,7 @@ fn handle_input_key(app: &mut App, cmd_tx: &mpsc::Sender, key: KeyEvent) // If the user was answering the XISO extract/raw prompt, drop the // stashed path so the next upload starts clean. app.pending_xiso_upload = None; + app.pending_stfs_upload = None; app.pending_delete = None; app.set_status("Cancelled."); } @@ -1784,6 +1948,20 @@ fn handle_input_key(app: &mut App, cmd_tx: &mpsc::Sender, key: KeyEvent) app.input_mode = InputMode::ConfirmXisoUpload; app.input_prompt = prompt; app.input_buffer.clear(); + } else if let Some(target) = stfs_extract_target(&path) { + // ── New STFS two-way prompt ── + let prompt = format!( + "Detected STFS '{}' (Arcade). e(X)tract / (R)aw / Esc:", + path.file_name() + .map(|n| n.to_string_lossy().to_string()) + .unwrap_or_else(|| "package".to_string()), + ); + app.pending_stfs_upload = Some((path.clone(), target)); + app.download_dir = + path.parent().unwrap_or(&PathBuf::from(".")).to_path_buf(); + app.input_mode = InputMode::ConfirmStfsUpload; + app.input_prompt = prompt; + app.input_buffer.clear(); } else { let fatx_path = app.full_path(&filename); app.set_status(&format!("Uploading '{}'...", filename)); @@ -1930,6 +2108,59 @@ fn handle_input_key(app: &mut App, cmd_tx: &mpsc::Sender, key: KeyEvent) app.set_status("Cleanup cancelled."); } } + InputMode::ConfirmStfsUpload => { + let (path, target) = match app.pending_stfs_upload.take() { + Some(pair) => pair, + None => { + app.set_error("Internal: missing pending STFS path."); + return; + } + }; + let trimmed = input.trim(); + let action = if trimmed.is_empty() { + StfsUploadAction::Extract + } else { + match trimmed.chars().next().map(|c| c.to_ascii_lowercase()) { + Some('x') => StfsUploadAction::Extract, + Some('r') => StfsUploadAction::Raw, + _ => { + app.set_error(&format!( + "Unknown choice {:?} — expected x or r.", + trimmed + )); + return; + } + } + }; + let filename = path + .file_name() + .map(|n| n.to_string_lossy().to_string()) + .unwrap_or_else(|| "stfs".to_string()); + + match action { + StfsUploadAction::Extract => { + let dest_dir = app.full_path(&target.dest_name); + app.set_status(&format!( + "Extracting STFS '{}' → {}...", + filename, dest_dir + )); + let _ = cmd_tx.send(IoCmd::ExtractStfsToFatx { + source: path, + dest_dir, + }); + app.is_busy = true; + } + StfsUploadAction::Raw => { + let fatx_path = app.full_path(&filename); + app.set_status(&format!("Uploading '{}' (raw)...", filename)); + let _ = cmd_tx.send(IoCmd::WriteFile { + local_path: path, + fatx_path, + }); + app.is_busy = true; + } + } + } InputMode::Normal => {} } } @@ -1939,7 +2170,10 @@ fn handle_input_key(app: &mut App, cmd_tx: &mpsc::Sender, key: KeyEvent) KeyCode::Char(c) => { if matches!( app.input_mode, - InputMode::ConfirmDelete | InputMode::ConfirmCleanup | InputMode::ConfirmXisoUpload + InputMode::ConfirmDelete + | InputMode::ConfirmCleanup + | InputMode::ConfirmXisoUpload + | InputMode::ConfirmStfsUpload ) { app.input_buffer = c.to_string(); } else { @@ -2232,4 +2466,77 @@ mod tests { "MyGame$SystemUpdate/foo" )); } + + #[test] + fn stfs_extract_target_returns_none_for_non_stfs() { + let tmp = tempfile::NamedTempFile::new().unwrap(); + std::fs::write(tmp.path(), b"not an stfs package\x00\x00").unwrap(); + assert!(stfs_extract_target(tmp.path()).is_none()); + } + + #[test] + fn stfs_extract_target_returns_none_for_stfs_without_default_xex() { + // Build a minimal STFS package with no default.xex at root. + let bytes = make_synthetic_stfs_no_default_xex(); + let tmp = tempfile::NamedTempFile::new().unwrap(); + std::fs::write(tmp.path(), &bytes).unwrap(); + assert!(stfs_extract_target(tmp.path()).is_none()); + } + + #[test] + fn stfs_extract_target_returns_dest_name_for_arcade_package() { + let bytes = make_synthetic_stfs_with_default_xex("My Test Game"); + let tmp = tempfile::NamedTempFile::new().unwrap(); + std::fs::write(tmp.path(), &bytes).unwrap(); + let target = stfs_extract_target(tmp.path()).expect("should detect"); + // Destination name comes from display_name, sanitized for FATX. + assert_eq!(target.dest_name, "My Test Game"); + } + + fn make_synthetic_stfs_no_default_xex() -> Vec { + // 32 KiB of zeros except the LIVE magic + a valid volume descriptor + // + a file-table block with one non-default.xex file. + let mut buf = vec![0u8; 0x1_0000]; + buf[0..4].copy_from_slice(b"LIVE"); + buf[0x379] = 0x24; + buf[0x37B] = 0x01; + buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); + buf[0x395..0x399].copy_from_slice(&2u32.to_be_bytes()); + // File table at block 0 starts at 0xC000 + let mut e = vec![0u8; 0x40]; + e[..9].copy_from_slice(b"other.bin"); + e[0x28] = 0x09 | 0x40; // length=9, consecutive + e[0x2C] = 1; // used_blocks + e[0x2F] = 1; // start_block = 1 + e[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); + e[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); + buf[0xC000..0xC000 + 0x40].copy_from_slice(&e); + buf + } + + fn make_synthetic_stfs_with_default_xex(display_name: &str) -> Vec { + let mut buf = vec![0u8; 0x1_0000]; + buf[0..4].copy_from_slice(b"LIVE"); + buf[0x379] = 0x24; + buf[0x37B] = 0x01; + buf[0x37C..0x37E].copy_from_slice(&1u16.to_be_bytes()); + buf[0x395..0x399].copy_from_slice(&2u32.to_be_bytes()); + + // Write display_name into the locale-0 UTF-16BE slot at 0x411. + for (i, c) in display_name.encode_utf16().enumerate() { + let off = 0x411 + i * 2; + buf[off..off + 2].copy_from_slice(&c.to_be_bytes()); + } + + // File-table block at 0xC000: one entry, default.xex, root. + let mut e = vec![0u8; 0x40]; + e[..11].copy_from_slice(b"default.xex"); + e[0x28] = 0x0B | 0x40; + e[0x2C] = 1; + e[0x2F] = 1; + e[0x32..0x34].copy_from_slice(&(-1i16).to_be_bytes()); + e[0x34..0x38].copy_from_slice(&4u32.to_be_bytes()); + buf[0xC000..0xC000 + 0x40].copy_from_slice(&e); + buf + } }