audit_epub.py gains a fourth analyzer: ocr, flagging OCR/conversion-damaged prose. The defect class none of the existing three catches: a bad conversion splits paragraphs mid-sentence at line-wrap or page-break positions, so the text reflows broken ("could just make out the shape" / "of another boat"). The motivating case was a damaged Jingo EPUB that measured 80 such splits where a clean edition of the same text measured 0. The primary signal is exactly that split shape (a paragraph ending without terminal punctuation, the next starting lowercase, paired only within one spine document), reported as a per-book rate normalized by paragraph count. all runs it inside the same single decompression pass as the other three.
The hard problem was separating damage from intentional style, and rate alone cannot do it: deliberately unpunctuated literary prose (Fosse's Septology, Evaristo's Girl, Woman, Other, Kingsnorth's The Wake, Faulkner) posts split rates far above genuinely damaged books. The discriminator that works is where the fragment ends. Style breaks at clause boundaries ("...she started out in theatre"); damage breaks at line-wrap positions, which land on function words ("sat the disembodied" / "heads who were..."). On the reference library every style book measured at most 11% of splits ending on a function word and every hand-confirmed damage case at least 26%, so the flag gate requires 25%, alongside minimum-splits, minimum-paragraphs, and rate floors. The function-word set is a small closed list, the same spirit as the content analyzer's stopword votes, not a dictionary.
Five false-positive idioms found during validation are guarded explicitly: paragraphs interrupted by a rendered figure (an inline formula or card-diagram image reads as a split otherwise; _Blocks now records when an image falls between two blocks' text), display math set as text (mostly non-alphabetic fragments), back-of-book indexes rendered as paragraph blocks (the "See also" signature), epistolary sign-offs (a dangling short unterminated fragment), and the block-quotation idiom of academic prose (pairs into or out of a <blockquote>, plus attribution fragments ending on "that"). Secondary signals are reported but never gate: en-dashes embedded inside words (bottom–feedin'), doubled opening quotes (' 'Course, with the first quote required to follow whitespace so British single-quote dialogue's close-then-open sequences stay invisible), and space-stripped proper nouns recurring alongside their hyphenated form (AnkhMorpork vs Ankh-Morpork).
Hand-validated against the full 4,605-EPUB reference library, every flagged book inspected: 105 flagged, 104 confirmed damage, one borderline residue (display quotes publisher-styled as plain paragraphs, indistinguishable without CSS). Documented out of scope: character-substitution errors ("sonic" for "some") need a wordlist the stdlib-only contract rules out, and damage whose signature is word truncation or whitespace corruption rather than paragraph splitting. Tests grow to 60 in tests/test_scripts.py (split detection, dialogue-fragment and scene-break non-splits, image-interrupted pairs, style-vs-damage discrimination, threshold boundaries, and an all run including the new analyzer).
The persistent-curses-screen rework, closing the last item in the roadmap's "Port from the Lattice TUI audit" section (Lattice T7, shipped there as v4.10.0). Purely a lifecycle change; no menu, prompt, or mode behavior differs.
One curses screen per session. Every menu, prompt, pause, and pager used to be its own curses.wrapper init/teardown, so multi-prompt flows visibly flashed to the shell between widgets. interactive_menu now opens the screen once and every widget draws into it (_with_screen); a widget invoked outside a session still gets its own one-shot wrapper, so nothing changes for direct callers. _reset_terminal becomes a no-op while the session screen lives (running stty sane under curses would undo cbreak/noecho beneath it). Measured under a pty: a full menu, prompt, Esc-cancel, mode, pager, quit session enters the terminal's alternate screen exactly once, where the same flow used to enter it once per widget.
One degradation path. A curses failure at session startup or mid-session (unknown terminal, capability lost) funnels through _degrade_to_text: the screen is suspended once and the rest of the session runs the text fallback. All the v3.3.2 guarantees hold (no stuck terminal, no silent exit 0); a pager that dies mid-display now also degrades and prints the mode's output as plain text instead of eating it. Ctrl-C at the menu (curses or text fallback alike) ends the session cleanly with exit code 130 and the screen restored, while EOF at the text menu stays a quiet Quit.
tests/test_tui.py grows to 29 cases; the session lifecycle was additionally verified end-to-end under a pty (alternate-screen count, degraded startup on an unknown TERM, and a full cancel-then-run flow against the live library).
The rest of the Lattice TUI audit ports (roadmap section "Port from the Lattice TUI audit": T2, T4, T6, and the fallback-menu generation). Lattice shipped all of these in its v4.9.0; this release keeps the two shared curses skeletons aligned.
Esc in a prompt now cancels back to the menu instead of accepting the default (behavior change, port of Lattice T2). In menus Esc has always meant back; in prompts it meant "accept the default", so mis-selecting a mode and mashing Esc launched it with all defaults. A cancelled prompt now unwinds the whole prompt chain back to the menu with nothing launched; bare Enter still accepts the default, and Ctrl-C or EOF at a prompt cancels the same way (the text-fallback prompts used to exit the whole program on Ctrl-C). The hint bar now reads "Enter Accept, Esc Cancel, Ctrl-U Clear". Cancelling the first-run prompt exits without persisting anything (exit code 1, the CLI's no-database signal, so unattended runs never read as success), and cancelling "Change database path" leaves the saved path untouched.
Output-file prompts expand ~ (port of Lattice T4). ~/reports/x.txt used to create a literal ./~/reports/ directory, since the TUI has no shell to expand it. Every output path the TUI collects now goes through a shared _prompt_out() that expands the tilde but does not absolutize, so relative paths keep meaning the current directory. The results pager also gains a footer naming the resolved absolute path, so "where did my report go" answers itself; the footer is suppressed when the mode errored or was cancelled, so it never claims a file that was not written.
The no-curses fallback menu is generated from the same sections the arrow-key menu renders (port of Lattice's v4.8.1 fix). The numbered listing and its key map were hand-maintained twins of _MAIN_SECTIONS, exactly the pattern that silently desynced in Lattice (fallback keys dispatching the wrong modes). _build_fallback now derives both from the sections, the word aliases ("catalog", "stats", ...) stay as an explicit supplemental dict, and tests pin that every entry is reachable with its number matching its label.
Widget paper cuts (port of Lattice T6). A bad answer at a number prompt re-asks with a "not a number, try again" note instead of silently using the default. On terminals shorter than the menu, the box shifts so the selected row stays visible instead of clipping blind below the bottom edge. The pager gains horizontal panning (arrow keys or h/l) with ellipsis markers on lines that continue off-screen, computes its width once instead of on every keypress, and keeps the scroll position valid across resizes. Ctrl-U clears a prompt field, so editing a long pre-filled path no longer means backspacing through all of it. And the export format prompt validates its answer against json/csv/ai, re-asking instead of accepting any string.
tests/test_tui.py grows from 7 to 24 cases, pinning all of the above.
TUI hardening ported from the Lattice TUI audit (2026-07-01). cquarry/tui.py shares its curses skeleton with Lattice's; the two carry-overs from that audit's high-severity findings land here (roadmap section "Port from the Lattice TUI audit", items H7 and H6's exception-boundary half). First: a curses init failure no longer reads as Quit. On capability-poor terminals (TERM=vt100, dumb terminals) the color setup or curs_set raised curses.error, which the menu loop treated as the user quitting, so the TUI silently exited 0 even though the text fallback menu works. Cosmetic capabilities are now non-fatal (a monochrome TUI beats a dead one), and a real curses.wrapper failure flips the session to the text fallback menu instead of exiting. Second: _run_with_capture now has an exception boundary. A mode error used to escape as a raw traceback and lose the captured output; it is now paged under an [Error] heading with the traceback plus whatever was captured, and Ctrl-C pages a [Cancelled] notice the same way. New tests/test_tui.py (7 cases) pins both behaviors. The remaining Lattice carry-overs (Esc-cancels-prompt, ~ expansion in output prompts, generated fallback menu) stay on the roadmap until their Lattice counterparts land.
audit_drm.py no longer flags a freed EPUB on a leftover marker file. v3.3.0 treated the mere presence of META-INF/rights.xml (Adobe ADEPT) or sinf.xml (Apple FairPlay) as DRM. But those are token/voucher files, not the lock itself: the actual lock is content encryption, which a DRM'd EPUB records in encryption.xml against its XHTML. When a book is freed, the content is decrypted but the marker can stay behind, so a bare marker with no content encryption is a residual artifact, not a locked book; it reads and embeds fine. The first whole-library sweep surfaced exactly one such case (Warhammer Helsreach: a sinf.xml, no encryption.xml, 37 plain-XHTML chapters), which is the same residual-artifact shape as the PDF that motivated the tool. EPUB classification now keys on actual content encryption (encryption.xml with non-font entries) and names the scheme from whichever marker is present; a standalone marker is reported BENIGN as a "residual DRM marker". PDFs are unchanged: a residual handler dictionary there still breaks metadata embedding, so it is still flagged. With this fix the library's real DRM count is 48 (all recoverable PDF ADEPT dictionaries), with the lone FairPlay EPUB correctly cleared. Tests extended to 21 cases (residual markers benign; markers plus encrypted content still DRM).
audit_drm.py: a cross-format DRM scanner (read-only). The metadata and structural audits never look at encryption, so a DRM-locked file can pass epubcheck, report its page count, and even import, yet silently refuse to let its embedded metadata be rewritten. The case that prompted this was a z-library PDF carrying a residual Adobe ADEPT EBX_HANDLER dictionary that qpdf --check and pdfinfo both reported as "not encrypted" while exiftool choked on it, failing the reconcile embed. The new script classifies EPUB, PDF, and Kindle (MOBI/AZW3) files; DJVU has no DRM scheme and is reported N/A. It runs in library mode (formats and paths from metadata.db, opened strictly mode=ro) or directory mode (a recursive scan of loose files before import, for the pre-import battery), with the usual exit codes (0 clean, 1 DRM found or scan error, 2 setup error).
The design priority was not detecting encryption; it was not crying wolf. Two benign things look like DRM to a crude check and are explicitly cleared. Font obfuscation: an EPUB META-INF/encryption.xml that scrambles only the embedded fonts (the IDPF or Adobe #RC algorithms) is not a content lock; because publishers often name obfuscated fonts fonts/00001.dat with no font extension, an entry is cleared when it uses a font-scrambling algorithm OR targets a font resource (extension or a fonts/ path), which an algorithm-only or extension-only check gets wrong. Permission flags: a PDF "encrypted" with the Standard handler and an empty user password opens with no password and is only flagged against printing or copying, so it is classed with qpdf as PERMISSIONS, not a lock. Real DRM is rights.xml (Adobe ADEPT) or sinf.xml (Apple FairPlay) in an EPUB, a content-encrypting encryption.xml, a non-Standard PDF security handler found by a streaming byte scan (so a residual or inactive dictionary is still caught, which is exactly the Irodov case), a password-locked PDF, or a non-zero Mobipocket encryption-type field.
The first whole-library sweep (6,651 files) found 50 DRM-locked files (49 Adobe ADEPT, 1 Apple FairPlay), with 135 benign font-obfuscation/permission cases correctly cleared and one early false positive fixed before release: five EPUBs whose Adobe #RC font obfuscation targets fonts/*.dat were initially misread as encrypted content, which is what drove the algorithm-or-target rule above. Ships with a unittest suite (tests/test_audit_drm.py, 19 cases) building zip, PDF-byte, and PalmDB fixtures for each format and verdict.
reconcile_file_metadata.py --repair-pdf now deletes the .~qpdf-orig backup qpdf leaves behind. qpdf --replace-input, used to rebuild a broken cross-reference table before re-embedding, writes the pre-repair original to <name>.~qpdf-orig beside the file and never removes it. Across many reconcile passes these full-size copies accumulated inside the library tree, which is the worst place for them: Calibre scans that tree, and each one is a complete duplicate PDF. A sweep of one library turned up 21 such files totalling 403 MB. embed_pdf now unlinks the backup as soon as qpdf reports success (return code 0 or 3), before retrying the embed; a missing backup is a no-op, so the change is safe whether or not qpdf wrote one. Regression tests mock the exiftool/qpdf boundary to assert the backup is removed after a successful repair and that an absent backup does not raise. Pre-existing strays from older runs are not cleaned by the tool; remove them once with fd -H '\.~qpdf-orig$' "<library>" -X rm.
audit_epub.py emptytext now flags partial / placeholder exports (new PARTIAL verdict). The whole-book character count missed a failure mode: a DRM-locked or sample export where most chapters are an identical "content unavailable" placeholder while one or two real chapters carry enough text to clear the THIN floor, so the book validates, repairs clean, and reads as full-length to the old total-char check. The canonical case was a BookShout export of Johannes Cabal: The Fear Institute, where 15 of 17 chapters were the same 138-char "something went wrong loading... bookshout.com" stub; even after structural repair to zero epubcheck fatals it stayed a 2-chapter sample. The analyzer now flags PARTIAL when a known DRM-placeholder signature appears anywhere in the spine, or when the same short stub (12 to 600 chars) repeats across at least 3 spine documents and at least 30% of the spine. PARTIAL is a real defect (counts as FOUND, exit 1, needs re-sourcing), distinct from the advisory THIN. The false-positive guard is the per-document distribution: a well-made book full of small but DISTINCT section dividers does not trip it (only repeated-identical stubs do), so the three full novels in the batch that surfaced this stayed OK. Library and directory modes both report it.
audit_epub.py now percent-decodes spine hrefs, fixing false EMPTY verdicts. OPF manifest hrefs are IRIs, so a content document whose archive filename contains a reserved character (commonly !, written %21; Sigil and calibre emit these routinely) was matched against the raw zip namelist undecoded, failed to resolve, and dropped out of the spine. A text-full book whose every chapter file had such a name resolved to zero readable spine documents and was reported EMPTY: the exact false positive hit on Martha Wells's The Serpent Sea (every split_NNN.html was named CR!RT...). The resolver now decodes the percent-encoding (UTF-8, with multi-byte runs decoded together) and strips any #fragment before matching the namelist. Stdlib-only via a small re-based decoder (_pct_decode); no urllib dependency added. The fix lands in the shared spine resolver, so all three analyzers (content, pagenumbers, emptytext) benefit. Regression tests cover the decoder (reserved char, multi-byte UTF-8, invalid escape) and an end-to-end encoded-spine EPUB.
The three EPUB-content audits are merged into one scripts/audit_epub.py. audit_epub_content.py, audit_epub_pagenumbers.py, and audit_epub_emptytext.py shared the same spine resolution, library/directory dual-mode, read-only contract, and exit codes, and differed only in the per-book verdict; they are now three analyzers behind one tool, selected by subcommand: audit_epub.py content|pagenumbers|emptytext|all [directory]. The detection logic of each is unchanged (same thresholds, same results). Two wins beyond removing the duplicated scaffolding: all opens each EPUB once and runs all three analyzers in a single decompression pass (the expensive part is decompression, so this is much faster than three separate full-library runs), and there is now one spine resolver to maintain instead of three slightly-diverging copies. The old script names are removed; update any caller to audit_epub.py <mode>. The --min-chars / --thin-chars knobs (emptytext) carry over.
scripts/audit_epub_emptytext.py: find empty / no-body-text EPUBs. Catches the failure mode every other audit misses: a content-less stub that still validates. The canonical case is the "Bookmate" export, where the archive holds only cover and promo images plus a tiny HTML placeholder, the OPF spine points only at that placeholder, and the book itself is absent. Such a file passes epubcheck, "repairs" clean in a structural repairer (its one referenced document is well-formed), and shows no foreign text to audit_epub_content.py because there is no text at all; the metadata looks perfect. The detector resolves the spine, drops <script>/<style>, strips tags, decodes entities, and counts the rendered characters: EMPTY (<=2000, --min-chars) is a real defect to re-source, THIN (<20000, --thin-chars) is advisory because a genuine short story or a publisher sample can also land there. A Bookmate origin is not itself a defect; most Bookmate exports carry their full text, so the flag is on empty content, not provenance. Library mode (DB-driven, mode=ro) and directory mode (vet downloads before import), mirroring the other two EPUB audits. First full-library run flagged four real defects (a Bujold stub, two image-only scans with no text layer, and a Draft2Digital sample of a full novel hiding in the THIN tier) against three genuinely short stories left alone.
spot_check.py no longer flags OCaml and NCurses as case garble. The intercaps allowlist (_CASE_OK) now includes OCaml and NCurses alongside SQLite, QBasic, and the rest, so legitimate library titles stop tripping the advisory case-garble heuristic.
scripts/audit_epub_pagenumbers.py: find print page numbers baked into EPUB body text. Bad PDF/OCR-to-EPUB conversions capture the print page number (and often the running header) as a literal paragraph in the flow instead of real EPUB pagination, so it reflows into the middle of a sentence ("where the hay cart 16 was taking him"). The detector reads each book's blocks in spine order and flags a number only when it genuinely interrupts prose: a lowercase continuation after it, a word split across it (the previous block ends in a hyphen), or it abuts a repeated running header/footer. Section and chapter numbers (which open the next block with a capital) and endnote/footnote numbers and chronology years are left alone. Library mode (DB-driven, mode=ro) and directory mode (vet downloads before import), mirroring audit_epub_content.py. Validated by hand against the full reference library: 21 flagged, every one a true positive; the false-positive tail (an experimental footnote-poem, a scraped web-serial's vote counts, placeholder section labels) all fell under the hit-count or book-span floors. It also surfaces piracy watermarks and bad OCR scans that ride along with the page-number cruft.
scripts/spot_check.py: randomized metadata + file-integrity audit. Samples N random books (reproducible with --seed) and checks what pattern-based sweeps miss: title corruption and mojibake, junk author entries, missing or stub descriptions, EPUB archive integrity (CRC, container/OPF sanity, spine completeness, text volume), PDF header/page count, and DJVU page count. Emits a machine-readable flag report plus a review bundle (title/author/tag/series/blurb per sampled book) for a human or LLM judgment pass. Read-only against metadata.db; validator-owned checks are not duplicated. First 600-book run against the live library caught a wrong-book description, a truncated description, three mojibake descriptions, and an EPUB with ten dangling spine references.
reconcile_file_metadata.py PDF embeds no longer fail silently. Two related defects: exiftool refuses to rewrite XMP packets containing duplicate properties (seen in the wild: a doubled prism:doi) unless -m is passed, and it exits 0 with "files unchanged", which the tool read as success; The embed now passes -m. (An interim attempt to also write a bare -Publisher tag was reverted: on PDFs exiftool maps it to the same XMP dc:publisher bag, so double assignment appended duplicate entries and broke the round-trip.)
write_catalog no longer corrupts the shared book cache. It sorted the list returned by get_all_books() in place, silently reordering the session cache for every later consumer. It now sorts a copy; a regression test pins the behavior (tests/test_modes.py).
compress_pdf.py cannot clobber a rollback original. If a .pre-compress rollback file already exists, an in-place run now aborts instead of overwriting the only copy of the true original with an already-compressed file.
audit_epub_content.py finds the library from either home. The library root resolves to wherever metadata.db sits: next to the script (the copy living inside the library) or the current working directory (running the repo copy from a library), in that order.
reconcile --id rejects malformed id lists cleanly via a dedicated parser instead of an unhandled ValueError.
Exception chaining (raise ... from) throughout search.py and helpers.py; unused loop variable removed in wing-overlap analytics; re.Scanner access satisfied for type checkers; new test coverage for the backup guard, library-root resolution, and cache isolation (suite: 87 tests).
Python 3.14+. The supported floor moved from 3.9 to 3.14 to match the development environment. The code does not depend on bleeding-edge syntax, but only 3.14+ is tested and supported.
Comprehensive search engine. The search expression parser was rebuilt as a dedicated, stdlib-only engine (src/cquarry/search.py) that ports Calibre's grammar and matching semantics. It now resolves field locations beyond tags and authors: series, publisher, rating, formats, languages, pubdate/date/last_modified, identifiers/isbn, comments, cover, id, uuid, and #custom columns, in addition to tags, authors, all, and vl:. It supports contains/=exact/~regex/^accent match kinds, numeric and date relational operators (rating:>=4, pubdate:>2015, date:30daysago), and field:true/field:false presence tests. Boolean grouping, implicit AND, quotes, and escapes follow Calibre's grammar, evaluated with its candidate-set semantics. The previous build only handled tags:, author(s):, and vl:; other prefixes silently matched nothing.
--search prints to stdout. With no --output, results stream to the terminal instead of forcing a file. --format json|csv|ai emits the matching books in that structured shape; otherwise a plain-text listing is produced. An empty query (--search '') returns the whole library, matching Calibre.
Deeper cover audit. Cover dimension reading no longer stops at the first 1 KB, so a JPEG whose SOF marker sits behind a large EXIF/ICC block is measured correctly; PNG covers are now read too.
Interactive TUI analytics no longer crashes. Selecting any item under the ANALYTICS menu (Author statistics, Reading pace, Tag tree, Wing overlap) raised a NameError because those functions were never imported into tui.py. They now work.
Half-star ratings are visibly distinct. A 2.5 rating rendered identically to 2.0 (both ★★☆☆☆); it now shows ★★½☆☆ using the universally available ½ glyph.
Series "complete" is computed correctly. Completeness now means "no missing integer volumes" rather than "book count equals the top index", so a series with novellas (0.5) or duplicate editions is no longer wrongly marked incomplete.
Portability of the series rollup. get_all_series was rewritten to aggregate in Python instead of using GROUP_CONCAT(... ORDER BY ...), which required SQLite 3.44+.
Normalized custom columns now load. Single-valued text and enumeration custom columns (e.g. a "Status" / reading-state column) are stored by Calibre in a value table plus a books_custom_column_N_link table, exactly like multi-valued columns. The loader keyed off is_multiple and tried to read a book column straight from the value table, which errored and silently returned nothing. It now detects the link table, so --show-custom and #column searches work for every custom-column type.
Honest parity claims. The README and spec no longer claim "100% parity"; they document exactly which locations and operators are supported and the deliberate, dependency-bound deviations (stdlib re instead of the regex module, unicodedata folding instead of ICU, no GPM templates or saved-search references, and cquarry's anchored hierarchical tags: match).
Companion scripts. scripts/compress_pdf.py (Ghostscript-based PDF shrinking with verify-or-rollback; writes files and metadata.db) and scripts/audit_epub_content.py (read-only EPUB content auditor) are now versioned alongside the toolkit and fully documented, explicitly outside the read-only cquarry package contract.
Portable test suite. tests/test_search.py and tests/test_helpers.py cover the parser grammar (adapted from Calibre's own tests), the matcher against an in-memory provider, a full-stack integration test on a temporary SQLite fixture, and the rating/series/image helpers, all without needing a live Calibre library.
Tag Dump (--tags). A flat, alphabetized list of every tag in the library with its book count, written to stdout. Drop-in replacement for the noisy calibredb list_categories -r tags shell pipeline — pipe it to a file with cquarry --tags > tags.txt. Also reachable as "Tag dump" under LISTS in the interactive TUI. Honors --quiet (suppresses header/footer, leaves the body intact for scripting). Distinct from --analytics tags, which renders the hierarchical tree.
Full Parity Search Engine. Refactored the --search expression parser to achieve 100% parity with Calibre's native syntax.
- Author Searching: Added full support for
author:andauthors:prefix tokens. - Fallback Text Search: Un-prefixed terms (e.g.
cquarry --search "author:Anne Rice") now correctly fall back to searching anywhere across book titles, authors, and tags. This accurately mimics Calibre's implicit booleanANDhandling of unquoted spaces. - Complex Grouping: Verified and documented support for nested boolean logic, parenthetical grouping, and negative lookaheads (e.g.,
NOT(tags:Fic.Romance OR tags:Fic.Contemporary)ortags:"Fic.Fantasy.Grimdark" AND author:"Phil Tucker"). - Test Suite. Added an automated test suite (
tests/test_search.py) mapped against Calibre's actualSearchQueryParserbehavior to guarantee ongoing expression fidelity.
Completed Software Status. Phase 4 has been concluded, and CalibreQuarry is now considered feature-complete and stable. It has undergone rigorous end-to-end testing against real-world Calibre databases.
Bug Fixes:
- Fixed a
NameErrorcrash in--auditmode where the newly introducedcolorhelper was not imported, preventing the final summary from printing when issues were found.
Custom Column Support. Added a --show-custom "Column Name" flag that extracts data from user-defined custom Calibre columns. The values are automatically appended to text catalogs and are natively included in JSON, CSV, and AI exports.
Color CLI Output. Introduced simple, lightweight ANSI color formatting for headers, warnings, and error highlights across CLI modes to improve readability when bypassing the interactive pager.
Extended Audit Checks. The --audit mode has been significantly expanded to include three new checks:
- Duplicate Detection: Identifies books with identical titles and primary authors across the library.
- Cover Quality Audit: Scans the actual JPEG cover files on disk (without external library dependencies) and flags covers with low resolution (below 500px on their longest edge).
- Format Migration Report: Flags books that are only available in deprecated legacy formats (MOBI, LIT, LRF, DJVU, PDB, AZW).
Extended Analytics. Added a new --analytics argument with four detailed reporting modes: author (per-author breakdowns of formats, ratings, and series), pace (books added per month/year trend), tags (hierarchical taxonomy tree visualization), and overlap (virtual library wing overlap analysis). These are also accessible via a new ANALYTICS section in the interactive TUI.
Search Query Export. You can now pass arbitrary Calibre search expressions directly to the CLI via --search "query" to export matching books. The results are written to a plain text file. This feature is also accessible via the interactive TUI under the OUTPUT menu.
AI-Readable Export. Added a new ai format to the --export option. This format outputs the library data as a highly token-efficient, flat text list designed specifically for LLM ingestion and recommendation prompts (e.g., Title by Author [Tags] - Rating/5).
Top Authors and Top Tags in --stats. The statistics output now includes a "Top authors" section (10 most prolific by book count) and a "Top tags" section (15 most-used tags), inserted between the ratings distribution and the tag taxonomy breakdown.
CalibreQuarry has been refactored from a single ~1450-line monolithic script (cquarry.py) into a proper Python package architecture, with TUI improvements modeled after the Lattice project.
Layer-Based Package Design. The codebase now lives in src/cquarry/ and is split by logical functionality: config.py, db.py, helpers.py, cli.py, tui.py, and a modes/ directory for individual feature operations (catalog.py, stats.py, audit.py, display.py, export.py). The monolithic script is gone.
Modern Build System (Hatch). CalibreQuarry now uses pyproject.toml managed by Hatch. Install via pip install . or pipx install . and the cquarry command is available globally. Also runnable via python -m cquarry.
Persistent Database Configuration. Both CLI and TUI now share a unified database resolution chain: explicit --db flag, saved config (~/.config/cquarry/config.json), default search paths, then an interactive prompt if running in a TTY. The path is saved on first successful resolution, eliminating the need to pass --db in future sessions. A "Change database path" option under the SETTINGS section in the TUI main menu allows updating the stored path.
Calibre Lock Handling. When metadata.db is locked by a running Calibre instance, CalibreQuarry now automatically copies the database (including WAL/SHM journal files) to a temporary snapshot and reads from that instead. A notice is printed to stderr, and the temp files are cleaned up on exit. Previously, a locked database would produce an unhandled sqlite3.OperationalError.
Fully Immersive TUI. All operations now run through a _run_with_capture() wrapper that intercepts stdout and stderr via io.StringIO buffers. Output is displayed within the scrollable curses pager rather than dropping the user back to raw terminal output. This matches the immersive TUI pattern established in Lattice v4.1.2.
Styled Curses Pause. The post-operation "Press Enter to continue" prompt now renders inside a styled Unicode box within the curses session (_tui_pause), instead of falling through to a raw input() call. Accepts Enter, q, or Esc to dismiss.
Null Byte Sanitization. The scrollable pager now strips null bytes from captured output before rendering, preventing ValueError: embedded null character crashes on corrupted data.
Curses TUI. Running the script with no arguments now launches a full-screen, arrow-key navigable terminal UI utilizing the curses library (matching the interface of getMusic). Non-TTY environments or systems without curses will gracefully fall back to a styled text-based box menu. The TUI features a custom scrollable pager that intercepts standard output, allowing you to comfortably read and navigate command outputs directly within the interface.
Implicit AND in VL expressions ignored subsequent tags. Calibre's parser evaluates adjacent tags like tags:Fic.Fantasy tags:Magic as an implicit AND. The _parse_and method previously discarded tags after the first one unless the AND keyword was explicitly written out. It now correctly intersects all implicit constraints.
Exact tag matching (=) was case-sensitive. Calibre's tag matches are always case-insensitive. The exact match SQL query (WHERE t.name = ?) was missing COLLATE NOCASE, causing tags:"=Fic.Fantasy" to fail if capitalization varied.
Duplicate author headers in --primary-only mode. Generating a catalog with --primary-only caused highly fragmented author groups. The script relied solely on the SQL ORDER BY b.author_sort (which sorts by the full multi-author string). Books are now presorted in Python natively by their derived primary-only display key.
Non-deterministic GROUP_CONCAT output. The metadata fields built via GROUP_CONCAT(DISTINCT ...) (like authors and tags) returned unpredictably ordered results depending on SQLite's internal row execution. This occasionally resulted in the wrong primary author being selected. The SQL query has been rewritten to use correlated subqueries with explicit ORDER BY clauses for deterministic structure.
Double NOT cascading crashes. Expressions combining consecutive exclusions (e.g., NOT NOT vl:Name) failed because _parse_not routed directly to _parse_atom for the inner operand. This has been updated to recursively call _parse_not to handle complex nested negations gracefully.
Fractional series indices ignored in recent display. show_recent dropped series identifiers completely if the index contained a decimal (e.g., 1.5) due to a missing fallback formatting block.
_read_value crashed on unmatched quotes in VL expressions.
expr.index('"') raised ValueError if a virtual library search expression
had an opening quote with no closing quote. A single malformed VL definition
took down the entire tool. Now consumes the rest of the string as the value.
Series index 0.0 silently dropped. if idx and idx == int(idx) treated
0.0 as falsy — any book at series position zero lost its series display. Hit
in both write_catalog and show_recent. Changed to explicit is not None
checks.
Division by zero in show_stats. Empty library crashed on three separate
lines: format bar chart (count * 40 // total), rating bar chart
(max(rating_counts.values())), and unrated percentage
(unrated * 100 / total). All three guarded.
max_index None dereference in show_series.
s['max_index'] == int(s['max_index']) threw TypeError when max_index was
None. Same fix propagated into detect_series_gaps.
format_stars produced garbage on corrupt ratings. A DB value of 12 (6.0
stars) yielded a negative empty count. Python silently returns "" for
"☆" * -1, so no crash — but the display was meaningless. Rating now clamped
to 0–5.
CSV export blanked zero-valued fields. stars or '' and
series_index or '' used the or pattern, which treats 0.0 as falsy.
Changed to explicit is not None checks.
JSON export had leading whitespace in split fields.
b['authors'].split(',') on GROUP_CONCAT output produced
["Author1", " Author2"]. All split fields now strip.
Tokenizer keyword boundary missed underscores. isalnum() doesn't match
_, so a hypothetical tag starting with or_ or not_ would misparse as a
boolean operator. Boundary check now includes underscore.
_prompt_str displayed [None] in interactive prompts. When called with
default=None, the user saw the literal text [None]. Now shows empty
brackets.
get_all_books() results cached. The 8-JOIN metadata query was called once
per wing in --all-wings mode — 18 times against a 3,800-book library. Now
fires once and returns the cached list.
_get_all_book_ids() cached for NOT operations. The VL parser queried
SELECT id FROM books on every NOT clause. Multiple NOT expressions in a
single VL definition hammered the DB. Cached on first call.
count_books() uses warm caches. If the books or IDs cache is already
populated, returns len() instead of hitting SQLite.
Parser built once in main(). Was constructing build_parser() to parse
args, then building it again on the help-output fallthrough path. Stored the
reference.
--version flag. Uses action="version" so argparse handles it during
parse_args() — works even when no database is present. Version also shown in
the interactive menu banner.
Unused imports removed. defaultdict and Path — imported, never
referenced.
f-string with no placeholders. f"\nLanguages:" → "\nLanguages:".
show_wings caught bare Exception. Narrowed to ValueError, which is
what resolve_vl actually raises.
quiet parameter wired up everywhere. show_recent, show_series, and
show_stats all accepted quiet but ignored it. Now suppresses headers and
decorative output when passed.
main() catches PermissionError. A read-only DB with wrong filesystem
permissions previously produced an unhandled traceback.
_match_tags docstring corrected. Said "containing" for the non-exact case
but the SQL does prefix match, not substring. Added a note that regex patterns
(tags:~regex) are unsupported.
prog="cquarry.py" added to ArgumentParser. Version and help output now
show the script name consistently regardless of invocation path.