fix: add streaming DMMF binary command to bypass V8 and WASM limits#5761
Open
chris-tophers wants to merge 1 commit intoprisma:mainfrom
Open
fix: add streaming DMMF binary command to bypass V8 and WASM limits#5761chris-tophers wants to merge 1 commit intoprisma:mainfrom
chris-tophers wants to merge 1 commit intoprisma:mainfrom
Conversation
Add a `get-dmmf` subcommand to the prisma-fmt CLI that streams DMMF JSON directly to stdout via serde_json::to_writer(). This approach has no memory ceiling — unlike WASM (limited to ~4GB linear memory), the native binary can stream arbitrarily large DMMF with only 1x peak memory (the in-memory DMMF struct, no serialized buffer). Changes: - dmmf crate: add dmmf_json_to_writer() using serde_json::to_writer() - prisma-fmt: add get_dmmf_to_writer() that validates + streams - prisma-fmt: export get_dmmf_to_writer() from lib.rs - prisma-fmt: add GetDmmf CLI variant, reads stdin params, streams to stdout Alternative approach to the buffered WASM API in prisma#5757. Fixes: prisma/prisma#29111 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This was referenced Feb 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Alternative approach to #5757 for bypassing V8's string length limit on large DMMF.
Adds a
get-dmmfsubcommand to theprisma-fmtCLI binary that streams DMMF JSON directly to stdout viaserde_json::to_writer(). Unlike the buffered WASM approach, this has no memory ceiling — the native binary streams incrementally with zero intermediate allocation.Root Cause
dmmf_json_from_validated_schema()usesserde_json::to_string(), producing a single RustString. Whenwasm-bindgenconverts this to a JS string, V8 rejects it if the string exceeds0x1fffffe8characters (~536MB). The WASM buffered approach (#5757) works around this by returningVec<u8>chunks, but WASM32 linear memory is capped at ~4GB, giving a practical ceiling of ~1.5-2GB DMMF.The binary streaming approach eliminates both limits:
serde_json::to_writer()writes JSON tokens incrementally to stdout with only 1x peak memory (the in-memory DMMF struct — no serialized buffer).Changes
query-compiler/dmmf/src/lib.rsdmmf_json_to_writer()usingserde_json::to_writer()prisma-fmt/src/get_dmmf.rsget_dmmf_to_writer()that validates schema + streamsprisma-fmt/src/lib.rsget_dmmf_to_writer()prisma-fmt/src/main.rsGetDmmfCLI variant: reads JSON params from stdin, streams DMMF to stdout+47 lines, 4 files changed. No existing behavior modified. No new dependencies.
Comparison: WASM Buffered vs Binary Streaming
Vec<u8>)prisma-fmt)to_writer+ CLI cmd)prisma-fmtin npm orPRISMA_FMT_BINARYenv varUsage
The TypeScript companion PR detects the V8 error and spawns this binary as a fallback, stream-parsing stdout with
@streamparser/json.Companion PR
TypeScript fallback: prisma/prisma#29149 (adds binary streaming fallback in TypeScript)
Notes
prisma-fmtis already compiled as a native binary in CI — it just isn't shipped via npm todayPRISMA_FMT_BINARYenv var allows users to point to a custom-built binary immediatelyFixes: prisma/prisma#29111