Fix/br 1151 error opening files read callback#278
Open
AlexisMora wants to merge 10 commits intomainfrom
Open
Conversation
…own system flag to enable vlc open files + remove private in diod classes
egalvis27
approved these changes
Mar 18, 2026
…download, instant resolve of promises if requested bytes are already available
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is Changed / Added
Instead of downloading the entire file into memory before serving bytes, the new implementation writes the download stream directly to disk and serves requested byte ranges as soon as they land. A waiter queue tracks which byte ranges have been written, allowing FUSE read callbacks to resolve as soon as their requested bytes are available. This way there is no need to wait for the full download.
A global registry tracks in-flight downloads per file with contentsId. If multiple read requests arrive for the same file, they all share the same download instead of starting new ones. Once the download completes, the file is registered in the database so future reads are served directly from disk.
When the virtual drive cache is cleared, all in-flight downloads are cancelled and partial files are cleaned up.
The shouldDownload check was blocking legitimate file opens from applications like VLC. Flag 34816 (O_RDONLY | O_LARGEFILE) was not recognized as a valid user open. The logic now only blocks the known system flag (294912)
If the user closes a file mid-download, the download continues in the background until it completes (to me this makes sense so that if accessed again, the load is faster.)
Why
The previous implementation buffered the entire file into an in-memory Map<string, Buffer> before any bytes could be served to the calling application. For large files (videos, PDFs), this caused high memory usage and slow file opens.
The new approach streams bytes to disk progressively and serves them as soon as they arrive, keeping memory usage constant regardless of file size.