Improve release assets compression #1570
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Change the release archive format from ZIP to 7-Zip
* Change the compression algorithm from DEFLATE to LZMA
Optimize release assets for file size, because the compress/decompress
performance difference between the settings is negligible for our data.
At time of this commit, the total release assets size before this patch
is about 946 MB. After this patch, the total size is about 638 MB. So
this change results in about 33% total size reduction for the release
assets.
Because the Windows file explorer has natively supported 7-Zip
decompression since Windows 11 22H2 (released late 2022), this change
should not affect the regular user experience for people with an
up-to-date Windows machine. And since 7z is a fairly well-known format,
I doubt it should cause confusion even for those whose machines don't
have native extraction capability.
Another alternative considered was changing the ZIP algorithm to LZMA,
as support for it was added to the ZIP spec already in 2006, but as of
this commit date, Windows still does not natively handle LZMA-compressed
ZIP archives extraction. Therefore, to avoid users confusing
LZMA-compressed ZIPs as being corrupted, I chose to instead change the
entire archive format to 7-Zip.
Note that this is a breaking change for the Steam release script, because both the
archive format (ZIP -> 7-Zip) and the file extension (".zip" -> ".7z")
have changed. The file name patterns aside from the file extension
remain unchanged.
Collaborator
Author
|
There's an example output of the 7z assets for preview here: https://github.com/Rainyan/rebuild/releases/tag/latest And in contrast, the current latest dev release assets of the main branch: https://github.com/NeotokyoRebuild/neo/releases/tag/latest |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Summary
Optimize release assets for file size, because the compress/decompress performance difference between these settings is negligible for our data.
At time of this commit, the total (release + debug) assets size before this patch is about 946 MB. After this patch, the total size is about 638 MB. So this change results in about 33% size (and download time) reduction for the release assets.
Compatibility
Users
Because the Windows file explorer GUI natively supports 7-Zip decompression since Windows 11 22H2 (released late 2022), this change should not affect the regular user experience for people with an up-to-date Windows machine. And since 7z is a fairly well-known format, I doubt it should cause confusion even for those whose OS doesn't have native extraction capability.
Steam release script
This is a breaking change for the Steam release script, because both the archive format (ZIP -> 7-Zip) and the file extension (
<filename>.zip-><filename>.7z) have changed. As such, this PR should not be merged before confirming compatibility with the Steam release script.The actual file name patterns aside from the file extension remain unchanged.
Alternatives considered
The asset format
Alternatively, we could've changed the ZIP algorithm used to LZMA, as support for it was added to the ZIP spec already in 2006, but as of this PR date, Windows file explorer still does not natively handle LZMA-compressed ZIP archives extraction. Therefore, to avoid users confusing LZMA-compressed ZIPs as being corrupted, I chose to instead change the entire archive format to 7-Zip (which is fully compatible for native .7z extraction with modern Windows 11).
Compressing intermediate build artifacts
I also experimented with changing the upload-artifact action's compression config between 1 (best speed) and 9 (best compression) from the default value of 6, but this made no meaningful difference in the CI job speeds (+/- a couple seconds, which could've been random load variance). So I decided to not change that from the default.
Toolchain
N/A
Linked Issues