Skip to content

Use permissively licensed LZO library #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wagoodman wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Use permissively licensed LZO library #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,7 @@ squashfs.test
# Memory and CPU pprof profiles
mem.out
cpu.out

# IDEs
.idea/
.vscode/
15 changes: 9 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,25 @@

[![PkgGoDev](https://pkg.go.dev/badge/github.com/CalebQ42/squashfs)](https://pkg.go.dev/github.com/CalebQ42/squashfs) [![Go Report Card](https://goreportcard.com/badge/github.com/CalebQ42/squashfs)](https://goreportcard.com/report/github.com/CalebQ42/squashfs)

A PURE Go library to read squashfs. There is currently no plans to add archive creation support as it will almost always be better to just call `mksquashfs`. I could see some possible use cases, but probably won't spend time on it unless it's requested (open a discussion if you want this feature).
A PURE Go library to read squashfs. There are currently no plans to add archive creation support as it will almost always be better to just call `mksquashfs`.
I could see some possible use cases, but probably won't spend time on it unless it's requested (open a discussion if you want this feature).

The library has two parts with this `github.com/CalebQ42/squashfs` being easy to use as it implements `io/fs` interfaces and doesn't expose unnecessary information. 95% this is the library you want. If you need lower level access to the information, use `github.com/CalebQ42/squashfs/low` where far more information is exposed.

Currently has support for reading squashfs files and extracting files and folders.

Special thanks to <https://dr-emann.github.io/squashfs/> for some VERY important information in an easy to understand format.
Thanks also to [distri's squashfs library](https://github.com/distr1/distri/tree/master/internal/squashfs) as I referenced it to figure some things out (and double check others).
Special thanks to <https://dr-emann.github.io/squashfs/> for some VERY important information in an easy-to-understand format.
Thanks also to [distri's squashfs library](https://github.com/distr1/distri/tree/master/internal/squashfs) as I referenced it to figure some things out (and double-check others).

## Build tags

As of `v1.1.0` this library has two optional build tags: `no_gpl` and `no_obsolete`. `no_gpl` disables the ability to read archives with lzo compression due to the library's gpl license. `no_obsolete` removes "obsolete" compression types for a reduced compilation size; currently this only disable lzma compression since it's superseded by xz.
This library has two optional build tags:
- `no_gpl` disables the ability to read archives with lzo compression due to the library's gpl license (this has been removed in `v1.2.0`).
Copy link
Author

@wagoodman wagoodman Jun 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note before merging: you might want to push a commit with the planned next version (or remove this note).

- `no_obsolete` removes "obsolete" compression types for a reduced compilation size; currently this only disables lzma compression since it's superseded by xz.

## FUSE

As of `v1.0`, FUSE capabilities has been moved to [a separate library](https://github.com/CalebQ42/squashfuse).
As of `v1.0`, FUSE capabilities have been moved to [a separate library](https://github.com/CalebQ42/squashfuse).

## Limitations

Expand All @@ -28,7 +31,7 @@ As of `v1.0`, FUSE capabilities has been moved to [a separate library](https://g

## Issues

* Noticably slower then `unsquashfs` for extraction, especially on larger images.
* Noticeably slower then `unsquashfs` for extraction, especially on larger images.
* This seems to be related to above along with the general optimization of `unsquashfs` and it's compression libraries.
* Times seem to be largely dependent on file tree size and compression type.
* My main testing image (~100MB) using Zstd takes ~2x longer.
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ module github.com/CalebQ42/squashfs
go 1.24.0

require (
github.com/anchore/go-lzo v0.1.0
github.com/klauspost/compress v1.18.0
github.com/mikelolasagasti/xz v1.0.1
github.com/pierrec/lz4/v4 v4.1.22
github.com/rasky/go-lzo v0.0.0-20200203143853-96a758eda86e
github.com/ulikunitz/xz v0.5.12
)
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
github.com/anchore/go-lzo v0.1.0 h1:NgAacnzqPeGH49Ky19QKLBZEuFRqtTG9cdaucc3Vncs=
github.com/anchore/go-lzo v0.1.0/go.mod h1:3kLx0bve2oN1iDwgM1U5zGku1Tfbdb0No5qp1eL1fIk=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/mikelolasagasti/xz v1.0.1 h1:Q2F2jX0RYJUG3+WsM+FJknv+6eVjsjXNDV0KJXZzkD0=
github.com/mikelolasagasti/xz v1.0.1/go.mod h1:muAirjiOUxPRXwm9HdDtB3uoRPrGnL85XHtokL9Hcgc=
github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=
github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/rasky/go-lzo v0.0.0-20200203143853-96a758eda86e h1:dCWirM5F3wMY+cmRda/B1BiPsFtmzXqV9b0hLWtVBMs=
github.com/rasky/go-lzo v0.0.0-20200203143853-96a758eda86e/go.mod h1:9leZcVcItj6m9/CfHY5Em/iBrCz7js8LcRQGTKEEv2M=
github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
10 changes: 4 additions & 6 deletions internal/decompress/lzo.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
//go:build !no_gpl

package decompress

import (
"bytes"

"github.com/rasky/go-lzo"
"github.com/anchore/go-lzo"
)

type Lzo struct{}
Expand All @@ -15,5 +11,7 @@ func NewLzo() (Lzo, error) {
}

func (l Lzo) Decompress(data []byte) ([]byte, error) {
return lzo.Decompress1X(bytes.NewReader(data), len(data), 0)
var dest []byte
_, err := lzo.Decompress(data, dest)
return dest, err
}
15 changes: 0 additions & 15 deletions internal/decompress/lzo_disabled.go
View file
Open in desktop

This file was deleted.