Skip to content

update buildkit #435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gilescope wants to merge 4 commits into
base: main
Choose a base branch
from
Open

update buildkit #435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3c5c9d7
feat: update buildkit
gilescope Mar 28, 2026
23f0c9f
feat: fix vulnerabilites
gilescope Mar 28, 2026
a8b7691
point to buildkit PR to check works
gilescope Apr 8, 2026
8b5ca75
feat: update go-grpc to support new security standard
gilescope Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ which will use the credentials which are stored in earthly's [cloud-hosted secre

## Updates to buildkit or fsutil

Earthly is built against a fork of [buildkit](https://github.com/earthly/buildkit) and [fsutil](https://github.com/earthly/fsutil).
Earthly is built against a fork of [buildkit](https://github.com/EarthBuild/buildkit) and [fsutil](https://github.com/EarthBuild/fsutil).
Comment thread
gilescope marked this conversation as resolved.

To work with changes to this fork, you can use `earthly +for-linux --BUILDKIT_PROJECT=../buildkit`. This will use the local directory `../buildkit` for the buildkit code, when using buildkit in both `go.mod` and when building the buildkitd image.

Expand Down
8 changes: 4 additions & 4 deletions Earthfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ code:
FROM +deps
# Use BUILDKIT_PROJECT to point go.mod to a buildkit dir being actively developed. Examples:
# --BUILDKIT_PROJECT=../buildkit
# --BUILDKIT_PROJECT=github.com/earthly/buildkit:<git-ref>
# --BUILDKIT_PROJECT=github.com/EarthBuild/buildkit:<git-ref>
Comment thread
gilescope marked this conversation as resolved.
ARG BUILDKIT_PROJECT
IF [ "$BUILDKIT_PROJECT" != "" ]
COPY --dir "$BUILDKIT_PROJECT"+code/buildkit /buildkit
Expand All @@ -81,8 +81,8 @@ code:
update-buildkit:
FROM +code # if we use deps, go mod tidy will remove a bunch of requirements since it won't have access to our codebase.
ARG BUILDKIT_GIT_SHA
ARG BUILDKIT_GIT_BRANCH=earthly-main
ARG BUILDKIT_GIT_ORG=earthly
ARG BUILDKIT_GIT_BRANCH=main
ARG BUILDKIT_GIT_ORG=earthbuild
ARG BUILDKIT_GIT_REPO=buildkit
COPY (./buildkitd+buildkit-sha/buildkit_sha --BUILDKIT_GIT_ORG="$BUILDKIT_GIT_ORG" --BUILDKIT_GIT_SHA="$BUILDKIT_GIT_SHA" --BUILDKIT_GIT_BRANCH="$BUILDKIT_GIT_BRANCH") buildkit_sha
BUILD ./buildkitd+update-buildkit-earthfile --BUILDKIT_GIT_ORG="$BUILDKIT_GIT_ORG" --BUILDKIT_GIT_SHA="$(cat buildkit_sha)" --BUILDKIT_GIT_REPO="$BUILDKIT_GIT_REPO"
Expand Down Expand Up @@ -554,7 +554,7 @@ build-ticktock:
LET ticktock="$(cat earthly-next)"
ARG EARTHLY_TARGET_TAG_DOCKER
LET BUILDKIT_TAG="dev-$EARTHLY_TARGET_TAG_DOCKER-ticktock"
BUILD --platform=linux/amd64 ./buildkitd+buildkitd --BUILDKIT_PROJECT="github.com/earthly/buildkit:$ticktock" --TAG=$BUILDKIT_TAG
BUILD --platform=linux/amd64 ./buildkitd+buildkitd --BUILDKIT_PROJECT="github.com/EarthBuild/buildkit:$ticktock" --TAG=$BUILDKIT_TAG
Comment thread
gilescope marked this conversation as resolved.
END

# for-linux builds earthly-buildkitd and the earthly CLI for the a linux amd64 system
Expand Down
6 changes: 3 additions & 3 deletions buildkitd/Earthfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ buildkitd:
ARG BUILDKIT_BASE_IMAGE=$BUILDKIT_PROJECT+build
END
ELSE
ARG BUILDKIT_BASE_IMAGE=github.com/EarthBuild/buildkit:c40c767e201de9ad72c8c52e207dc8f4b5b51e9d+build
ARG BUILDKIT_BASE_IMAGE=github.com/earthbuild/buildkit:d9c219825d099da530cfaf9e9c8acdbdbbb88a1b+build
END
ARG EARTHLY_TARGET_TAG_DOCKER
ARG TAG="dev-$EARTHLY_TARGET_TAG_DOCKER"
Expand Down Expand Up @@ -96,13 +96,13 @@ buildkit-sha:
ARG --required BUILDKIT_GIT_ORG
RUN --no-cache set -e; \
if [ "$(echo -n $BUILDKIT_GIT_SHA | wc -c)" = 40 ]; then \
echo "pinning github.com/earthly/buildkit to reference git sha1: $BUILDKIT_GIT_SHA"; \
echo "pinning github.com/${BUILDKIT_GIT_ORG}/buildkit to reference git sha1: $BUILDKIT_GIT_SHA"; \
buildkit_sha1="$BUILDKIT_GIT_SHA"; \
else \
test -z "$BUILDKIT_GIT_SHA"; \
echo "looking up branch $BUILDKIT_GIT_BRANCH"; \
buildkit_sha1=$(git ls-remote --refs -q https://github.com/$BUILDKIT_GIT_ORG/buildkit.git "$BUILDKIT_GIT_BRANCH" | awk 'BEGIN { FS = "[ \t]+" } {print $1}'); \
echo "pinning github.com/earthly/buildkit@${BUILDKIT_BRANCH} to reference git sha1: $buildkit_sha1"; \
echo "pinning github.com/${BUILDKIT_GIT_ORG}/buildkit@${BUILDKIT_BRANCH} to reference git sha1: $buildkit_sha1"; \
Comment thread
gilescope marked this conversation as resolved.
fi && \
test -n "$buildkit_sha1" && \
echo "$buildkit_sha1" > buildkit_sha
Expand Down
2 changes: 1 addition & 1 deletion docs-internals/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ the [buildkit/docs/dev](https://github.com/moby/buildkit/tree/master/docs/dev) s
| **pullping** | Once the build function returns (passing a set of LLB references back to buildkit), the BuildKit server will execute the commands, and call the earthlyoutputs exporter, which will call back to the client (Earthly), which will be received by the pullping handler. This will cause earthly to perform a `docker pull ...` against the embedded registry |
| **dockertar** | The legacy approach for exporting images from BuildKit to the host via a `tar` file; we try to use pullping instead, since it only pulls the needed layers |
| **logbus** | An interface for writing output to both stdout and the web-based log viewer under cloud.earthly.dev |
| **earthlyoutputs** | A custom buildkit exporter (within the [earthly/buildkit fork](https://github.com/earthly/buildkit/tree/earthly-main/exporter/earthlyoutputs)), which is used to send images back to earthly |
| **earthlyoutputs** | A custom buildkit exporter (within the [earthly/buildkit fork](https://github.com/EarthBuild/buildkit/tree/main/exporter/earthlyoutputs)), which is used to send images back to earthly |
Comment thread
gilescope marked this conversation as resolved.
| **embedded registry** | A [docker registry](https://github.com/distribution/distribution) which runs within the earthly-buildkitd container, used in combination with earthlyoutputs and the pullping callback; also referred to as "local registry" |

## Guides
Expand Down
20 changes: 11 additions & 9 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ require (
golang.org/x/sync v0.19.0
golang.org/x/term v0.40.0
golang.org/x/text v0.34.0
google.golang.org/grpc v1.66.3
google.golang.org/grpc v1.80.0
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
)
Expand All @@ -82,8 +82,8 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/console v1.0.3 // indirect
github.com/containerd/containerd v1.7.27 // indirect
github.com/containerd/console v1.0.5 // indirect
github.com/containerd/containerd v1.7.29 // indirect
github.com/containerd/containerd/api v1.8.0 // indirect
github.com/containerd/continuity v0.4.4 // indirect
github.com/containerd/errdefs v0.3.0 // indirect
Expand All @@ -92,7 +92,7 @@ require (
github.com/containerd/typeurl/v2 v2.1.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/docker v24.0.0-rc.2.0.20230905130451-032797ea4bcb+incompatible // indirect
github.com/docker/docker v25.0.13+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/elastic/go-windows v1.0.2 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
Expand All @@ -109,11 +109,13 @@ require (
github.com/in-toto/in-toto-golang v0.5.0 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/sys/mountinfo v0.7.1 // indirect
github.com/moby/sys/signal v0.7.0 // indirect
github.com/moby/sys/user v0.3.0 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/runc v1.1.9 // indirect
github.com/opencontainers/runtime-spec v1.1.0 // indirect
github.com/opencontainers/runtime-spec v1.2.0 // indirect
github.com/opencontainers/selinux v1.12.0 // indirect
github.com/otiai10/mint v1.6.3 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/poy/onpar v1.1.2 // indirect
Expand Down Expand Up @@ -151,13 +153,13 @@ require (
golang.org/x/sys v0.42.0 // indirect
golang.org/x/time v0.14.0 // indirect
google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260120221211-b8f7ae30c516 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260120221211-b8f7ae30c516 // indirect
gotest.tools/v3 v3.4.0 // indirect
howett.net/plist v0.0.0-20181124034731-591f970eefbb // indirect
)

replace (
github.com/moby/buildkit => github.com/earthbuild/buildkit v0.0.0-20260219223358-c40c767e201d
github.com/moby/buildkit => github.com/earthbuild/buildkit v0.0.0-20260408093444-d9c219825d09
github.com/tonistiigi/fsutil => github.com/earthbuild/fsutil v0.0.0-20231030221755-644b08355b65
)
Loading
Loading