docker-compose: Add bb-remote-asset to setup#91
Open
minor-fixes wants to merge 1 commit intobuildbarn:mainfrom
Open
docker-compose: Add bb-remote-asset to setup#91minor-fixes wants to merge 1 commit intobuildbarn:mainfrom
minor-fixes wants to merge 1 commit intobuildbarn:mainfrom
Conversation
This change adds a bb-remote-asset setup to the docker-compose
deployment example, which allows for:
* easier local testing of bb-remote-asset with the other buildbarn
components
* a working example of how bb-remote-asset should be configured*
Tested: Started locally via `docker-compose/run.sh`; ran:
```
bazel clean --expunge && \
rm -rf ~/.cache/bazel/_bazel_$USER/cache/ && \
bazel build \
--remote_executor=grpc://localhost:8980 \
--remote_instance_name=fuse \
--experimental_remote_downloader=grpc://localhost:8984 \
//...
```
Build completed successfully, and remote-asset logs showed blobs being
downloaded.
Ran the following twice:
```
grpc_cli call localhost:8984 \
build.bazel.remote.asset.v1.Fetch.FetchBlob \
'instance_name: "fuse" uris: "https://github.com/bazelbuild/remote-apis/raw/main/build/bazel/remote/asset/v1/remote_asset.proto" qualifiers { name: "checksum.sri" value: "sha256-e8wR0NA0I8XHMeF8Fzk7EjHrLYQp4zUpZtXZIe3c3Fs=" }'
```
and got the following responses:
```
connecting to localhost:8984
status {
message: "Blob fetched successfully!"
}
uri: "https://github.com/bazelbuild/remote-apis/raw/main/build/bazel/remote/asset/v1/remote_asset.proto"
qualifiers {
name: "checksum.sri"
value: "sha256-e8wR0NA0I8XHMeF8Fzk7EjHrLYQp4zUpZtXZIe3c3Fs="
}
blob_digest {
hash: "7bcc11d0d03423c5c731e17c17393b1231eb2d8429e3352966d5d921eddcdc5b"
size_bytes: 21947
}
Rpc succeeded with OK status
```
```
connecting to localhost:8984
status {
message: "Blob fetched successfully from asset cache"
}
uri: "https://github.com/bazelbuild/remote-apis/raw/main/build/bazel/remote/asset/v1/remote_asset.proto"
qualifiers {
name: "checksum.sri"
value: "sha256-e8wR0NA0I8XHMeF8Fzk7EjHrLYQp4zUpZtXZIe3c3Fs="
}
blob_digest {
hash: "7bcc11d0d03423c5c731e17c17393b1231eb2d8429e3352966d5d921eddcdc5b"
size_bytes: 21947
}
Rpc succeeded with OK status
```
which indicates that caching functionality works as well.
---
*I don't actually know the best way to configure bb-remote-asset
minor-fixes
commented
May 10, 2023
|
|
||
| remote-asset: | ||
| # TODO(minor-fixes): Replace with actual container once pushed to ghcr | ||
| image: bazel/cmd/bb_remote_asset:bb_remote_asset_container |
Author
There was a problem hiding this comment.
This can't be merged as-is until this image is put somewhere public (ghcr.io?)
Collaborator
There was a problem hiding this comment.
Would you mind updating the image reference now when ghcr.io/buildbarn/bb-remote-asset:20230910T205857Z-52b25db is available?
2 tasks
minor-fixes
added a commit
to buildbarn/bb-remote-asset
that referenced
this pull request
Sep 10, 2023
This change brings bb-remote-asset up-to-date with bb-storage/bb-remote-execution, so that nominally they may be able to use compatible configs, and likely inherit the other benefits (more recent toolchain, better bazel integration, etc.) This is done largely by copying `WORKSPACE` from bb-storage, making sure to match versions and orderings to ensure that the same versions of dependencies are loaded. Other transforms/migrations that proved necessary:0000 * `HTTPClient` was eliminated from bb-storage; mocking had to be done on a `http.RoundTripper` instead, in the same manner as buildbarn/bb-storage@b9951ac * Addition of structured concurrency to main, similar to buildbarn/bb-storage@4f243ea * Addition of `http.ClientConfiguration` proto options to `HttpFetcherConfiguration` * `bazel.canonical_id` added to the list of supported qualifiers, which prevents errors in some (most?) WORKSPACE configurations Tested: * `bazel test //...` with buildbarn/bb-storage#170 works on my system * Seems to work as expected when integrated with the bb-deployments docker-compose setup as in buildbarn/bb-deployments#91
Collaborator
|
See comment in the code. Shall some test execution also be updated to run with |
moroten
requested changes
Sep 12, 2023
|
|
||
| remote-asset: | ||
| # TODO(minor-fixes): Replace with actual container once pushed to ghcr | ||
| image: bazel/cmd/bb_remote_asset:bb_remote_asset_container |
Collaborator
There was a problem hiding this comment.
Would you mind updating the image reference now when ghcr.io/buildbarn/bb-remote-asset:20230910T205857Z-52b25db is available?
| ports: | ||
| - 8984:8984 | ||
| volumes: | ||
| - ./config:/config No newline at end of file |
Collaborator
There was a problem hiding this comment.
Nit: Please add an empty line at the end of file.
| listenAddresses: [':8984'], | ||
| authenticationPolicy: { allow: {} }, | ||
| }], | ||
| allowUpdatesForInstances: ['foo'], |
Collaborator
There was a problem hiding this comment.
Is there a more descriptive name, or just a comment, about what this instance name refers to?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds a bb-remote-asset setup to the docker-compose deployment example, which allows for:
Tested: Started locally via
docker-compose/run.sh; ran:Build completed successfully, and remote-asset logs showed blobs being downloaded.
Ran the following twice:
and got the following responses:
which indicates that caching functionality works as well.
*I don't actually know the best way to configure bb-remote-asset