Using filegroup_external to download files has the downside that if
one bumps the version number of the source URL, but forgets to update
the SHA-256 checksum, then Bazel will assume that the content is
unchanged and will silently use the old version. The canonical issue
thread for this is: bazelbuild/bazel#5144
Bazel now exposes a way to mitigate this problem—ctx.download takes a
new argument canonical_id, which forms part of the cache key:
https://github.com/bazelbuild/proposals/blob/master/designs/2019-04-29-cache.md
The filegroup_external rule could set the canonical_id to (a
lossless encoding of) the whole list of URLs, such that any change to
the list would cause a re-download. This has the slight downside that
adding a new URL to an existing list purely to increase redundancy would
also cause a one-time cache miss, which seems acceptable.
This would have saved a fair amount of confusion on the following PR:
tensorflow/tensorboard#2898
Using
filegroup_externalto download files has the downside that ifone bumps the version number of the source URL, but forgets to update
the SHA-256 checksum, then Bazel will assume that the content is
unchanged and will silently use the old version. The canonical issue
thread for this is: bazelbuild/bazel#5144
Bazel now exposes a way to mitigate this problem—
ctx.downloadtakes anew argument
canonical_id, which forms part of the cache key:https://github.com/bazelbuild/proposals/blob/master/designs/2019-04-29-cache.md
The
filegroup_externalrule could set thecanonical_idto (alossless encoding of) the whole list of URLs, such that any change to
the list would cause a re-download. This has the slight downside that
adding a new URL to an existing list purely to increase redundancy would
also cause a one-time cache miss, which seems acceptable.
This would have saved a fair amount of confusion on the following PR:
tensorflow/tensorboard#2898