v3.115.2

v3.115.2 (2025-12-18)

Full Changelog

Fixed

• Try to avoid overriding BUILDKITE_PLUGINS_ALWAYS_CLONE_FRESH with false #3644 (@DrJosh9000)
• SUP-5826: Remove experiment from 'env' command #3635 (@Mykematt)

Internal

• Nested-loop jitter structure for log processing #3645 (@DrJosh9000)
• Add E2E test for Azure Blob storage #3642 (@DrJosh9000)
• PB-1007: add e2e test for gcs artifact upload/download #3633 (@zhming0)
• PB-1025: improve e2e test DevEX #3634 (@zhming0)

Dependency updates

• chore(deps): bump zstash to v0.7.0 #3632 (@wolfeidau)
• build(deps): bump the cloud-providers group with 2 updates #3638 (@dependabot[bot])
• build(deps): bump the otel group with 5 updates #3637 (@dependabot[bot])
• build(deps): bump github.com/DataDog/datadog-go/v5 from 5.8.1 to 5.8.2 #3639 (@dependabot[bot])
• build(deps): bump the container-images group across 5 directories with 1 update #3640 (@dependabot[bot])
• build(deps): bump docker/library/golang from cf1272d to 54528d1 in /.buildkite in the container-images group across 1 directory #3641 (@dependabot[bot])

v3.115.1

v3.115.1 (2025-12-12)

Full Changelog

Fixes

• PS-1491: Fix double retry issue for k8s mode bootstrap #3628 (@zhming0)

Internal

• PB-1023: remove old kubernetes bootstrap setup #3629 (@zhming0)
• chore(deps): update zstash to v0.6.0 and update progress callback #3630 (@wolfeidau)
• feat: add support for concurrent save and restore operations #3627 (@wolfeidau)

v3.115.0

v3.115.0 (2025-12-10)

Full Changelog

Added

• --changed-files-path for pipeline upload, which allows users to specify a list of files changed for if_changed computation #3620 (@pyrocat101)

Fixes

• Further fixes to custom bucket artifact uploads/downloads #3615 (@moskyb)

Internal

• Dependabot updates #3618 #3619 #3622 #3623 #3621 (@dependabot[bot])

v3.114.1

v3.114.1 (2025-12-05)

Full Changelog

Fixed

• Fix issue where artifacts uploaded to customer-managed s3 buckets could not be downloaded #3607 (@moskyb)

Internal

• Add an end-to-end testing framework! #3611 #3610 #3609 #3608 #3606 #3604 #3599 (@DrJosh9000)
• Dependency updates #3601 #3600 (@dependabot[bot])
• Update MIME types #3603 (@DrJosh9000)

v3.114.0

v3.114.0 (2025-11-25)

Full Changelog

Added

• feat: add agent metadata to OTEL trace attributes #3587 (@pyrocat101)

Fixed

• Fix for the agent sometimes failing to disconnect properly when exiting - agent pool: Send error after disconnecting #3596 (@DrJosh9000)

Internal

• internal/redact: Add another test with minor cleanup #3591 (@DrJosh9000)
• Run gofumpt as part of CI #3589 (@moskyb)

Dependency updates

• build(deps): bump the cloud-providers group with 7 updates #3593 (@dependabot[bot])
• build(deps): bump the container-images group across 5 directories with 1 update #3594 (@dependabot[bot])
• build(deps): bump the container-images group across 1 directory with 2 updates #3595 (@dependabot[bot])
• build(deps): bump golang.org/x/crypto from 0.44.0 to 0.45.0 #3590 (@dependabot[bot])

v3.113.0

v3.113.0 (2025-11-18)

Full Changelog

Added

• Add Prometheus /metrics handler and some basic metrics #3576 (@DrJosh9000)

Fixed

• Fix the pipeline upload --reject-secrets flag not rejecting secrets #3580 (@moskyb)
• Fix idle tracking for agents that never received jobs #3579 (@scadu)

Internal

• Clarify agent idlemonitor states in comment #3582 (@DrJosh9000)
• Put secret scan error into exit message #3581 (@DrJosh9000)

Dependency updates

• build(deps): bump the golang-x group with 3 updates #3583 (@dependabot[bot])
• build(deps): bump the cloud-providers group with 7 updates #3584 (@dependabot[bot])

v3.112.0

v3.112.0 (2025-11-12)

Full Changelog

Warning

Known issue: the disconnect-after-idle-timeout option in this version does not take effect until an agent has run a job, and so agents that never run a job will never disconnect. Fixed in the following version v3.113.0 via #3579

Added

The agent can now annotate jobs as well as builds! Job annotations will show up in a dedicated section of the job detail
in the build UI. This is a great way to provide additional, richly-formatted context and information about specific jobs.

See the PR for more details.

Changed

• Agents will now check for new work more quickly immediately after finishing a job #3571 (@DrJosh9000)

Fixed

• IdleMonitor-related fixes #3570 (@DrJosh9000)
• Fix confusing error message when hashing artifact payloads #3565 (@moskyb)

Internal

• Dependency updates #3575 #3574 #3573 #3572 (@dependabot[bot])

v3.111.0

v3.111.0 (2025-11-05)

Full Changelog

Warning

If you use a custom S3 bucket for artifacts, this applies to you.

As part of updating to AWS Go SDK v2, the "credential chain" for providing
authentication credentials to access artifacts in custom S3 buckets, is now
more standard. The existing BUILDKITE_S3_ env vars are still available and
take precedence, but when these are not set, the AWS-default mechanisms are
used as provided by the SDK, with as few customisations as possible.

This means additional ways to pass credentials to the AWS S3 client may be
accepted, and where multiple credentials are available, the precedence may
have changed (to match what the AWS SDK expects by default).

Because of this, and the number of combinations of different ways to provide
credentials, this change may inadvertently break pipelines using custom S3
buckets for artifacts. Please reach out to support@buildkite.com or raise
issues in GitHub if this impacts you!

Added

• Add cache save and restore using github.com/buildkite/zstash #3551 (@wolfeidau)

Changed

• Upgrade to AWS Go SDK v2 #3554 (@DrJosh9000)
• Catch all 'ignored' vars #3502 (@DrJosh9000)

Internal

• chore: go modernize to do a bit of a tidy up and remove some junk #3560 (@wolfeidau)
• Enforce that command descriptions indent using spaces, not tabs #3553 (@moskyb)

Dependency updates

• build(deps): bump the cloud-providers group across 1 directory with 9 updates #3566 (@dependabot[bot])
• build(deps): bump golangci/golangci-lint from v2.5-alpine to v2.6-alpine in /.buildkite in the container-images group across 1 directory #3563 (@dependabot[bot])
• build(deps): bump the container-images group across 4 directories with 1 update #3564 (@dependabot[bot])
• build(deps): bump gopkg.in/DataDog/dd-trace-go.v1 from 1.74.7 to 1.74.8 #3555 (@dependabot[bot])
• build(deps): bump the cloud-providers group with 6 updates #3556 (@dependabot[bot])
• build(deps): bump the container-images group across 4 directories with 1 update #3557 (@dependabot[bot])
• build(deps): bump docker/library/golang from 02ce1d7 to 5034fa4 in /.buildkite in the container-images group across 1 directory #3558 (@dependabot[bot])

v3.110.0

v3.110.0 (2025-10-22)

Full Changelog

Added

• Configurable chunks interval #3521 (@catkins)
• Inject OpenTelemetry context to all child processes #3548 (@zhming0)
  • This is done using environment variables. This may interfere with existing OTel environment variables if they are manually added some other way.
• Add --literal and --delimiter flags to artifact upload #3543 (@DrJosh9000)

Changed

Various improvements and fixes to do with signal and cancel grace periods, and signal handling, most notably:

• When cancelling a job, the timeout before sending a SIGKILL to the job has changed from cancel-grace-period to signal-grace-period (--signal-grace-period-seconds flag, BUILDKITE_SIGNAL_GRACE_PERIOD_SECONDS env var) to allow the agent some extra time to upload job logs and mark the job as finished. By default, signal-grace-period is 1 second shorter than cancel-grace-period. You may wish to increase cancel-grace-period accordingly.
• When SIGQUIT is handled by the bootstrap, the exit code is now 131, and it no longer dumps a stacktrace.
• The recently-added --kubernetes-log-collection-grace-period flag is now deprecated. Instead, use --cancel-grace-period.
• When running the agent interactively, you can now Ctrl-C a third time to exit immediately.
• In Kubernetes mode, the agent now begins shutting down on the first SIGTERM. The kubernetes-bootstrap now swallows SIGTERM with a logged message, and waits for the agent container to send an interrupt.
• When the agent is cancelling jobs because it is stopping, all jobs start cancellation simultaneously. This allows the agent to exit sooner when multiple workers (--spawn flag) are used.
  See #3549, #3547, #3534 (@DrJosh9000)

Fixed

• Refresh checkout root file handle after checkout hook #3546 (@zhming0)
• Bump zzglob to v0.4.2 to fix uploading artifact paths containing ~ #3539 (@DrJosh9000)

Internal

• Docs: Add examples for step update commands for priority and notify attributes #3532 (@tomowatt)
• Docs: Update URLs in agent cfg comments #3536 (@petetomasik)

Dependency updates

• Upgrade Datadog-go to v5.8.1 to work around mod checksum issues #3538 (@dannyfallon)
• build(deps): bump the container-images group across 3 directories with 2 updates #3545 (@dependabot[bot])
• build(deps): bump gopkg.in/DataDog/dd-trace-go.v1 from 1.74.6 to 1.74.7 #3544 (@dependabot[bot])
• build(deps): bump github.com/gofrs/flock from 0.12.1 to 0.13.0 #3523 (@dependabot[bot])
• build(deps): bump docker/library/golang from 1.24.8 to 1.24.9 in /.buildkite in the container-images group across 1 directory #3542 (@dependabot[bot])
• build(deps): bump the cloud-providers group across 1 directory with 6 updates #3541 (@dependabot[bot])
• build(deps): bump the container-images group across 3 directories with 1 update #3540 (@dependabot[bot])
• build(deps): bump the golang-x group with 5 updates #3525 (@dependabot[bot])

v3.109.1

v3.109.1 (2025-10-15)

Full Changelog

Fixed

• Pass aws config to ec2 client for fetching tags #3529 (@migueleliasweb)
• PS-1245: Fix artifact search output format escape sequence handling #3522 (@zhming0)
• Fix inconsistency in artifact search --format flag documentation #3520 (@ivannalisetska)