Skip to content

fix(object-store): make bucket error-rate respond to successes#89

Merged
shikhar merged 1 commit intomainfrom
codex/fix-bucket-error-rate-87
Feb 28, 2026
Merged

fix(object-store): make bucket error-rate respond to successes#89
shikhar merged 1 commit intomainfrom
codex/fix-bucket-error-rate-87

Conversation

@shikhar
Copy link
Member

@shikhar shikhar commented Feb 28, 2026

Summary

  • switch bucket error-rate updates to request-level EMA so successes decrease penalty and failures increase it
  • retain existing time-decay and circuit-breaker behavior
  • add regression coverage for high-success traffic after an error and update the zero-elapsed success expectation

Testing

  • cargo +nightly fmt
  • cargo clippy --all-features --all-targets -- -D warnings --allow deprecated
  • cargo nextest run test_success_without_elapsed_time_reduces_error_penalty test_success_traffic_drives_error_rate_down test_error_rate_simple test_error_rate_accumulation test_error_rate_decay

Closes #87

@greptile-apps
Copy link

greptile-apps bot commented Feb 28, 2026

Greptile Summary

Switched bucket error-rate tracking from time-decay-only to proper request-level EMA where successes actively reduce error penalty and failures increase it.

Key changes:

  • Successes now multiply error rate by (1 - α) ≈ 0.985, actively driving it down
  • Failures apply the same decay factor then add α, properly penalizing errors
  • Time-decay behavior preserved through initial error_rate(now) call
  • Circuit breaker logic unchanged

Test coverage:

  • Updated test_success_without_elapsed_time_reduces_error_penalty to verify successes reduce penalty even with zero latency
  • Added test_success_traffic_drives_error_rate_down to verify 1000 successes suppress error rate to <5%
  • Existing tests validate error accumulation and decay behavior remain sound

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • Clean implementation of the EMA formula with mathematically sound logic. Comprehensive test coverage validates both the new success-driven error reduction and existing failure accumulation behavior. Code follows Rust best practices and project style guidelines.
  • No files require special attention

Important Files Changed

Filename Overview
src/object_store/stats.rs Implements request-level EMA for error rates - successes now actively decrease error penalty by factor (1-α), failures increase it. Includes comprehensive test coverage for the new behavior.

Last reviewed commit: c57bfa0

@shikhar shikhar merged commit 4978bef into main Feb 28, 2026
6 checks passed
@shikhar shikhar deleted the codex/fix-bucket-error-rate-87 branch February 28, 2026 01:47
@github-actions github-actions bot mentioned this pull request Feb 28, 2026
shikhar pushed a commit that referenced this pull request Feb 28, 2026
## 🤖 New release

* `cachey`: 0.10.3 -> 0.10.4

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

##
[0.10.4](0.10.3...0.10.4)
- 2026-02-28

### Fixed

- *(object-store)* make bucket error-rate respond to successes
([#89](#89))

### Other

- dep updates + future size fix for clippy
([#91](#91))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Detail Bug] Object store: Bucket error rate ignores successes, unfairly penalizing high-throughput buckets

1 participant