Skip to content

Benchmark: Sliding window for MAB-UCB #9

Description

@alltheseas

Summary

Add temporal decay to MAB-UCB so it forgets stale relay performance data. Currently all rounds are weighted equally.

Why

Relay quality changes over time — a relay that was great 6 months ago may be dead now. The current MAB-UCB implementation weights all 500 simulated rounds equally, which works in a static benchmark but would mislead a real client. Using only the last N observations (or exponential decay) would make the algorithm responsive to relay health changes.

What to do

  1. Implement a sliding window variant: only use the last N observations per relay when computing UCB scores
  2. Alternatively, implement exponential decay: weight recent observations more heavily
  3. Benchmark both variants against vanilla MAB-UCB across all 6 time windows

Effort

Medium — requires modifying the MAB state management and benchmarking multiple decay parameters.

Reference

  • IMPLEMENTATION-GUIDE.md: Improvement Opportunities (medium effort)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions