After multiple updates to the inference package, there has been a significant change in the median runtime (3.6 -> 2.6). The two main factors that contribute to this, imo, are:
num_workers being set to 0 in predict. The process creation overhead was actually quite significant.
- SDPA
We should update the runtimes for Chronos-2 in the leaderboard.