Skip to content

Add E2E wall-clock timing for store requests (GPU IPC + async data paths)#373

Draft
Copilot wants to merge 3 commits into
copilot/ww24-pr-async-againfrom
copilot/add-e2e-timing-submission
Draft

Add E2E wall-clock timing for store requests (GPU IPC + async data paths)#373
Copilot wants to merge 3 commits into
copilot/ww24-pr-async-againfrom
copilot/add-e2e-timing-submission

Conversation

Copilot AI commented Jun 17, 2026

Copy link
Copy Markdown

Adds a unified end-to-end timing mechanism in LMCacheMPWorkerAdapter that measures true wall-clock time from submit_store_request (forward thread) to when get_finished observes the future resolved — capturing actual GPU copy completion, not just kernel launch time.

Changes

  • import time added to standard imports
  • _store_submit_times: dict[str, float] added to LMCacheMPWorkerAdapter.__init__ to track per-request submit timestamps
  • submit_store_request: records time.perf_counter() at entry, keyed by request_id
  • get_finished: when s_future.query() returns True, pops the submit timestamp and emits:
    [E2E-STORE] req=<id> e2e=X.XXX ms
    
  • Unhealthy drain path: _store_submit_times.clear() alongside the existing store_futures.clear() calls

Why this works for both paths

Path What the future resolves after
GPU IPC (HandleTransferContext) MQ latency + server kernel + D2H copy + event sync + poll
Async data (AsyncDataTransferContext) Thread scheduling + gather launch + actual GPU copy (sync) + commit + poll

Both paths surface a MessagingFuture polled in get_finished, so a single instrumentation point gives an apples-to-apples comparison. Grep [E2E-STORE] to extract timings.

Copilot AI changed the title [WIP] Add E2E timing for submit_store to get_finished Add E2E wall-clock timing for store requests (GPU IPC + async data paths) Jun 17, 2026
Copilot AI requested a review from hlin99 June 17, 2026 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants