Skip to content

[Feature 7/N] PdL2Adapter: Integration Test, Controller Wiring & Documentation #224

@hlin99

Description

@hlin99

Label
Please label your issue with "new feature", "l2-adapter", "multiprocess", "pd-backend", "pr/7", and other relevant labels.

Is your feature request related to a problem? Please describe.
PRs 1–6 deliver config, skeleton, key mapper, submit/query, store/load data path, and proxy notification individually. However, no integration test validates the full sender→receiver→proxy flow end-to-end through the StoreController and PrefetchController. There is also no documentation or usage guide for the new PD L2 Adapter in MP mode.

Describe the solution you'd like

1. Integration Tests

  • File: tests/v1/distributed/l2_adapters/test_pd_l2_adapter_integration.py
  • Full E2E scenario with mock_memory TransferChannel:
    • test_sender_receiver_store_load_roundtrip: Sender stores N keys → receiver alloc server handles → receiver lookup finds keys → receiver loads into L1 MemoryObj → verify tensor data matches
    • test_store_controller_with_pd_adapter: Wire PdL2Adapter into StoreController, submit store, poll eventfd, verify completion
    • test_prefetch_controller_with_pd_adapter: Wire PdL2Adapter into PrefetchController, submit prefetch, verify lookup → load → prefix hits
    • test_proxy_notif_fires_on_last_prefill: Full store flow with is_last_prefill=True → verify ProxyNotif sent
    • test_concurrent_store_and_load: Interleave store and load from multiple threads
    • test_error_recovery: Simulate alloc failure mid-flow, verify partial completion and no leak

2. Controller Wiring

  • Verify register_l2_adapter_factory("pd", ...) creates a working adapter from JSON config
  • Ensure PdL2Adapter plugs into existing StoreController and PrefetchController without code changes to controllers
  • Verify eventfd polling works correctly with select

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions