Skip to content

Remove redundant GIL handoff in VM dispatch#328

Open
proboscis wants to merge 2 commits intomainfrom
issue/vm-perf-gil-cleanup/run-20260314-164318
Open

Remove redundant GIL handoff in VM dispatch#328
proboscis wants to merge 2 commits intomainfrom
issue/vm-perf-gil-cleanup/run-20260314-164318

Conversation

@proboscis
Copy link
Copy Markdown
Owner

Summary

  • share one materialized dispatch effect object across type-filter checks and handler expression construction
  • remove py: Python from Kleisli::{apply, apply_with_run_token} and IRStreamProgram::start, while pushing Python attachment into Python-backed implementations
  • drop RustKleisliStream GIL wrappers and add/update regression tests for the new call signatures and dispatch object reuse

Validation

  • cargo test --manifest-path packages/doeff-vm-core/Cargo.toml
  • PYTHONHOME=/home/kento/.local/share/uv/python/cpython-3.14.0b1+freethreaded-linux-x86_64-gnu PYO3_PYTHON=/home/kento/.orch/worktrees/vm-perf-gil-cleanup/5c5ab6_codex_20260314-164318/.venv/bin/python LD_LIBRARY_PATH=/home/kento/.local/share/uv/python/cpython-3.14.0b1+freethreaded-linux-x86_64-gnu/lib cargo test --manifest-path packages/doeff-vm/Cargo.toml
  • uv run pytest packages/doeff-vm-core/tests/test_gil_cleanup_architecture.py
  • uv run pytest tests/ -> 1193 passed, 12 skipped
  • uv tool run semgrep --config .semgrep.yaml packages/doeff-vm-core/src/vm/step.rs packages/doeff-vm-core/src/kleisli.rs packages/doeff-vm-core/src/handler.rs packages/doeff-vm-core/src/vm/dispatch.rs packages/doeff-vm-core/src/vm/vm_trace.rs -> 0 findings

Acceptance Criteria

  • Kleisli::apply / apply_with_run_token no longer take py: packages/doeff-vm-core/src/kleisli.rs, packages/doeff-vm-core/src/handler.rs
  • IRStreamProgram::start no longer takes py: packages/doeff-vm-core/src/handler.rs, packages/doeff-core-effects/src/handlers/mod.rs, packages/doeff-core-effects/src/scheduler/mod.rs
  • RustKleisliStream::resume/throw no longer wrap Python::attach: packages/doeff-vm-core/src/kleisli.rs
  • dispatch_to_pyobject is materialized once and passed into handler expansion through ensure_dispatch_effect_object(...): packages/doeff-vm-core/src/vm/dispatch.rs, packages/doeff-vm-core/src/vm/vm_trace.rs
  • no production behavior regression observed: full Python suite passed after rebuild

Notes

  • cargo test --manifest-path packages/doeff-core-effects/Cargo.toml already fails on origin/main at baseline commit 9aa8f77c because the crate test target aliases DispatchEffect = Effect, which diverges from the doeff-vm-core trait signature. This PR does not introduce that failure.
  • Ref: vm-perf-gil-cleanup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant