fix(tests): skip pyinstrument profiling for benchmark tests

pyinstrument 5.1.2 uses a C-level ProfilerState object via sys.setprofile() that is not a callable Python function. pytest-benchmark's PauseInstrumentation saves and restores the profiler via sys.setprofile(), which fails with TypeError on the non-callable ProfilerState.

Skip the auto_profile fixture for tests using the benchmark fixture, since profiling during benchmarks is counterproductive anyway (adds overhead that corrupts measurements).

Author: johnnymillergh

tests/conftest.py

@@ -28,8 +28,16 @@ def auto_profile(request: Any) -> Generator[Any, Any]:
     """
     Generate an HTML file for each test node in your test suite inside the .profiles directory.
 
+    Skips profiling for benchmark tests to avoid incompatibility between pyinstrument's
+    C-level ProfilerState and pytest-benchmark's PauseInstrumentation (which calls
+    ``sys.setprofile()`` with the non-callable ProfilerState object).
+
     https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-pytest-tests
     """
+    if "benchmark" in request.fixturenames:
+        yield
+        return
+
     profile_root = PROJECT__ROOT / "build/.profiles"
     logger.info("Starting to profile Pytest unit tests...")
     # Turn profiling on