Skip to content

Segfault in OpenVINOExecutionProvider with NPU on PVE kernel 6.17.9 (libopenvino.so.2541) #1043

@eckmannmiles91

Description

@eckmannmiles91

Description

onnxruntime-openvino 1.24.1 segfaults immediately when creating an InferenceSession with OpenVINOExecutionProvider on an Intel Arrow Lake system with NPU. The crash occurs in the bundled libopenvino.so.2541 (OpenVINO 2025.4.1) at a fixed offset during NPU plugin initialization.

CPUExecutionProvider works fine with the same model files and environment.

Environment

  • onnxruntime-openvino: 1.24.1 (latest on PyPI)
  • Bundled OpenVINO: 2025.4.1 (libopenvino.so.2541)
  • Python: 3.12
  • OS: Ubuntu 24.04.4 LTS (LXC container on Proxmox VE 9.1.5)
  • Kernel: 6.17.9-1-pve (Proxmox, host kernel)
  • CPU: Intel Core Ultra 7 265 (Arrow Lake)
  • NPU device: /dev/accel0 passed through to LXC
  • Kernel module: intel_vpu 1.0.0 (built into PVE kernel 6.17.9)
  • NPU userspace: intel-level-zero-npu 1.32.0, intel-fw-npu 1.32.0, intel-driver-compiler-npu 1.32.0
  • Also tested with: intel-level-zero-npu 1.28.0 — same segfault
  • Standalone openvino 2026.1.0: installed alongside ORT — no effect (ORT loads its bundled libs)

Reproduction

import onnxruntime as ort

# This segfaults:
sess = ort.InferenceSession(
    "encoder_model.onnx",
    providers=["OpenVINOExecutionProvider"]
)

# This works:
sess = ort.InferenceSession(
    "encoder_model.onnx",
    providers=["CPUExecutionProvider"]
)

The model is Moonshine Base encoder (~78MB ONNX). Any ONNX model triggers the crash — it happens during EP initialization, not model loading.

Crash Details

signal=SEGV (status=11)
segfault at 0 ip <offset 0xcdd56c> in libopenvino.so.2541
error 6 (write to NULL pointer in userspace)
  • Crash is 100% reproducible, always at the same offset in libopenvino.so.2541
  • dmesg on host confirms: python3[PID]: segfault at 0 ip ...cdd56c in libopenvino.so.2541

What We've Tried

  1. ✅ Upgraded NPU userspace from 1.28.0 → 1.32.0 (latest from intel/linux-npu-driver) — still segfaults
  2. ✅ Installed standalone openvino==2026.1.0 in the venv — ORT still loads its bundled libopenvino.so.2541, segfault persists
  3. ✅ Confirmed /dev/accel0 exists and intel_vpu module is loaded
  4. ldd shows no missing libraries
  5. ✅ CPUExecutionProvider works perfectly as a workaround

Expected Behavior

OpenVINOExecutionProvider should either:

  1. Successfully initialize and use the NPU, or
  2. Fail gracefully with an error message instead of segfaulting

Request

  • Is there a known incompatibility between ORT-OpenVINO 1.24.1's bundled OpenVINO 2025.4.1 and the intel_vpu driver in kernel 6.17.x?
  • Will the next onnxruntime-openvino release bundle a newer OpenVINO (2026.x) that may resolve this?
  • Is there a way to force ORT to use a system-installed OpenVINO instead of its bundled libopenvino.so.2541?

Workaround

Using CPUExecutionProvider — encoder inference is 88ms on CPU vs ~50ms on NPU. Acceptable but we'd prefer NPU when available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions