Kokoro is running on CPU in GPU mode

[NonlethalGhost]

I was monitoring the app from the task manger when the app is active. I have an 8GB RTX 3060 Ti and have the processing mode set to GPU, however, there seems to be heavy CPU usage, particularly when every sentence is processed (picture). Cannot pin down the reason.

[revisionhiep-create]

Looked into this. The kokoro_onnx library has broken GPU detection -- it checks for a pip package name instead of the actual importable module, so it always falls back to CPU.

Added auto-detection of available ONNX providers (DirectML, CUDA) in the engine loader. If one is found, we create the session ourselves with both GPU and CPU providers for proper fallback. Also had to convert the Kokoro model from opset 17 to opset 22 because DML's ConvTranspose implementation was updated for opset 22 and errors out on older opsets. The converted model gets cached so it only happens once.

Thread pool reduced to 1 worker for GPU mode (GPU serializes internally) and 2 for CPU mode.

Added onnx and onnxruntime-directml to requirements.txt so new installs get GPU support out of the box.

[revisionhiep-create]

Update: The GPU acceleration changes from v3.6 have been reverted in v3.6.1. While the DirectML session creation, opset 22 conversion, and provider ordering did enable GPU inference, it introduced static noise across all voices — likely caused by certain operations falling back to CPU mid-inference, producing dtype mismatches in the audio pipeline.

The core GPU detection issue in kokoro_onnx remains. We'll revisit this once the upstream library provides a proper fix or we find a way to handle the GPU↔CPU fallback without audio corruption. For now the app runs on CPU mode (which is still fast enough for real-time TTS on modern hardware).

Reopening this as unresolved — the symptom (CPU usage spikes) is expected behavior until a clean GPU solution is available.