Segmentation fault with Vulkan

[thomas-0816]

I'm using:

git clone --depth=1 https://github.com/predict-woo/qwen3-tts.cpp.git
cd qwen3-tts.cpp
git submodule update --init --recursive
cmake -S ggml -B ggml/build -DGGML_VULKAN=1
cmake --build ggml/build -j8
cmake -S . -B build -DCMAKE_POSITION_INDEPENDENT_CODE=ON
cmake --build build -j8
mkdir models; cd models
wget https://huggingface.co/koboldcpp/tts/resolve/main/qwen3-tts-0.6b-f16.gguf
wget https://huggingface.co/koboldcpp/tts/resolve/main/qwen3-tts-tokenizer-f16.gguf

./build/qwen3-tts-cli -l de --threads 8 -m models -r examples/readme_clone_input.wav -t "Test 123" -o output.wav

output:

Loading models from: models
  [mem] load/start               rss=11.29 MB  phys=11.29 MB
Loading TTS model from models/qwen3-tts-0.6b-f16.gguf...
  Text tokenizer loaded: vocab_size=151936 (196 ms)
  [mem] load/after-tokenizer     rss=55.49 MB  phys=55.49 MB
  Speaker encoder: deferred (lazy load)
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 780M Graphics (RADV PHOENIX) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
  TTSTransformer backend: Vulkan0
  TTS transformer loaded: hidden_size=1024, n_layers=28 (912 ms)
  [mem] load/after-transformer   rss=748.44 MB  phys=748.44 MB
Loading vocoder from models/qwen3-tts-tokenizer-f16.gguf...
Segmentation fault

Building "cmake -S ggml -B ggml/build" without GGML_VULKAN=1 creates the output.wav successfully, no segfault.

System: Ubuntu 24.04, Kernel 6.17, AMD Ryzen 7 PRO 7840U

Vulkan Packages from ppa:

deb https://packages.lunarg.com/vulkan noble main
deb-src https://packages.lunarg.com/vulkan noble main

Installed packages:

dpkg --list | grep -i vulkan | grep lunarg
ii  libvulkan-dev:amd64                            1.4.313.0~rc1-1lunarg24.04-1                amd64        This package contains the Vulkan loader
ii  libvulkan1:amd64                               1.4.313.0~rc1-1lunarg24.04-1                amd64        This package contains the Vulkan loader
ii  lunarg-via                                     1.4.313.0~rc2-1lunarg24.04-1                amd64        LunarG Vulkan Installation Analyzer
ii  lunarg-vkconfig                                1.4.313.0~rc2-1lunarg24.04-1                amd64        LunarG GUI tool to configure Vulkan layers
ii  lunarg-vulkan-layers                           1.4.313.0~rc2-1lunarg24.04-1                amd64        Extra Vulkan development layers from LunarG
ii  vma                                            3.2.1~rc1-1lunarg24.04-1                    amd64        Easy to integrate Vulkan memory allocation library.
ii  volk                                           1.4.313.0~rc1-1lunarg24.04-1                amd64        Volk is a meta-loader for Vulkan. It allows
ii  vulkan-extensionlayer                          1.4.313.0~rc1-1lunarg24.04-1                amd64        Layers providing Vulkan features when native support is unavailable
ii  vulkan-headers                                 1.4.313.0~rc1-1lunarg24.04-1                all          Vulkan header files and API registry
ii  vulkan-profiles                                1.4.313.0~rc1-1lunarg24.04-1                amd64        Collection of tools to leverage Vulkan Profiles
ii  vulkan-sdk                                     1.4.313.0~rc3-1lunarg24.04-1                all          Tools and runtime for Vulkan development.The Vulkan
ii  vulkan-tools                                   1.4.313.0~rc1-1lunarg24.04-1                amd64        This project provides Vulkan tools and utilities that can
ii  vulkan-utility-libraries                       1.4.313.0~rc1-1lunarg24.04-1                amd64        Utility Libraries for various
ii  vulkan-utility-libraries-dev                   1.4.313.0~rc1-1lunarg24.04-1                amd64        Utility Libraries for various
ii  vulkan-validationlayers                        1.4.313.0~rc2-1lunarg24.04-1                amd64        Vulkan is an Explicit API, enabling direct control over how GPUs actually
ii  vulkancapsviewer                               4.01~rc1-1lunarg24.04-1                     amd64        Client application to display

[madushan1000]

It comes from this line

qwen3-tts.cpp/src/audio_tokenizer_decoder.cpp

Line 334 in 7a762e2

ggml_backend_tensor_set(t, t->data, 0, ggml_nbytes(t));

not sure why.

If I comment it out, It works, but not great.

ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not valid
ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not valid
ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not val

Looks like it's probably much slower than it should be, so I guess this needs to fixed some other way

[Naster17]

Also for me:

./build/qwen3-tts-cli -m models -r examples/readme_clone_input.wav -t "test test" -o examples/readme_example_clone.wav
Loading models from: models
  [mem] load/start               rss=11.68 MB  phys=11.68 MB
Loading TTS model from models/qwen3-tts-0.6b-f16.gguf...
  Text tokenizer loaded: vocab_size=151936 (556 ms)
  [mem] load/after-tokenizer     rss=55.77 MB  phys=55.77 MB
  Speaker encoder: deferred (lazy load)
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD BC-250 (RADV GFX1013) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 0 | matrix cores: none
  TTSTransformer backend: Vulkan0
  TTS transformer loaded: hidden_size=1024, n_layers=28 (929 ms)
  [mem] load/after-transformer   rss=710.18 MB  phys=710.18 MB
Loading vocoder from models/qwen3-tts-tokenizer-f16.gguf...
[1]    17227 segmentation fault (core dumped)  ./build/qwen3-tts-cli -m models -r examples/readme_clone_input.wav -t  -o

[SiaoZeng]

Root cause found and fix posted in #20 — the segfault is caused by normalize_codebooks() writing directly into Vulkan device-mapped tensor->data. Fix: download→normalize on host→upload back.

[thomas-0816]

Tested with https://github.com/SiaoZeng/qwen3-tts.cpp working, thank you!

Tested with:

git clone --depth=1 https://github.com/SiaoZeng/qwen3-tts.cpp
cd qwen3-tts.cpp
git submodule update --init --recursive
cmake -S ggml -B ggml/build -DGGML_VULKAN=1
cmake --build ggml/build -j8
cmake -S . -B build -DCMAKE_POSITION_INDEPENDENT_CODE=ON
cmake --build build -j8
mkdir models; cd models
wget https://huggingface.co/koboldcpp/tts/resolve/main/qwen3-tts-0.6b-f16.gguf
wget https://huggingface.co/koboldcpp/tts/resolve/main/qwen3-tts-tokenizer-f16.gguf

./build/qwen3-tts-cli -l de --threads 8 -m models -o output.wav -r source.wav -t "Test 123"
Loading models from: models
  [mem] load/start               rss=11.64 MB  phys=11.64 MB
Loading TTS model from models/qwen3-tts-0.6b-f16.gguf...
  Text tokenizer loaded: vocab_size=151936 (195 ms)
  [mem] load/after-tokenizer     rss=55.75 MB  phys=55.75 MB
  Speaker encoder: deferred (lazy load)
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 780M Graphics (RADV PHOENIX) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
  TTSTransformer backend: Vulkan0
  TTS transformer loaded: hidden_size=1024, n_layers=28 (1383 ms)
  [mem] load/after-transformer   rss=749.73 MB  phys=749.73 MB
Loading vocoder from models/qwen3-tts-tokenizer-f16.gguf...
  AudioTokenizerDecoder backend: Vulkan0
  Vocoder loaded: sample_rate=24000, n_codebooks=16 (161 ms)
  [mem] load/after-vocoder       rss=749.73 MB  phys=749.73 MB
All models loaded in 1741 ms
  [mem] load/end                 rss=749.73 MB  phys=749.73 MB
Synthesizing with voice cloning: "Test 123"
Reference audio: source.wav
  AudioTokenizerEncoder backend: Vulkan0
  Speaker encoder lazy-loaded in 39 ms
  [mem] voice/after-encoder-load rss=749.73 MB  phys=749.73 MB
  [mem] synth/start              rss=749.73 MB  phys=749.73 MB
  [mem] synth/after-tokenize     rss=749.73 MB  phys=749.73 MB
  [mem] synth/after-generate     rss=749.73 MB  phys=749.73 MB
  [mem] synth/after-decode       rss=749.73 MB  phys=749.73 MB
  [mem] synth/end                rss=749.73 MB  phys=749.73 MB

Timing:
  Tokenization:    0 ms
  Speaker encode:  22548 ms
  Code generation: 3042 ms
  Vocoder decode:  1095 ms
  Total:           4137 ms
  Audio duration:  2.30 s
  Throughput:      0.56x realtime (RTF=1.801)

Memory:
  RSS start/end:   749.73 MB -> 749.73 MB
  RSS peak:        749.73 MB
  Phys start/end:  749.73 MB -> 749.73 MB
  Phys peak:       749.73 MB

Output saved to: output.wav
Audio duration: 2.30 seconds

Timing:
  Load:           0 ms
  Tokenize:       0 ms
  Encode:     22548 ms
  Generate:    3042 ms
  Decode:      1095 ms
  Total:       4137 ms

[SiaoZeng]

spend my fork a star pleas ^^

[KLL535]

For CUDA similar error:

ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not valid
ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not valid
ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_gallocr_needs_realloc: node code_pred_v_cache_0 (view) (permuted) (transposed) (cont) is not valid
ggml_gallocr_alloc_graph: cannot reallocate multi buffer graph automatically, call reserve
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 0)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)
ggml_backend_sched_alloc_splits: failed to allocate graph, reserving (backend_ids_changed = 1)

Update:
It was all about the compilation falags, we need to make a release version.

call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
cd H:\src\qwen3-tts.cpp\ggml
cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DGGML_CUDA=ON
cmake --build build --config Release -j8

cmake -S . -B build -G "Visual Studio 17 2022" -A x64 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CXX_FLAGS="/D_USE_MATH_DEFINES /EHsc /O2 /Ob2 /DNDEBUG"
cmake --build build --config Release -j8
set PATH=%PATH%;H:\src\qwen3-tts.cpp\ggml\build\bin\Release

Code changes:

qwen3_tts.cpp (Plugs for Windows)

#ifndef _WIN32
#include <sys/resource.h>
#endif
...
#elif defined(_WIN32)
    out.rss_bytes = 0;
    out.phys_footprint_bytes = 0;
    return true;
#else
    struct rusage usage = {};
    if (getrusage(RUSAGE_SELF, &usage) != 0) {
        return false;
    }
    out.rss_bytes = (uint64_t) usage.ru_maxrss * 1024ULL;
    out.phys_footprint_bytes = out.rss_bytes;
    return true;
#endif

main.cpp (Russian language support, without these changes Russian text was distorted)

#ifdef _WIN32
#include <windows.h>
std::string wstr_to_utf8(const std::wstring& wstr) {
    if (wstr.empty()) return std::string();
    int size_needed = WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), NULL, 0, NULL, NULL);
    std::string strTo(size_needed, 0);
    WideCharToMultiByte(CP_UTF8, 0, &wstr[0], (int)wstr.size(), &strTo[0], size_needed, NULL, NULL);
    return strTo;
}
#endif
...
int main(int argc, char ** argv) {

#ifdef _WIN32
    //UTF-16
    int wargc;
    wchar_t** wargv = CommandLineToArgvW(GetCommandLineW(), &wargc);
    std::vector<std::string> args_utf8;
    for (int i = 0; i < wargc; ++i) {
        args_utf8.push_back(wstr_to_utf8(wargv[i]));
    }
    LocalFree(wargv);

    //UTF-8
    std::vector<char*> argv_p(wargc);
    for (int i = 0; i < wargc; ++i) {
        argv_p[i] = &args_utf8[i][0];
    }
    argv = argv_p.data();
    argc = wargc;

    SetConsoleCP(CP_UTF8);
    SetConsoleOutputCP(CP_UTF8);
#endif

audio_tokenizer_decoder.cpp (The last true to false)

state_.sched = ggml_backend_sched_new(backends.data(), nullptr, (int)backends.size(), QWEN3_TTS_DEC_MAX_NODES, false, false);

What doesn't work:

1. No other models work except qwen3-tts-0.6b-f16.gguf
Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf - not work
   Qwen3-TTS-12Hz-1.7B-CustomVoice-Q8_0.gguf - not work
2. Vocoder runs on the CPU (slow)
3. The Russian accents are incorrect, but as I understand it, this is a problem with the entire model and cannot be fixed in any way.