Bad performance on 12th gen iGPU

[flyize]

I'm pretty sure that I've done something wrong, because the Wyoming addon I use in Home Assistant only takes a few seconds, and this takes like 20. I'm running a 12th gen Alder Lake. How do I verify its actually working?

[tannisroot]

Some logs from the whisper container would help

[tannisroot]

As well as system specs

[flyize]

whisper-cpp container logs: https://pastebin.com/iJ5JFwyp
wyoming-api container logs: https://pastebin.com/MyBgHnx4

Server is running Unraid 7 with 78GB of RAM.

[flyize]

Also, debug screenshot from Home Assistant.

edit: And FWIW, if I just run it through the CPU via the Home Assistant whisper addon, it processes in a couple of seconds.

Hi!

Same situation here, OPENVINO also shows as = 0 when I check the whisper-cpp container logs.

The converted model files are called the same than the original + "-encoder-openvino" (both files bin and xml) and they are placed in the same folder than the original.

Tried "--ov-v-device GPU" too, within the command param at the docker-compose.yml file too but it changes nothing.

Maybe something else is missing?

I'm running a N100/32GB RAM in an Alpine Linux LXC container in Proxmox.

[tannisroot]

Also, debug screenshot from Home Assistant.

edit: And FWIW, if I just run it through the CPU via the Home Assistant whisper addon, it processes in a couple of seconds.

Hm, tbh it's possible the GPU on that Intel CPU is just really bad or badly optimized for oneapi... I've only ever tested it with a discrete Arc A380. But it's definitely using it and detecting it according to logs.
I would maybe try changing the base image to a newer version, see tags here https://hub.docker.com/r/intel/oneapi/tags

[tannisroot]

Hi!

Same situation here, OPENVINO also shows as = 0 when I check the whisper-cpp container logs.

The converted model files are called the same than the original + "-encoder-openvino" (both files bin and xml) and they are placed in the same folder than the original.

Tried "--ov-v-device GPU" too, within the command param at the docker-compose.yml file too but it changes nothing.

Maybe something else is missing?

I'm running a N100/32GB RAM in an Alpine Linux LXC container in Proxmox.

This doesn't use Openvino.

Hi!
Same situation here, OPENVINO also shows as = 0 when I check the whisper-cpp container logs.
The converted model files are called the same than the original + "-encoder-openvino" (both files bin and xml) and they are placed in the same folder than the original.
Tried "--ov-v-device GPU" too, within the command param at the docker-compose.yml file too but it changes nothing.
Maybe something else is missing?
I'm running a N100/32GB RAM in an Alpine Linux LXC container in Proxmox.

This doesn't use Openvino.

Oh my bad, I thought it was compiled to use OpenVino lol

Thank you for taking time to answer!

[flyize]

Also, debug screenshot from Home Assistant.

edit: And FWIW, if I just run it through the CPU via the Home Assistant whisper addon, it processes in a couple of seconds.

Hm, tbh it's possible the GPU on that Intel CPU is just really bad or badly optimized for oneapi... I've only ever tested it with a discrete Arc A380. But it's definitely using it and detecting it according to logs. I would maybe try changing the base image to a newer version, see tags here https://hub.docker.com/r/intel/oneapi/tags

I used the :latest tag and got this

[tannisroot]

Did you rebuild the image after changing the tag?

[flyize]

I did.

[baurmatt]

Same problem here. Running on an Intel i3-N305 with 32GB RAM using whisper-cpp 1.8.2 and intel/oneapi 2025.3.0-0-devel-ubuntu24.04

whisper_init_from_file_with_params_no_state: loading model from '/models/ggml-large-v2.bin'
whisper_init_with_params_no_state: use gpu    = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw        = 0
whisper_init_with_params_no_state: devices    = 2
whisper_init_with_params_no_state: backends   = 2
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1280
whisper_model_load: n_audio_head  = 20
whisper_model_load: n_audio_layer = 32
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1280
whisper_model_load: n_text_head   = 20
whisper_model_load: n_text_layer  = 32
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 5 (large)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:        SYCL0 total size =  3093.99 MB
whisper_model_load: model size    = 3093.99 MB
whisper_backend_init_gpu: using SYCL0 backend
Running with Environment Variables:
  GGML_SYCL_DEBUG: 0
  GGML_SYCL_DISABLE_OPT: 0
  GGML_SYCL_DISABLE_GRAPH: 1
  GGML_SYCL_DISABLE_DNN: 0
  GGML_SYCL_PRIORITIZE_DMMV: 0
Build with Macros:
  GGML_SYCL_FORCE_MMQ: no
  GGML_SYCL_F16: yes
Found 1 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|                     Intel UHD Graphics|   12.4|     32|     512|   32| 30912M|         1.6.33578+15|
SYCL Optimization Feature:
|ID|        Device Type|Reorder|
|--|-------------------|-------|
| 0| [level_zero:gpu:0]|      Y|
whisper_init_state: kv self size  =   83.89 MB
whisper_init_state: kv cross size =  251.66 MB
whisper_init_state: kv pad  size  =    7.86 MB
whisper_init_state: compute buffer (conv)   =   35.67 MB
whisper_init_state: compute buffer (encode) =   78.57 MB
whisper_init_state: compute buffer (cross)  =    9.27 MB
whisper_init_state: compute buffer (decode) =  112.36 MB

system_info: n_threads = 4 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | OPENMP = 1 | REPACK = 1 |

operator(): processing 'upload' (48640 samples, 3.0 sec), 4 threads, 1 processors, lang = de, task = transcribe, timestamps = 1 ...


system_info: n_threads = 4 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | OPENMP = 1 | REPACK = 1 |

operator(): processing 'upload' (59680 samples, 3.7 sec), 4 threads, 1 processors, lang = de, task = transcribe, timestamps = 1 ...


system_info: n_threads = 4 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | OPENMP = 1 | REPACK = 1 |

operator(): processing 'upload' (54720 samples, 3.4 sec), 4 threads, 1 processors, lang = de, task = transcribe, timestamps = 1 ...