Feature request: Propagate language attribute to whisper

[nadiamoe]

Heya,

Currently it seems like we're not cueing Whisper any particular language in params:

wyoming-whisper-api-client/wyoming_whisper_api_client/handler.py

Lines 61 to 65 in 45382bc

	params = {
	"temperature": "0.0",
	"temperature_inc": "0.2",
	"response_format": "json"
	}

This works okay if only one language is used, as whisper can be configured to assume a certain language for all requests. However, if one wants to have multiple assist pipelines in homeassistant in different languages, this won't work.

Whisper.cpp's --lang auto is in my experience not an option as it is very prone to misdetections.

It would be nice if we could just tell whisper which language homeassistant advertised.

[coderabbitai]

A summary of the changes CodeRabbit can apply:

• Implement language propagation in wyoming_whisper_api_client/handler.py: add a language attribute, extract and store language from Transcribe events, include the language parameter in Whisper API requests when present, add debug logging, and reset the language after each transcription.

• Implement language propagation by adding FEATURE_LANGUAGE_PROPAGATION.md and modifying wyoming_whisper_api_client/handler.py to add a new self.language field, extract and log language from Transcribe events, include the language in whisper.cpp API request parameters when present, and reset the language after each transcription.

• Generated copyable edits - (🔄 Check again to refresh)

[coderabbitai]

Here are the copyable edits based on the planned changes:

Copyable Edits

wyoming_whisper_api_client/handler.py

@@ -78,7 +78,7 @@
             return False
 
         if Transcribe.is_type(event.type):
-            _LOGGER.debug("Transcibe event")
+            _LOGGER.debug("Transcribe event")
             return True
 
         if Describe.is_type(event.type):

• ✅ Create PR with these edits

[coderabbitai]

Here are the copyable edits based on the planned changes:

Copyable Edits

wyoming_whisper_api_client/main.py

@@ -3,8 +3,6 @@
 import asyncio
 import logging
 from functools import partial
-from pathlib import Path
-from typing import Optional
 
 from wyoming.info import AsrModel, AsrProgram, Attribution, Info
 from wyoming.server import AsyncServer
@@ -74,7 +72,6 @@
 
     server = AsyncServer.from_uri(args.uri)
     _LOGGER.info("Ready")
-    model_lock = asyncio.Lock()
     await server.run(
         partial(
             WhisperAPIEventHandler,

wyoming_whisper_api_client/handler.py

@@ -64,7 +64,6 @@
                             "response_format": "json"
                         }
                         r = await client.post(self.cli_args.api, files=files, params=params, timeout=120.0)
-                        #_LOGGER.debug(r.json())
                         text = r.json()['text']
 
             _LOGGER.info(text)
@@ -78,7 +77,7 @@
             return False
 
         if Transcribe.is_type(event.type):
-            _LOGGER.debug("Transcibe event")
+            _LOGGER.debug("Transcribe event")
             return True
 
         if Describe.is_type(event.type):

• ✅ Create PR with these edits

[coderabbitai]

Caution

The CodeRabbit agent's plans did not produce any file changes.

[nikolaykasyanov]

@ser hello 👋 unfortunately I only noticed the recent contribution policy change after having submitted a PR (#13).

I'd like to have a conversation about the matter at hand. I propose the following: the language provided in the Wyoming protocol transcribe request should be passed to whisper.cpp as-is. Implementation-wise it means:

• read the request language
• add it as a parameter to the whisper.cpp call in wyoming_whisper_api_client/handler.py.

Please let me know if you have any questions or yay/nay.

[ser]

I am using now this model:

https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3

which is faster and better than whisper, and, crucially, multilingual. Ergo setting language manually is not necessary in my opinion.