Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I noticed that there is no "cancel" or "stop" button. Even when i delete the task, the query is still running (tested with llama.cpp backend)
Describe the solution you'd like
A clear and concise description of what you want to happen.
Clearly respond by cancelling the LLM request when the user requests task cancellation or deletes the chat from the library.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Restarting the container works, but it's nuclear.
Additional context
Add any other context or screenshots about the feature request here.
Stopping the queries ASAP could save a lot of compute locally. Especially when dealing with multiple users on flaky network (in an aggressive case, if user disconnects or drops, stop the stream, and later the user will press retry if needed)
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I noticed that there is no "cancel" or "stop" button. Even when i delete the task, the query is still running (tested with llama.cpp backend)
Describe the solution you'd like
A clear and concise description of what you want to happen.
Clearly respond by cancelling the LLM request when the user requests task cancellation or deletes the chat from the library.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Restarting the container works, but it's nuclear.
Additional context
Add any other context or screenshots about the feature request here.
Stopping the queries ASAP could save a lot of compute locally. Especially when dealing with multiple users on flaky network (in an aggressive case, if user disconnects or drops, stop the stream, and later the user will press retry if needed)