It would be nice if queries could be executed in parallel. A simple solution is to do it multithreaded. However, for multithreading to be meaningful the GIL needs to be released. I think bm25s is almost there, since it already uses njit from numba heavily. The only remaining thing is to add nogil=True to njit().
https://numba.pydata.org/numba-doc/dev/user/jit.html
I can see that _retrieve_internal_jitted_parallel already does parallel=True, which can be a different way to achieve parallelization. However, this is a batched parallelization. What I'm looking for is a version of _retrieve_internal_jitted_parallel that does a single observation with nogil=True. Then I can run it asynchronously with asyncio.to_thread or similar.
It would be nice if queries could be executed in parallel. A simple solution is to do it multithreaded. However, for multithreading to be meaningful the GIL needs to be released. I think bm25s is almost there, since it already uses
njitfrom numba heavily. The only remaining thing is to addnogil=Truetonjit().https://numba.pydata.org/numba-doc/dev/user/jit.html
I can see that
_retrieve_internal_jitted_parallelalready doesparallel=True, which can be a different way to achieve parallelization. However, this is a batched parallelization. What I'm looking for is a version of_retrieve_internal_jitted_parallelthat does a single observation withnogil=True. Then I can run it asynchronously withasyncio.to_threador similar.