Is your feature request related to a problem? Please describe.
Memory usage can have a significant disk footprint when SotA embedding models are used, such as text-embedding-3-large (3072 dimensions) or Qwen3-Embedding-8B (4096 dimensions), while some other models reach 8192 dimensions.
Describe the solution you'd like
I want to benefit from LanceDB's implementation of the RaBitQ data quantization that could dramatically reduce disk footprint by sacrificing a small portion of recall accuracy.
Describe alternatives you've considered
Use embedding models with low dimensionality.
Additional context
https://www.lancedb.com/blog/feature-rabitq-quantization
Is your feature request related to a problem? Please describe.
Memory usage can have a significant disk footprint when SotA embedding models are used, such as text-embedding-3-large (3072 dimensions) or Qwen3-Embedding-8B (4096 dimensions), while some other models reach 8192 dimensions.
Describe the solution you'd like
I want to benefit from LanceDB's implementation of the RaBitQ data quantization that could dramatically reduce disk footprint by sacrificing a small portion of recall accuracy.
Describe alternatives you've considered
Use embedding models with low dimensionality.
Additional context
https://www.lancedb.com/blog/feature-rabitq-quantization