How to pin model version ? Models keep updating.

[rnik12]

Can we add 'revision' number?

https://discuss.huggingface.co/t/how-to-properly-handle-model-versions/24108

[codingjaguar]

@rnik12 Thanks for the FR. I think it makes sense to add. It feels a very advanced use case of milvus model lib, which is mostly designed for developers learning and prototyping vector search applications. Curious, what is your use case and which specific model and revision would you like to use?

@junjiejiangjjj can you help check if that's feasible to add this feature?

[rnik12]

@codingjaguar I use the Milvus Model Library in our FastAPI production backend to convert text into vectors before performing a hybrid search in Milvus. I’ve found that it's best to stick with the Milvus Model Library rather than using plain Transformers or AutoTransformers packages. This is because the Milvus Model Library applies several useful default settings that enhance performance compared to the vanilla models. For example, it includes built-in normalization, increases the maximum token limit, and offers several other optimizations.

[codingjaguar]

@rnik12 This is really interesting. Thanks for sharing your use case! We were under the impression that most users choose it over plain Transformers in prototyping for its simplicity and easy to ramp up, then pick up inference frameworks like TritonRT or embedding API service for production. I'm really happy that its wrapping of Transformers is also useful in production setups! That being said, for production at scale, using a dedicated inference server on GPU can largely improve the efficiency, thus we are working on a new project in Milvus to open source the inference API server containers for popular embedding models. If you would like to sign up for trying that out early, please reach out to me jiang.chen@zilliz.com and I'm happy to share more details!

[junjiejiangjjj]

@rnik12 Thanks for the FR. I think it makes sense to add. It feels a very advanced use case of milvus model lib, which is mostly designed for developers learning and prototyping vector search applications. Curious, what is your use case and which specific model and revision would you like to use?

@junjiejiangjjj can you help check if that's feasible to add this feature?

Models implemented using transformers and sentence-transformers support the revision parameter

[codingjaguar]

@rnik12 Thanks for the FR. I think it makes sense to add. It feels a very advanced use case of milvus model lib, which is mostly designed for developers learning and prototyping vector search applications. Curious, what is your use case and which specific model and revision would you like to use?
@junjiejiangjjj can you help check if that's feasible to add this feature?

Models implemented using transformers and sentence-transformers support the revision parameter

@junjiejiangjjj can you give a short code snippet showing how users can include revision parameter?

[junjiejiangjjj]

@rnik12 Thanks for the FR. I think it makes sense to add. It feels a very advanced use case of milvus model lib, which is mostly designed for developers learning and prototyping vector search applications. Curious, what is your use case and which specific model and revision would you like to use?
@junjiejiangjjj can you help check if that's feasible to add this feature?

Models implemented using transformers and sentence-transformers support the revision parameter

@junjiejiangjjj can you give a short code snippet showing how users can include revision parameter?

Similar to this

from pymilvus.model.dense import SentenceTransformerEmbeddingFunction
ef_sentence_transformer = SentenceTransformerEmbeddingFunction(revision="main")