Uses official ollama testcontainers

[Martin7-1]

Closes langchain4j#2101

[langchain4j]

@Martin7-1 thank you! I am wondering if there is a way to make it faster? Right now it takes around 2 minutes to download the (quite small) phi model, and it happens on every test run :(

[Martin7-1]

@langchain4j I think the best way right now is to cache docker image (like langchain4j-ollama do, pull model and commit to new docker image). WDYT?

[langchain4j]

@Martin7-1 do you mean revert to the existing logic?

[Martin7-1]

@langchain4j Yes. What we do now is just migrating from GenericContainer to OllamaContainer. But we can still have our own private docker image for testing. WDYT?

And maybe we need to update docker image in docker hub... Looks like it updated 5 months ago and it's too old.

[langchain4j]

@Martin7-1 you're right! We should probably revive https://hub.docker.com/search?q=langchain4j
But I am surprised ollama does not provide official images for most popular models yet

[Martin7-1]

But I am surprised ollama does not provide official images for most popular models yet

@langchain4j Maybe they just want to manage the platform itself :).

BTW, I found another Ollama image which just support CPU version: https://hub.docker.com/r/alpine/ollama. Maybe it's enough for us as I think our tests are all run on CPU (Github Actions or locally).

This image is just ~70MB compared to the original Ollama image is ~4GB.

You can refer to my latest test code and run it. I think is faster and maybe we do not need to push it to docker hub anymore?

[Martin7-1]

And, all Ollama models are stored at ~/.ollama, what about caching it and mounting it to container? It will reduce the time Ollama pull model.

[langchain4j]

BTW, I found another Ollama image which just support CPU version: https://hub.docker.com/r/alpine/ollama. Maybe it's enough for us as I think our tests are all run on CPU (Github Actions or locally).7
This image is just ~70MB compared to the original Ollama image is ~4GB.

You can refer to my latest test code and run it. I think is faster and maybe we do not need to push it to docker hub anymore?

Sounds interesting!

And, all Ollama models are stored at ~/.ollama, what about caching it and mounting it to container? It will reduce the time Ollama pull model.

This will work for local env, but not for running on github CI, right?

It seems that the slowest part is downloading the model from the ollama hub. And it seems that downloading container with backed-in model from docker hub is faster? IDK, just my feeling.

[Martin7-1]

This will work for local env, but not for running on github CI, right?

Maybe we can use actions/cache@v3 to cache ~/.ollama in Github Actions. But Github Actions can only cache 10GB files...

It seems that the slowest part is downloading the model from the ollama hub. And it seems that downloading container with backed-in model from docker hub is faster? IDK, just my feeling.

Maybe integrate them (alpine/ollama and ~/.ollama cache) will make it faster. But huge model size will execeed Github Actions cache limit I thinik...

[langchain4j]

Maybe we can use actions/cache@v3 to cache ~/.ollama in Github Actions. But Github Actions can only cache 10GB files...

Does it actually work? In my experience, caching (e.g. maven cache) does not really work on github actions. Or maybe I am doing something wrong

[langchain4j]

If you have a bit of time and want to spend it on this, I would compare download speeds from docker and from ollama hubs and calculate which option is faster :)

[Martin7-1]

Does it actually work? In my experience, caching (e.g. maven cache) does not really work on github actions. Or maybe I am doing something wrong

Hmmm... Let me test it.

If you have a bit of time and want to spend it on this, I would compare download speeds from docker and from ollama hubs and calculate which option is faster :)

Thank you! I think I will focus on it recently as langchain4j-ollama is an important part of the whole project. And ensure the integration test (locally or github actions) is indeed importatnt.

I will check whether Github Actions cache works or not.

[langchain4j]

@Martin7-1 thank you so much for your help! ❤️