I want run ollama with docker-compose and using nvidia-gpu. What should I write in the docker-compose.yml file?
I run ollama with docker-compose, but gpu was not been used, this is what i write:
ollama:
container_name: ollama
image: ollama/ollama:rocm
ports:
- 11434:11434
volumes:
- ollama:/root/.ollama
networks:
- fastgpt
restart: always
I need a docker-compose.yaml file example.
I'm assuming that you have the GPU configured and that you can successfully execute nvidia-smi.
If do then you can adapt your docker-compose.yml as follows:
version: "3.9"
services:
ollama:
container_name: ollama
image: ollama/ollama:rocm
deploy:
resources:
reservations:
devices:
- driver: nvidia
capabilities: ["gpu"]
count: all
volumes:
- ollama:/root/.ollama
restart: always
volumes:
ollama:
When you bring it up you should see "Nvidia GPU detected via cudart" in the logs.
