Support CUDA 13 / RTX 5070 Ti

[Toluwani5778]

Hi Kokoro Team,

Currently, the Docker image uses CUDA 12.8. My RTX 5070 Ti requires CUDA 13, and PyTorch fails to load models with CUDA errors.
I tested a modified Dockerfile based on CUDA 13.0.1 + cuDNN and installing PyTorch with CUDA 13 wheels, and the model works correctly on newer GPUs.

Proposed changes:

• Update base image: nvcr.io/nvidia/cuda:13.0.1-cudnn-devel-ubuntu24.04
• Ensure PyTorch installs the CUDA 13 wheel: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu13

This allows support for the latest GPUs (like RTX 5070 Ti) while keeping compatibility with the existing stack.

Thank you!

The modified dockerfile found at
docker/gpu/Dockerfile

# Use CUDA 13 base with cuDNN, Ubuntu 24.04
FROM --platform=$BUILDPLATFORM nvcr.io/nvidia/cuda:13.0.1-cudnn-devel-ubuntu24.04

# Enable universe repository and install dependencies
# Install dependencies
RUN apt-get update -y && \
    apt-get install -y software-properties-common && \
    add-apt-repository universe && \
    apt-get update -y && \
    apt-get install -y \
        python3 python3-venv python3-pip \
        espeak-ng espeak-ng-data \
        git libsndfile1 curl ffmpeg g++ cmake && \
    apt-get clean && rm -rf /var/lib/apt/lists/* && \
    mkdir -p /usr/share/espeak-ng-data && \
    ln -s /usr/lib/*/espeak-ng-data/* /usr/share/espeak-ng-data/ && \
    curl -LsSf https://astral.sh/uv/install.sh | sh && \
    mv /root/.local/bin/uv /usr/local/bin/ && \
    mv /root/.local/bin/uvx /usr/local/bin/ && \
    useradd -m -u 1001 appuser && \
    mkdir -p /app/api/src/models/v1_0 && \
    chown -R appuser:appuser /app

USER appuser
WORKDIR /app

# Copy dependency files
COPY --chown=appuser:appuser pyproject.toml ./pyproject.toml

# Install dependencies with GPU extras using system Python 3
RUN uv venv --python python3 && \
    uv sync --extra gpu --no-cache

# Copy project files including models
COPY --chown=appuser:appuser api ./api
COPY --chown=appuser:appuser web ./web
COPY --chown=appuser:appuser docker/scripts/ ./
RUN chmod +x ./entrypoint.sh

# Set environment variables
ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app:/app/api \
    UV_LINK_MODE=copy \
    USE_GPU=true \
    PHONEMIZER_ESPEAK_PATH=/usr/bin \
    PHONEMIZER_ESPEAK_DATA=/usr/share/espeak-ng-data \
    ESPEAK_DATA_PATH=/usr/share/espeak-ng-data \
    DEVICE="gpu"

ENV DOWNLOAD_MODEL=true

# Download model if enabled
RUN if [ "$DOWNLOAD_MODEL" = "true" ]; then \
    python download_model.py --output api/src/models/v1_0; \
    fi

# Run FastAPI server through entrypoint.sh
CMD ["./entrypoint.sh"]

[digitalassassins]

I posted on this before, you can manually change the dependencies in the Composer build file to pull newer versions of PyTorch & CUDA..

Might be worth testing with the newer PyTorch and CUDA versions on your machine/docker container, see if it still works:

#320

The Dev should think about adding an environment variable that has the PyTorch version and Cuda version.
Then we can just set an Env variable in our Docker Compose.. That way, we can tune the package to suit our needs from the main image without having to build all the different versions for each of our graphics cards.

The Entrypoint should check the PyTorch version in ENV against what was last installed and saved into a txt file or env file inside the container. If it differs, it then removes CUDA, PyTorch, and all dependencies using pip, then reinstalls the version stated in the Env file.

• CUDA_INSTALL=13.0.1
• PYTORCH_INSTALL=cu13

This way, it can support all generations of NVIDIA cards with a simple ENV variable change.