|
| 1 | +# Upgrading Kubeflow Manifests |
| 2 | + |
| 3 | +This document describes breaking changes and manual upgrade steps required when moving between versions of Kubeflow manifests. |
| 4 | + |
| 5 | +## Upgrading KServe from v0.16.x to v0.17.0 |
| 6 | + |
| 7 | +This section covers the upgrade of KServe to v0.17.0, which is part of the latest synchronization with upstream manifests. |
| 8 | + |
| 9 | +> **Note:** These instructions apply only to Kubeflow Manifests revisions where the KServe component in the [README component matrix](README.md#kubeflow-components-versions) is listed as v0.17.0. If your README entry still shows KServe v0.16.x, defer this section until you have applied the companion manifests upgrade (see [PR #3406](https://github.com/kubeflow/manifests/pull/3406)). |
| 10 | +
|
| 11 | +### 1. Pre-upgrade Step: Delete Conflicting ClusterRoleBinding (Breaking Change) |
| 12 | + |
| 13 | +Before applying the new manifests, you **must** manually delete the following ClusterRoleBinding: |
| 14 | + |
| 15 | +```bash |
| 16 | +kubectl delete clusterrolebinding llmisvc-manager-rolebinding --ignore-not-found |
| 17 | +``` |
| 18 | + |
| 19 | +**Why this is required:** |
| 20 | +KServe v0.17.0 restructured the `llmisvc` role bindings. The old `ClusterRoleBinding` from v0.16.x is incompatible with the new one defined in the v0.17.0 manifests. If you do not delete it manually, `kubectl apply` (or `kustomize build | kubectl apply`) will fail because Kubernetes does not allow in-place updates to immutable fields on an existing `ClusterRoleBinding` (such as `roleRef` or `subjects`). |
| 21 | + |
| 22 | +### 2. Note on the v0.17.0 Tag and Manifest Source |
| 23 | + |
| 24 | +The official `v0.17.0` tag in the upstream [kserve/kserve](https://github.com/kserve/kserve) repository was released without the necessary `install` files (see [kserve/kserve#5255](https://github.com/kserve/kserve/issues/5255)). |
| 25 | + |
| 26 | +To ensure a working installation, the manifests in this repository were synchronized manually from the upstream **`release-0.17`** branch. The exact files vendored into this repository are tracked in [PR #3406](https://github.com/kubeflow/manifests/pull/3406) (commit `5b2604281da8512bb97bb170a796b6b2b3726385`), which serves as the auditable reference point for this sync. Users should be aware of this distinction when cross-referencing upstream source code or release notes. |
| 27 | + |
| 28 | +### 3. Webhook Certificate SAN Fix |
| 29 | + |
| 30 | +KServe v0.17.0 introduced duplicate `Certificate` resources that continued to reference the upstream `kserve` namespace for Subject Alternative Names (SANs) instead of the `kubeflow` namespace used in this distribution, causing TLS verification failures on the webhook server. |
| 31 | + |
| 32 | +The manifests in this repository have been patched to correct SANs for the following resources: |
| 33 | +- `serving-cert` |
| 34 | +- `llmisvc-serving-cert` |
| 35 | +- `localmodel-serving-cert` |
| 36 | + |
| 37 | +If you encounter TLS verification failures after upgrading, ensure you are using the manifests from this repository and not directly from the upstream KServe repository. |
| 38 | + |
| 39 | +### 4. How to Apply the Upgrade |
| 40 | + |
| 41 | +After performing the pre-upgrade step in Section 1, apply the upgrade using the standard Kustomize pattern. |
| 42 | + |
| 43 | +**Single Command Upgrade:** |
| 44 | +```bash |
| 45 | +while ! kustomize build example | kubectl apply --server-side --force-conflicts -f -; do echo "Retrying to apply resources"; sleep 20; done |
| 46 | +``` |
| 47 | + |
| 48 | +**Individual Component Upgrade (KServe only):** |
| 49 | +```bash |
| 50 | +kustomize build applications/kserve/kserve | kubectl apply -f - |
| 51 | +``` |
0 commit comments