Replica changes on an admitted, running InferenceService currently trigger a full suspend/requeue bounce: the PodSet count comes from spec.replicas, so scaling 2 to 3 makes the job non-equivalent to its Workload, and the jobframework suspends the service, deletes the old Workload, creates a new one, and re-admits. Scale-to-zero is graceful (Workload deactivation), but N to M with both nonzero is a routine serving operation that becomes a service interruption.
Kueue's mechanism for this is elastic workloads / workload slices (ElasticJobsViaWorkloadSlices, alpha as of v0.19.0, plus the kueue.x-k8s.io/elastic-job annotation and the ElasticWorkloadNameProvider hook). Adopt it once it matures, or document a resize procedure until then.
Until fixed this is a documented limitation (see the issue #2 PR body).
Replica changes on an admitted, running InferenceService currently trigger a full suspend/requeue bounce: the PodSet count comes from spec.replicas, so scaling 2 to 3 makes the job non-equivalent to its Workload, and the jobframework suspends the service, deletes the old Workload, creates a new one, and re-admits. Scale-to-zero is graceful (Workload deactivation), but N to M with both nonzero is a routine serving operation that becomes a service interruption.
Kueue's mechanism for this is elastic workloads / workload slices (
ElasticJobsViaWorkloadSlices, alpha as of v0.19.0, plus thekueue.x-k8s.io/elastic-jobannotation and theElasticWorkloadNameProviderhook). Adopt it once it matures, or document a resize procedure until then.Until fixed this is a documented limitation (see the issue #2 PR body).