Skip to content

fix(manifests): add namespace: kubeflow to kustomization overlays and add namespace regression check#2709

Closed
Raakshass wants to merge 1 commit into
kubeflow:mainfrom
Raakshass:fix-hub-default-namespace
Closed

fix(manifests): add namespace: kubeflow to kustomization overlays and add namespace regression check#2709
Raakshass wants to merge 1 commit into
kubeflow:mainfrom
Raakshass:fix-hub-default-namespace

Conversation

@Raakshass
Copy link
Copy Markdown

@Raakshass Raakshass commented May 14, 2026

Summary

Three of the four kustomization overlays consumed by kubeflow/manifests example/kustomization.yaml were missing the namespace: kubeflow directive. This causes all model-registry, istio networking, and catalog resources to deploy into the default namespace instead of kubeflow when using kustomize build.

This pull request adds namespace: kubeflow to the three affected overlays and introduces a namespace regression check script to prevent recurrence.

Fixes kubeflow/manifests#3457
Supersedes kubeflow/manifests#3458 (closed per maintainer direction to fix upstream)

Root Cause

The CI install script (model_registry_install.sh) passes -n kubeflow to kubectl apply, which overrides the namespace at apply-time. This masked the fact that the kustomization overlays themselves had no namespace directive. When users follow the documented deployment path (kustomize build example/ | kubectl apply -f -), there is no namespace override — all resources land in the default namespace.

This was confirmed by istioctl analyze output showing VirtualService and DestinationRule reference errors in the default namespace.

Changes

File Change
manifests/kustomize/overlays/postgres/kustomization.yaml Add namespace: kubeflow
manifests/kustomize/options/istio/kustomization.yaml Add namespace: kubeflow
manifests/kustomize/options/catalog/overlays/demo/kustomization.yaml Add namespace: kubeflow
manifests/kustomize/check-namespace.sh New — namespace regression check script

No change needed for options/ui/overlays/istio/kustomization.yaml — it already has namespace: kubeflow.

Design Decisions

Dependencies and Related Issues

Contributor Checklist

  • DCO sign-off included
  • AI assistance disclosed (Assisted-by: Antigravity AI in commit message)
  • Matches existing codebase pattern (options/ui/overlays/istio/kustomization.yaml)
  • Minimal scope — only the 3 files that need the fix plus 1 regression check
  • Root cause identified and stated in one sentence
  • Namespace regression check added per maintainer request

cc @tarilabs @pboyd

Three of the four kustomization overlays referenced by kubeflow/manifests
example/kustomization.yaml were missing namespace: kubeflow. The CI install
script masked this by using kubectl apply -n kubeflow, but kustomize build
has no such override — resources land in the default namespace.

Add namespace: kubeflow to:
- overlays/postgres/kustomization.yaml
- options/istio/kustomization.yaml
- options/catalog/overlays/demo/kustomization.yaml

The fourth overlay (options/ui/overlays/istio) already had it.

Also add check-namespace.sh to guard against namespace regression.

Relates-to: kubeflow/manifests#3457
Relates-to: kubeflow/manifests#3458

Assisted-by: Antigravity AI
Signed-off-by: TruthMesh AI Assistant <[email protected]>
@google-oss-prow google-oss-prow Bot requested a review from rareddy May 14, 2026 21:33
@google-oss-prow
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign rareddy for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pboyd
Copy link
Copy Markdown
Member

pboyd commented May 15, 2026

Hi @Raakshass, thanks for looking at this. Unfortunately, I don't think we can simply add the namespace because model-registry needs to actually be in the user namespace for the dashboard to work (kubeflow-user-example-com by default in the manifests).

The manifests actually did deploy to kubeflow at one time, but we removed it to address #1045. The installation docs provide instructions to set the namespace before applying them, which is how we've been able to get away without setting the manifest.

I'm not sure a good way to solve this, but I'll spend some time today and try to find something. Let me know if you have any ideas.

@Raakshass
Copy link
Copy Markdown
Author

youre right, i missed the user-namespace dependency and the #1045 history. hub should stay namespace-agnostic since model-registry needs to live in the profile namespace for dashboard integration
the overlay approach you proposed on the issue is the right fix set the namespace at the manifests layer instead of here. i can pick that up and raise the manifests-side overlay PR based on your diff. would be great to have your input on it given you know the deployment topology best
should i close this PR?

@Raakshass
Copy link
Copy Markdown
Author

closing this per pboyd's feedback. the hub repo should stay namespace-agnostic since model-registry needs to live in the profile namespace for dashboard integration. the fix belongs at the manifests layer via an overlay as discussed in kubeflow/manifests#3457.

@pboyd
Copy link
Copy Markdown
Member

pboyd commented May 17, 2026

Sorry for the delay, @Raakshass. I know we need to close this, but thank you for tackling this problem. It's been an issue for a while now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hub / Model Registry is being deployed to the namespace default

2 participants