Skip to content

feat: implemented status resource routing for remote cluster mode #4579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JaydipGabani merged 7 commits into from
Jun 17, 2026
Merged

feat: implemented status resource routing for remote cluster mode #4579

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
403f7e5
implement routing for status resources in remote cluster mode
abhisheksheth28 May 19, 2026
4ae5990
address review feedback
abhisheksheth28 Jun 6, 2026
7ecef7b
lint fix
abhisheksheth28 Jun 6, 2026
0e847d4
address review feedback, remote refactor
abhisheksheth28 Jun 16, 2026
7908fa4
e2e workflow race fix
abhisheksheth28 Jun 17, 2026
91d558e
update docs
abhisheksheth28 Jun 17, 2026
98439ec
Merge branch 'master' into status-resource-routing
JaydipGabani Jun 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
457 changes: 297 additions & 160 deletions .github/workflows/remote-cluster-e2e.yaml
View file
Open in desktop

@JaydipGabani JaydipGabani Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test template is Rego-only, so VAP generation is not really asserted. External data is not meaningfully exercised either. Can you increase the coverage to test these as well, we want to make sure we are not releasing something that break other features to make adoption or trying out for users as easy as possible to get feedback easily as well.

@JaydipGabani JaydipGabani Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • remote-cluster-e2e.yaml does not really prove “no GC churn.” It records a ConstraintTemplatePodStatus UID, sleeps 2 seconds, and checks the UID again. A broken owner-ref cleanup path on pod restart, or delayed delete/recreate churn, could still pass. If this is meant to cover the bug, it should delete/restart a Gatekeeper pod and assert the old pod-owned status is cleaned up and the new one appears only on the management cluster.

  • remote-cluster-e2e.yaml calls this “external data Provider routing,” but it only creates a Provider and checks ProviderPodStatus. It does not deploy the dummy provider service or exercise an external-data policy/admission request. So it proves status routing for ProviderPodStatus, not that external data works cross-cluster. The existing BATS external-data test around test.bats is a better model because it actually deploys the provider and validates admission behavior.

  • The workflow embeds almost the entire test suite as inline shell from remote-cluster-e2e.yaml. That makes it hard to run locally, hard to reuse the repo’s BATS helpers, and easy for CI-only logic to drift from make test-e2e in Makefile.

I’d move most of the assertions into BATS. I’d keep the GitHub workflow responsible for CI-only orchestration: matrix, checkout, tool install, two Kind clusters, image build/load, deployment, artifact upload. Then call something like:

make test-e2e BATS_TESTS_FILE=test/bats/remote-cluster.bats

or add a small make test-e2e-remote-cluster target.

Cluster creation, image build/load, and log artifact upload can stay in the workflow or Make targets. But the behavioral checks, waits, owner-ref assertions, VAP/webhook enforcement checks, and Provider routing checks belong in BATS so reviewers and contributors can run the same test locally.

Not a blocking issue for this pr but we should definetly follow up with this.

Large diffs are not rendered by default.

7 changes: 2 additions & 5 deletions apis/status/v1alpha1/connectionpodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,8 @@ func NewConnectionStatusForPod(pod *corev1.Pod, connectionNamespace, connectionN
v1beta1.PodLabel: pod.Name,
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
52 changes: 0 additions & 52 deletions apis/status/v1alpha1/connectionpodstatus_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import (
"github.com/open-policy-agent/gatekeeper/v3/apis/status/v1beta1"
"github.com/open-policy-agent/gatekeeper/v3/pkg/fakes"
"github.com/open-policy-agent/gatekeeper/v3/pkg/operations"
"github.com/open-policy-agent/gatekeeper/v3/pkg/util"
"github.com/open-policy-agent/gatekeeper/v3/test/testutils"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
Expand Down Expand Up @@ -69,54 +68,3 @@ func TestNewConnectionStatusForPod(t *testing.T) {
t.Fatal("got status.Name != n, want equal")
}
}

func TestNewConnectionStatusForPod_SkipsOwnerRefInRemoteClusterMode(t *testing.T) {
const podName = "some-gk-pod"
const podNS = "a-gk-namespace"
const connectionName = "audit"
const connectionNamespace = "a-gk-ns"

testutils.Setenv(t, "POD_NAMESPACE", podNS)

// Enable skip OwnerRef mode (remote cluster mode)
util.SetSkipPodOwnerRef(true)
t.Cleanup(func() {
util.SetSkipPodOwnerRef(false)
})

scheme := runtime.NewScheme()
if err := v1beta1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}
if err := corev1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}

pod := fakes.Pod(
fakes.WithNamespace(podNS),
fakes.WithName(podName),
)

status, err := v1alpha1.NewConnectionStatusForPod(pod, connectionNamespace, connectionName, scheme)
if err != nil {
t.Fatal(err)
}

// Verify OwnerReference is NOT set
if len(status.GetOwnerReferences()) != 0 {
t.Errorf("Expected no OwnerReferences in remote cluster mode, got %d", len(status.GetOwnerReferences()))
}

// Verify all other fields are still populated correctly
if status.Status.ID != podName {
t.Errorf("Expected Status.ID = %q, got %q", podName, status.Status.ID)
}

labels := status.GetLabels()
if labels[v1beta1.PodLabel] != podName {
t.Errorf("Expected PodLabel = %q, got %q", podName, labels[v1beta1.PodLabel])
}
if labels[v1beta1.ConnectionNameLabel] != connectionName {
t.Errorf("Expected ConnectionNameLabel = %q, got %q", connectionName, labels[v1beta1.ConnectionNameLabel])
}
}
7 changes: 2 additions & 5 deletions apis/status/v1beta1/configpodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,8 @@ func NewConfigStatusForPod(pod *corev1.Pod, configNamespace string, configName s
PodLabel: pod.Name,
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
52 changes: 0 additions & 52 deletions apis/status/v1beta1/configpodstatus_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"github.com/open-policy-agent/gatekeeper/v3/apis/status/v1beta1"
"github.com/open-policy-agent/gatekeeper/v3/pkg/fakes"
"github.com/open-policy-agent/gatekeeper/v3/pkg/operations"
"github.com/open-policy-agent/gatekeeper/v3/pkg/util"
"github.com/open-policy-agent/gatekeeper/v3/test/testutils"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
Expand Down Expand Up @@ -68,54 +67,3 @@ func TestNewConfigStatusForPod(t *testing.T) {
t.Fatal("got status.Name != n, want equal")
}
}

func TestNewConfigStatusForPod_SkipsOwnerRefInRemoteClusterMode(t *testing.T) {
const podName = "some-gk-pod"
const podNS = "a-gk-namespace"
const configName = "a-config"
const configNameSpace = "a-gk-ns"

testutils.Setenv(t, "POD_NAMESPACE", podNS)

// Enable skip OwnerRef mode (remote cluster mode)
util.SetSkipPodOwnerRef(true)
t.Cleanup(func() {
util.SetSkipPodOwnerRef(false)
})

scheme := runtime.NewScheme()
if err := v1beta1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}
if err := corev1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}

pod := fakes.Pod(
fakes.WithNamespace(podNS),
fakes.WithName(podName),
)

status, err := v1beta1.NewConfigStatusForPod(pod, configNameSpace, configName, scheme)
if err != nil {
t.Fatal(err)
}

// Verify OwnerReference is NOT set
if len(status.GetOwnerReferences()) != 0 {
t.Errorf("Expected no OwnerReferences in remote cluster mode, got %d", len(status.GetOwnerReferences()))
}

// Verify all other fields are still populated correctly
if status.Status.ID != podName {
t.Errorf("Expected Status.ID = %q, got %q", podName, status.Status.ID)
}

labels := status.GetLabels()
if labels[v1beta1.PodLabel] != podName {
t.Errorf("Expected PodLabel = %q, got %q", podName, labels[v1beta1.PodLabel])
}
if labels[v1beta1.ConfigNameLabel] != configName {
t.Errorf("Expected ConfigNameLabel = %q, got %q", configName, labels[v1beta1.ConfigNameLabel])
}
}
7 changes: 2 additions & 5 deletions apis/status/v1beta1/constraintpodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,8 @@ func NewConstraintStatusForPod(pod *corev1.Pod, constraint *unstructured.Unstruc
ConstraintTemplateNameLabel: strings.ToLower(constraint.GetKind()),
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err = controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err = controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
51 changes: 0 additions & 51 deletions apis/status/v1beta1/constraintpodstatus_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"github.com/open-policy-agent/gatekeeper/v3/apis/status/v1beta1"
"github.com/open-policy-agent/gatekeeper/v3/pkg/fakes"
"github.com/open-policy-agent/gatekeeper/v3/pkg/operations"
"github.com/open-policy-agent/gatekeeper/v3/pkg/util"
"github.com/open-policy-agent/gatekeeper/v3/test/testutils"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
Expand Down Expand Up @@ -82,53 +81,3 @@ func TestNewConstraintStatusForPod(t *testing.T) {
t.Errorf("got Constraint key %q, want %q", cmVal, gotStatus.Name)
}
}

func TestNewConstraintStatusForPod_SkipsOwnerRefInRemoteClusterMode(t *testing.T) {
testutils.Setenv(t, "POD_NAMESPACE", podNS)

// Enable skip OwnerRef mode (remote cluster mode)
util.SetSkipPodOwnerRef(true)
t.Cleanup(func() {
util.SetSkipPodOwnerRef(false)
})

scheme := runtime.NewScheme()
if err := v1beta1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}
if err := corev1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}

pod := fakes.Pod(
fakes.WithNamespace(podNS),
fakes.WithName(podName),
)

cstr := &unstructured.Unstructured{}
cstr.SetGroupVersionKind(schema.GroupVersionKind{Group: v1beta1.ConstraintsGroup, Version: "v1beta1", Kind: cstrKind})
cstr.SetName(cstrName)

status, err := v1beta1.NewConstraintStatusForPod(pod, cstr, scheme)
if err != nil {
t.Fatal(err)
}

// Verify OwnerReference is NOT set
if len(status.GetOwnerReferences()) != 0 {
t.Errorf("Expected no OwnerReferences in remote cluster mode, got %d", len(status.GetOwnerReferences()))
}

// Verify all other fields are still populated correctly
if status.Status.ID != podName {
t.Errorf("Expected Status.ID = %q, got %q", podName, status.Status.ID)
}

labels := status.GetLabels()
if labels[v1beta1.PodLabel] != podName {
t.Errorf("Expected PodLabel = %q, got %q", podName, labels[v1beta1.PodLabel])
}
if labels[v1beta1.ConstraintNameLabel] != cstrName {
t.Errorf("Expected ConstraintNameLabel = %q, got %q", cstrName, labels[v1beta1.ConstraintNameLabel])
}
}
7 changes: 2 additions & 5 deletions apis/status/v1beta1/constrainttemplatepodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,8 @@ func NewConstraintTemplateStatusForPod(pod *corev1.Pod, templateName string, sch
PodLabel: pod.Name,
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
59 changes: 0 additions & 59 deletions apis/status/v1beta1/constrainttemplatepodstatus_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"github.com/open-policy-agent/gatekeeper/v3/apis/status/v1beta1"
"github.com/open-policy-agent/gatekeeper/v3/pkg/fakes"
"github.com/open-policy-agent/gatekeeper/v3/pkg/operations"
"github.com/open-policy-agent/gatekeeper/v3/pkg/util"
"github.com/open-policy-agent/gatekeeper/v3/test/testutils"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
Expand Down Expand Up @@ -67,61 +66,3 @@ func TestNewConstraintTemplateStatusForPod(t *testing.T) {
t.Fatal("got status.Name != n, want equal")
}
}

func TestNewConstraintTemplateStatusForPod_SkipsOwnerRefInRemoteClusterMode(t *testing.T) {
podName := "some-gk-pod"
podNS := "a-gk-namespace"
templateName := "a-template"

testutils.Setenv(t, "POD_NAMESPACE", podNS)

// Enable skip OwnerRef mode (remote cluster mode)
util.SetSkipPodOwnerRef(true)
t.Cleanup(func() {
util.SetSkipPodOwnerRef(false)
})

scheme := runtime.NewScheme()
if err := v1beta1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}
if err := corev1.AddToScheme(scheme); err != nil {
t.Fatal(err)
}

pod := fakes.Pod(
fakes.WithNamespace(podNS),
fakes.WithName(podName),
)

status, err := v1beta1.NewConstraintTemplateStatusForPod(pod, templateName, scheme)
if err != nil {
t.Fatal(err)
}

// Verify OwnerReference is NOT set
if len(status.GetOwnerReferences()) != 0 {
t.Errorf("Expected no OwnerReferences in remote cluster mode, got %d", len(status.GetOwnerReferences()))
}

// Verify all other fields are still populated correctly
if status.Status.ID != podName {
t.Errorf("Expected Status.ID = %q, got %q", podName, status.Status.ID)
}

labels := status.GetLabels()
if labels[v1beta1.PodLabel] != podName {
t.Errorf("Expected PodLabel = %q, got %q", podName, labels[v1beta1.PodLabel])
}
if labels[v1beta1.ConstraintTemplateNameLabel] != templateName {
t.Errorf("Expected ConstraintTemplateNameLabel = %q, got %q", templateName, labels[v1beta1.ConstraintTemplateNameLabel])
}

expectedName, err := v1beta1.KeyForConstraintTemplate(podName, templateName)
if err != nil {
t.Fatal(err)
}
if status.Name != expectedName {
t.Errorf("Expected Name = %q, got %q", expectedName, status.Name)
}
}
7 changes: 2 additions & 5 deletions apis/status/v1beta1/expansiontemplatepodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,8 @@ func NewExpansionTemplateStatusForPod(pod *corev1.Pod, templateName string, sche
PodLabel: pod.Name,
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
7 changes: 2 additions & 5 deletions apis/status/v1beta1/mutatorpodstatus_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,8 @@ func NewMutatorStatusForPod(pod *corev1.Pod, mutatorID mtypes.ID, scheme *runtim
PodLabel: pod.Name,
})

// Skip OwnerReference in remote cluster mode
if !util.ShouldSkipPodOwnerRef() {
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}
if err := controllerutil.SetOwnerReference(pod, obj, scheme); err != nil {
return nil, err
}

return obj, nil
Expand Down
Loading
Loading