Skip to content

Fix helm chart 404 and applicationset delete permission for caas operator in production#11333

Open
amisstea wants to merge 1 commit intoredhat-appstudio:mainfrom
amisstea:KONFLUX-12098
Open

Fix helm chart 404 and applicationset delete permission for caas operator in production#11333
amisstea wants to merge 1 commit intoredhat-appstudio:mainfrom
amisstea:KONFLUX-12098

Conversation

@amisstea
Copy link
Copy Markdown
Contributor

@amisstea amisstea commented Apr 16, 2026

What/Why

The cluster-aas-operator-controller-manager in production is experiencing crash loops partly due to two recurring errors that contribute to tight reconcile loops:

  1. A trailing slash in the repoURL for the hypershift-aws-template helm chart causes a double-slash in the index.yaml request path, resulting in 404 errors.
  2. The controller's service account lacks permission to delete applicationsets, causing repeated reconciler errors.

This PR fixes both issues in the base configuration and removes the now-redundant duplicates from the dev and staging overlays.

Staging PR

#11293

Verification in staging (kflux-stg-es01)

  • Confirmed the ApplicationSet has the correct repoURL (no trailing slash): 
    $ oc get applicationset hypershift-aws-cluster -n openshift-gitops -o jsonpath='{.spec.template.spec.source.repoURL}'
https://konflux-ci.dev/cluster-template-charts
  • Confirmed the RBAC Role and RoleBinding exist and the service account can delete applicationsets: 
    $ oc auth can-i delete applicationsets.argoproj.io -n cluster-aas-operator \
    --as=system:serviceaccount:openshift-operators:cluster-aas-operator-controller-manager
yes
  • Confirmed zero ERROR or forbidden messages in the controller logs since the staging fix was deployed.

Assisted-by: Claude claude-opus-4-6

@amisstea
Fix helm chart 404 and applicationset delete permission for caas oper…
8895b04 
…ator in production

Move the repoURL trailing slash fix and RBAC resources from
dev/staging overlays into the base so they apply to all environments
including production.

Assisted-by: Claude claude-opus-4-6
Signed-off-by: amisstea <[email protected]>
@openshift-ci openshift-ci bot requested review from avi-biton and ifireball April 16, 2026 19:25
@qodo-code-review
Copy link
Copy Markdown

qodo-code-review bot commented Apr 16, 2026

Review Summary by Qodo

Move cluster-aas-operator fixes to base for production deployment

🐞 Bug fix

Grey Divider

Walkthroughs

Description 
• Remove trailing slash from helm chart repoURL to fix 404 errors
• Add RBAC resources to base for applicationset delete permissions
• Consolidate duplicated fixes from dev/staging overlays into base
• Ensure consistent configuration across all environments
Diagram 
flowchart LR
  base["Base Configuration"]
  dev["Development Overlay"]
  staging["Staging Overlay"]
  prod["Production Environment"]
  
  base -- "repoURL fix + RBAC" --> prod
  dev -- "removes duplicates" --> base
  staging -- "removes duplicates" --> base
Loading

Grey Divider

File Changes

1. components/cluster-as-a-service/base/clustertemplates.yaml 🐞 Bug fix +1/-1

Remove trailing slash from helm chart repoURL

• Remove trailing slash from repoURL in hypershift-aws-template helm chart
• Change from https://konflux-ci.dev/cluster-template-charts/ to
 https://konflux-ci.dev/cluster-template-charts
• Fixes 404 errors caused by double-slash in index.yaml request path

components/cluster-as-a-service/base/clustertemplates.yaml

2. components/cluster-as-a-service/base/kustomization.yaml ⚙️ Configuration changes +1/-0

Include RBAC resources in base kustomization

• Add cluster-aas-operator-rbac.yaml to resources list
• Enables RBAC configuration for cluster-aas-operator in base

components/cluster-as-a-service/base/kustomization.yaml

3. components/cluster-as-a-service/base/cluster-aas-operator-rbac.yaml 🐞 Bug fix +0/-0

Add RBAC resources for applicationset deletion

• New file containing Role and RoleBinding for applicationset deletion
• Grants cluster-aas-operator-controller-manager service account permission to delete
 applicationsets
• Prevents repeated reconciler errors due to missing permissions

components/cluster-as-a-service/base/cluster-aas-operator-rbac.yaml

View more (5)
4. components/cluster-as-a-service/development/add-hypershift-params.yaml Miscellaneous +0/-4

Remove redundant repoURL patch from development

• Remove repoURL replacement patch (now in base)
• Keep hypershift-specific parameter patches for development environment

components/cluster-as-a-service/development/add-hypershift-params.yaml

5. components/cluster-as-a-service/development/kustomization.yaml Miscellaneous +0/-1

Remove RBAC resource from development overlay

• Remove cluster-aas-operator-rbac.yaml from resources (moved to base)
• Simplify development overlay by removing duplicated RBAC configuration

components/cluster-as-a-service/development/kustomization.yaml

6. components/cluster-as-a-service/staging/add-hypershift-params.yaml Miscellaneous +0/-4

Remove redundant repoURL patch from staging

• Remove repoURL replacement patch (now in base)
• Keep staging-specific hypershift parameters and configuration

components/cluster-as-a-service/staging/add-hypershift-params.yaml

7. components/cluster-as-a-service/staging/cluster-aas-operator-rbac.yaml Miscellaneous +0/-28

Remove RBAC file from staging overlay

• Delete file containing Role and RoleBinding definitions
• RBAC resources now defined in base configuration

components/cluster-as-a-service/staging/cluster-aas-operator-rbac.yaml

8. components/cluster-as-a-service/staging/kustomization.yaml Miscellaneous +0/-1

Remove RBAC resource from staging overlay

• Remove cluster-aas-operator-rbac.yaml from resources (moved to base)
• Consolidate RBAC configuration into base for all environments

components/cluster-as-a-service/staging/kustomization.yaml

Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown

qodo-code-review bot commented Apr 16, 2026 •
edited
Loading

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

Kustomize Render Diff

Comparing ccba1b54e → 2e728ce6c

Component Environment Changes
components/cluster-as-a-service/production production +28 -1

Total: 1 components, +28 -1 lines

📋 Full diff available in the workflow summary and as a downloadable artifact.

@amisstea
Copy link
Copy Markdown
Contributor Author

amisstea commented Apr 16, 2026

/assign @kelchen123 @hmariset

@openshift-ci openshift-ci bot added the lgtm label Apr 16, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 16, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amisstea, kelchen123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 16, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.62%. Comparing base (457e74e) to head (8895b04).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main   #11333   +/-   ##
=======================================
  Coverage   51.62%   51.62%           
=======================================
  Files          18       18           
  Lines        1263     1263           
=======================================
  Hits          652      652           
  Misses        539      539           
  Partials       72       72
Flag Coverage Δ
go 51.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@avi-biton avi-biton Awaiting requested review from avi-biton

@ifireball ifireball Awaiting requested review from ifireball

1 more reviewer

@kelchen123 kelchen123 kelchen123 approved these changes

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

@hmariset hmariset

@kelchen123 kelchen123

Labels

approved environment/development environment/production environment/staging infra/hold-production lgtm prod/needs-approval

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@amisstea @kelchen123 @hmariset