Skip to content

pause localqueues with namespace annotations#11325

Open
filariow wants to merge 2 commits intoredhat-appstudio:mainfrom
filariow:queue-config-exclusion-label
Open

pause localqueues with namespace annotations#11325
filariow wants to merge 2 commits intoredhat-appstudio:mainfrom
filariow:queue-config-exclusion-label

Conversation

@filariow
Copy link
Copy Markdown
Member

This PR proposes to add support for a Namespace annotation to use to drive the value of the LocalQueue's spec.stopPolicy field.
With this we can allow/disallow admission of Workloads at tenant namespace level.

The release of this change needs to go through the 3-phases rollout strategy described at https://github.com/redhat-appstudio/infra-deployments/tree/main/components/policies\#deletion-of-generated-resources-not-acceptable

Signed-off-by: Francesco Ilario [email protected]

@qodo-code-review
Copy link
Copy Markdown

Review Summary by Qodo

Add namespace annotation support for LocalQueue pause control

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• Add namespace annotation support to control LocalQueue pause state
  - New kueue.konflux-ci.dev/stop-policy annotation drives stopPolicy field
  - Annotation value hold sets stopPolicy: Hold, other values set stopPolicy: None
• Expand test coverage with four new test scenarios
  - Tests for paused, unpaused, and dynamic state transitions
  - Tests for annotation addition and removal workflows
• Update expected LocalQueue fixtures to include stopPolicy field
Diagram
flowchart LR
  NS["Namespace with<br/>stop-policy annotation"]
  POLICY["Kyverno ClusterPolicy<br/>with JMESPath context"]
  LQ["Generated LocalQueue<br/>with stopPolicy field"]
  NS -- "annotation value<br/>extracted" --> POLICY
  POLICY -- "evaluates annotation<br/>to Hold or None" --> LQ
Loading

Grey Divider

File Changes

1. components/policies/development/kueue/queue-config/cluster-policy.yaml ✨ Enhancement +6/-0

Add annotation-driven stopPolicy context variable

• Added JMESPath context variable stopPolicy to evaluate namespace annotation
• Context extracts kueue.konflux-ci.dev/stop-policy annotation value
• Maps hold annotation to Hold stopPolicy, all others to None
• Updated LocalQueue generation to include stopPolicy field from context variable

components/policies/development/kueue/queue-config/cluster-policy.yaml


2. components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml 🧪 Tests +237/-1

Add comprehensive pause state transition tests

• Updated first test description to clarify no annotation requirement
• Added three new comprehensive test scenarios for pause/unpause workflows
• Test for unpaused namespace with explicit none annotation
• Test for paused-to-unpaused transition by removing annotation
• Test for unpaused-to-paused transition by adding annotation

components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml


3. components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue.yaml 🧪 Tests +1/-0

Add stopPolicy field to default fixture

• Added stopPolicy: None field to expected LocalQueue fixture

components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue.yaml


View more (5)
4. components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-paused.yaml 🧪 Tests +8/-0

Create paused LocalQueue fixture

• New fixture file for paused LocalQueue state
• Defines expected LocalQueue with stopPolicy: Hold

components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-paused.yaml


5. components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-kanary.yaml 🧪 Tests +1/-0

Add stopPolicy field to kanary fixture

• Added stopPolicy: None field to kanary namespace fixture

components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-kanary.yaml


6. components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-mintmaker.yaml 🧪 Tests +1/-0

Add stopPolicy field to mintmaker fixture

• Added stopPolicy: None field to mintmaker namespace fixture

components/policies/development/kueue/queue-config/.chainsaw-test/resources/expected-localqueue-mintmaker.yaml


7. components/policies/development/kueue/queue-config/.chainsaw-test/resources/namespace-tenant-paused.yaml 🧪 Tests +8/-0

Create paused tenant namespace fixture

• New namespace fixture with pause annotation
• Includes kueue.konflux-ci.dev/stop-policy: hold annotation
• Labeled with konflux-ci.dev/type: tenant

components/policies/development/kueue/queue-config/.chainsaw-test/resources/namespace-tenant-paused.yaml


8. components/policies/development/kueue/queue-config/.chainsaw-test/resources/namespace-tenant-unpaused.yaml 🧪 Tests +8/-0

Create unpaused tenant namespace fixture

• New namespace fixture with explicit unpause annotation
• Includes kueue.konflux-ci.dev/stop-policy: none annotation
• Labeled with konflux-ci.dev/type: tenant

components/policies/development/kueue/queue-config/.chainsaw-test/resources/namespace-tenant-unpaused.yaml


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown

qodo-code-review bot commented Apr 16, 2026

Code Review by Qodo

🐞 Bugs (1)   📘 Rule violations (0)   📎 Requirement gaps (0)
🐞\ ☼ Reliability (1)

Grey Divider


Action required

1. Flaky shared test namespace 🐞
Description
Multiple Chainsaw Test documents reuse the same (spec.namespace, suffix) values, so they all
create/update the same tenant Namespace and generated LocalQueue names. This makes the suite
order-dependent and can lead to flaky CI (or false passes) if resources persist between tests or
tests run in parallel at suite level.
Code

components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[R62-67]

+  concurrent: false
+  namespace: kueue-queue-new
+  bindings:
+  - name: suffix
+    value: labeled
+  steps:
Relevance

⭐⭐⭐ High

All new Chainsaw Tests reuse same namespace+suffx, targeting one Namespace/LocalQueue; risks
order-dependent/flaky behavior.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
In the dev kueue queue-config Chainsaw suite, the newly added tests repeatedly set `namespace:
kueue-queue-new and suffix: labeled`, which (via the templated Namespace name in the fixtures)
targets the same Namespace/LocalQueue across multiple Test documents. Other Chainsaw suites in this
repo vary suffix between tests to avoid collisions, indicating the intended isolation pattern.

components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[11-16]
components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[62-67]
components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[115-120]
components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[181-186]
components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[246-251]
components/policies/development/kueue/queue-config/.chainsaw-test/resources/namespace-tenant-paused.yaml[4-8]
components/policies/development/integration/bootstrap-namespace/.chainsaw-test/chainsaw-test.yaml[11-15]
components/policies/development/integration/bootstrap-namespace/.chainsaw-test/chainsaw-test.yaml[55-59]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Multiple `Test` documents in the same Chainsaw suite reuse the same `spec.namespace` and `bindings.suffix` values, causing them to act on the same generated Namespace/LocalQueue names. This can make the suite order-dependent and flaky.

### Issue Context
The namespace fixtures derive the created Namespace name from `(join('-', [$namespace, $suffix]))`. With the same `$namespace` (`kueue-queue-new`) and `$suffix` (`labeled`) across several tests, they collide.

### Fix Focus Areas
- components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[11-120]
- components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[115-251]

### Suggested fix
Assign a unique `suffix` value per `Test` document (e.g., `labeled-no-annotation`, `labeled-unpaused`, `labeled-paused-resumed`, `labeled-update-paused`, `labeled-paused`) so each test targets a distinct Namespace/LocalQueue. Keep the suffix consistent within a single test where you intentionally update the same Namespace across steps.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2026

Kustomize Render Diff

Comparing bd9a404d650e41996f

Component Environment Changes
components/policies/development development +6 -1
components/policies/staging/stone-stage-p01 staging +6 -1
components/policies/staging/stone-stg-rh01 staging +6 -1

Total: 3 components, +18 -3 lines

📋 Full diff available in the workflow summary and as a downloadable artifact.

@filariow
Copy link
Copy Markdown
Member Author

/hold

The release of this change needs to go through the 3-phases rollout strategy described at https://github.com/redhat-appstudio/infra-deployments/tree/main/components/policies#deletion-of-generated-resources-not-acceptable.

@filariow filariow requested a review from gbenhaim April 16, 2026 11:50
@filariow filariow force-pushed the queue-config-exclusion-label branch from 0cce31c to 209cdf0 Compare April 16, 2026 11:52
this change proposes to add support for a Namespace annotation to use
to drive the value of the LocalQueue's `spec.stopPolicy` field.
With this we can allow/disallow admission of Workloads at tenant
namespace level.

The release of this change needs to go through the 3-phases
rollout strategy described at
https://github.com/redhat-appstudio/infra-deployments/tree/main/components/policies\#deletion-of-generated-resources-not-acceptable

Signed-off-by: Francesco Ilario <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@filariow filariow force-pushed the queue-config-exclusion-label branch from 209cdf0 to 6baa288 Compare April 16, 2026 11:53
Comment on lines +62 to +67
concurrent: false
namespace: kueue-queue-new
bindings:
- name: suffix
value: labeled
steps:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Flaky shared test namespace 🐞 Bug ☼ Reliability

Multiple Chainsaw Test documents reuse the same (spec.namespace, suffix) values, so they all
create/update the same tenant Namespace and generated LocalQueue names. This makes the suite
order-dependent and can lead to flaky CI (or false passes) if resources persist between tests or
tests run in parallel at suite level.
Agent Prompt
### Issue description
Multiple `Test` documents in the same Chainsaw suite reuse the same `spec.namespace` and `bindings.suffix` values, causing them to act on the same generated Namespace/LocalQueue names. This can make the suite order-dependent and flaky.

### Issue Context
The namespace fixtures derive the created Namespace name from `(join('-', [$namespace, $suffix]))`. With the same `$namespace` (`kueue-queue-new`) and `$suffix` (`labeled`) across several tests, they collide.

### Fix Focus Areas
- components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[11-120]
- components/policies/development/kueue/queue-config/.chainsaw-test/chainsaw-test.yaml[115-251]

### Suggested fix
Assign a unique `suffix` value per `Test` document (e.g., `labeled-no-annotation`, `labeled-unpaused`, `labeled-paused-resumed`, `labeled-update-paused`, `labeled-paused`) so each test targets a distinct Namespace/LocalQueue. Keep the suffix consistent within a single test where you intentionally update the same Namespace across steps.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.62%. Comparing base (bd9a404) to head (73d7a8f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #11325   +/-   ##
=======================================
  Coverage   51.62%   51.62%           
=======================================
  Files          18       18           
  Lines        1263     1263           
=======================================
  Hits          652      652           
  Misses        539      539           
  Partials       72       72           
Flag Coverage Δ
go 51.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gbenhaim
Copy link
Copy Markdown
Member

worth mentioning this feature in the Kueue SOP as well.

@openshift-ci openshift-ci bot removed the lgtm label Apr 16, 2026
apply uses SSA, so it's not removing the annotation

Signed-off-by: Francesco Ilario <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@filariow filariow force-pushed the queue-config-exclusion-label branch from a653269 to 73d7a8f Compare April 16, 2026 12:56
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 16, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: filariow, gbenhaim, sadlerap

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants