calicoctl: add optimize sub-command that can optimize GNPs for better efficiency

[fasaxc]

Description

Add a new calicoctl optimize sub-command that takes in one or more Calico resources and emits (possibly) optimized versions to stdout.

In this initial version, only global network policies are optimized, other resources are passed through unaltered. The tool makes several optimizations to global network policies:

• Hoist selectors up from rules to the top-level subject selector.

  • When an ingress rule uses a destination selector, that results in rendering an expensive IP set to the dataplane (including every IP that the selector matches) even though Felix already knows which workload it is applying the policy to.
  • Similarly, source selectors in egress are inefficient.
  • When the tool spots such a selector, it splits the policy into smaller policies, each with a different selector. Then it hoists the selector up to the top-level of the policy so that the policy only applies to that subset of workloads. This removes the IP set, and, if the top-level selector was previously broad, it reduces the number of workloads that the sub policy even applies to. Policies that don't apply to any endpoint on a given node are not even calculated and do not get rendered to iptables.

• Remove duplicate rules within a policy.

• Remove unreachable rules (rules after an unconditional Allow/Deny/Pass).

• Normalize policy selectors.

Related issues/PRs

CORE-11913

Todos

• Tests
• Documentation
• Release note

Release Note

calicoctl: add optimize sub-command that can optimize GNPs for better efficiency including splitting large policies that misuse source selectors on egress rules/dest selectors on ingress rules.

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

• docs-pr-required: This change requires a change to the documentation that has not been completed yet.
• docs-completed: This change has all necessary documentation completed.
• docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

• release-note-required: This PR has user-facing changes. Most PRs should have this label.
• release-note-not-required: This PR has no user-facing changes.

Other optional labels:

• cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
• needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

[Copilot]

Pull Request Overview

This PR introduces a new calicoctl optimize sub-command that optimizes Calico Global Network Policies (GNPs) for better performance by canonicalizing selectors, hoisting rule-level selectors to the top-level subject selector when safe, and splitting policies to reduce dataplane overhead.

Key Changes:

• New optimization library in libcalico-go/lib/optimize with functions to optimize GNPs by hoisting inefficient selectors and removing duplicate/unreachable rules
• New calicoctl optimize command that validates resources offline and outputs optimized YAML
• Comprehensive unit and system tests for the optimization logic

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
libcalico-go/lib/optimize/optimize_gnp.go	Core GNP optimization logic including selector canonicalization, rule splitting, and duplicate removal
libcalico-go/lib/optimize/optimize_objects.go	High-level API for optimizing runtime.Object slices with support for lists
libcalico-go/lib/optimize/optimize_objects_test.go	Unit tests for object optimization pass-through behavior and list handling
libcalico-go/lib/optimize/gnp_split_test.go	Comprehensive unit tests for GNP splitting, selector hoisting, and edge cases
calicoctl/calicoctl/commands/optimize.go	CLI command implementation that validates and optimizes resources from file/stdin
calicoctl/calicoctl/calicoctl.go	Registers the new optimize sub-command in calicoctl
calicoctl/tests/st/calicoctl/test_optimize.py	System tests verifying optimize command behavior with various input formats
calicoctl/tests/st/manifests/optimize-multidoc.yaml	Test fixture with multi-document YAML for pass-through validation
calicoctl/tests/st/manifests/optimize-gnp-split.yaml	Test fixture demonstrating GNP splitting optimization scenarios
.semaphore/semaphore.yml	Updates CI trigger paths to include new optimize package
.github/copilot-instructions.md	Documentation clarification about test locations and languages

[github-actions]

This PR is stale because it has been open for 60 days with no activity.