nfpm: support chaining multiple InjectNfpmPackageFieldsRequests (sorted w/ simple priority integer)#22864
Merged
cognifloyd merged 6 commits intomainfrom Nov 8, 2025
Merged
Conversation
This is a convenience method that handles retrieving a field that was injected by an earlier rule in the chaing, or falling back to getting the field from the target.
3 tasks
Member
Author
|
I extracted this PR from #22861 |
benjyw
reviewed
Nov 6, 2025
benjyw
approved these changes
Nov 7, 2025
cognifloyd
added a commit
that referenced
this pull request
Nov 20, 2025
…22899) ## PR Series Overview This is the second in a series of PRs that introduces a new backend: `pants.backend.npm.native_libs` Initially, the backend will be available as: `pants.backend.experimental.nfpm.native_libs` I proposed this new backend (originally named `bindeps`) in discussion #22396. This backend will inspect ELF bin/lib files (like `lib*.so`) in packaged contents (for this PR series, only in `pex_binary` targets) to identify package dependency metadata and inject that metadata on the relevant `nfpm_deb_package` or `nfpm_rpm_package` targets. Effectively, it will provide an approximation of these native packager features: - `rpm`: `rpmdeps` + `elfdeps` - `deb`: `dh_shlibdeps` + `dpkg-shlibdeps` (These substitute `${shlibs:Depends}` in debian control files have) ### Goal: Host-agnostic package builds This pants backend is designed to be host-agnostic, like [nFPM](https://nfpm.goreleaser.com/). Native packaging tools are often restricted to a single release of a single distro. Unlike native package builders, this new pants backend does not use any of those distro-specific or distro-release-specific utilities or local package databases. This new backend should be able to run (help with building deb and rpm packages) anywhere that pants can run (MacOS, rpm linux distros, deb linux distros, other linux distros, docker, ...). ### Previous PRs in series - #22873 ## PR Overview This PR adds rules in `nfpm.native_libs` to add package dependency metadata to `nfpm_rpm_package`. The 2 new rules are: - `inject_native_libs_dependencies_in_package_fields`: - An implementation of the polymorphic rule `inject_nfpm_package_fields`. This rule is low priority (`priority = 2`) so that in-repo plugins can override/augment what it injects. (See #22864) - Rule logic overview: - find any pex_binaries that will be packaged in an `nfpm_rpm_package` (using utility introduced in #22863) - Run new `rpm_depends_from_pex` rule (see below) - Inject identified SONAMEs in `nfpm_rpm_package` dependency fields (rpm accepts raw SONAMEs in these fields, so the SONAME does not need to be translated to a package name when building the package). - The `requires` field gets SONAMEs required by ELF binaries or libraries in the package contents - The `provides` field gets SONAMEs provided by ELF libraries in the package contents - How the rule outputs are used: The package dependency fields (like `requires` and `provides`) will be used when generating the config passed to `nFPM` so that `nFPM` includes the package dependency metadata in the built rpm package. - `rpm_depends_from_pex`: - runs `elfdeps_analyze_pex` on a pex (added in #22873) - returns only the ELF metadata that can be injected in `nfpm_rpm_package` fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This extends then
pants.backend.experimental.nfpmbackend's plugin API, allowing multiple implementations of this polymorphic-rule plugin hook (before this change, only one implementation was allowed):inject_nfpm_package_fields(InjectNfpmPackageFieldsRequest) -> InjectedNfpmPackageFieldsUnlike many polymorphic/union rules, each rule runs sequentially instead of concurrently. This is conceptually similar to middleware, because the request passed to each rule includes the results of the previous (lower priority) rule. So, one rule can override or extend fields that were injected in previous rules, not just the original fields from the BUILD-file-based
nfpm_*_packagetarget.I tried several approaches to sorting the request classes, but most were error prone or simply didn't work. Heuristic-based sorting proved to be complex and didn't work in all of my test cases (eg one heuristic was looking at the class's module to make pants-provided requests lower priority). A simple
priorityinteger is far simpler and more reliable, so that's what this implements.Another thing that didn't work was adding
__lt__as a@classmethodon the abstractInjectNfpmPackageFieldsRequest. That fails becausesorted()requires an instance method, not a class method. The only way I found to add an__lt__method on a class is via a metaclass. So, this adds a_PrioritizedSortableClassMetaclasstoInjectNfpmPackageFieldsRequestallowing for simpleOneRequest < TwoRequestsorting of the class type like this:inject_nfpm_config_request_types = union_membership.get(InjectNfpmPackageFieldsRequest) applicable_inject_nfpm_config_request_types = tuple( - request - for request in inject_nfpm_config_request_types - if request.is_applicable(target) + sorted( + request_type + for request_type in inject_nfpm_config_request_types + if request_type.is_applicable(target) + ) )Then, to make subclasses higher priority than their parent classes, I added an
__init_subclass__method that increasescls.priority(unless the subclass already has a different priority than the parent class). It's a little odd to use both a metaclass and__init_subclass__, but I did not like the idea of re-implementing the on-subclass-creation logic in our metaclass.InjectNfpmPackageFieldsRequestwas initially modeled on the python backend'sSetupKwargsRequest. If the rule-chaining proves itself in the experimentalnfpmbackend, maybe we can move_PrioritizedSortableClassMetaclassout of the backend to somewhere in pants core, updatingSetupKwargsRequestand related rules to make use of rule-chaining as well.