Skip to content

nfpm: support chaining multiple InjectNfpmPackageFieldsRequests (sorted w/ simple priority integer)#22864

Merged
cognifloyd merged 6 commits intomainfrom
cognifloyd/nfpm-inject_fields_chain
Nov 8, 2025
Merged

nfpm: support chaining multiple InjectNfpmPackageFieldsRequests (sorted w/ simple priority integer)#22864
cognifloyd merged 6 commits intomainfrom
cognifloyd/nfpm-inject_fields_chain

Conversation

@cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Nov 6, 2025

This extends then pants.backend.experimental.nfpm backend's plugin API, allowing multiple implementations of this polymorphic-rule plugin hook (before this change, only one implementation was allowed):

  • inject_nfpm_package_fields(InjectNfpmPackageFieldsRequest) -> InjectedNfpmPackageFields

Unlike many polymorphic/union rules, each rule runs sequentially instead of concurrently. This is conceptually similar to middleware, because the request passed to each rule includes the results of the previous (lower priority) rule. So, one rule can override or extend fields that were injected in previous rules, not just the original fields from the BUILD-file-based nfpm_*_package target.

I tried several approaches to sorting the request classes, but most were error prone or simply didn't work. Heuristic-based sorting proved to be complex and didn't work in all of my test cases (eg one heuristic was looking at the class's module to make pants-provided requests lower priority). A simple priority integer is far simpler and more reliable, so that's what this implements.

Another thing that didn't work was adding __lt__ as a @classmethod on the abstract InjectNfpmPackageFieldsRequest. That fails because sorted() requires an instance method, not a class method. The only way I found to add an __lt__ method on a class is via a metaclass. So, this adds a _PrioritizedSortableClassMetaclass to InjectNfpmPackageFieldsRequest allowing for simple OneRequest < TwoRequest sorting of the class type like this:

     inject_nfpm_config_request_types = union_membership.get(InjectNfpmPackageFieldsRequest)
     applicable_inject_nfpm_config_request_types = tuple(
-        request
-        for request in inject_nfpm_config_request_types
-        if request.is_applicable(target)
+        sorted(
+            request_type
+            for request_type in inject_nfpm_config_request_types
+            if request_type.is_applicable(target)
+        )
     )

Then, to make subclasses higher priority than their parent classes, I added an __init_subclass__ method that increases cls.priority (unless the subclass already has a different priority than the parent class). It's a little odd to use both a metaclass and __init_subclass__, but I did not like the idea of re-implementing the on-subclass-creation logic in our metaclass.

InjectNfpmPackageFieldsRequest was initially modeled on the python backend's SetupKwargsRequest. If the rule-chaining proves itself in the experimental nfpm backend, maybe we can move _PrioritizedSortableClassMetaclass out of the backend to somewhere in pants core, updating SetupKwargsRequest and related rules to make use of rule-chaining as well.

@cognifloyd
Copy link
Member Author

I extracted this PR from #22861

@cognifloyd cognifloyd requested a review from benjyw November 7, 2025 01:23
@cognifloyd cognifloyd merged commit a20fa8d into main Nov 8, 2025
28 checks passed
@cognifloyd cognifloyd deleted the cognifloyd/nfpm-inject_fields_chain branch November 8, 2025 01:49
cognifloyd added a commit that referenced this pull request Nov 20, 2025
…22899)

## PR Series Overview

This is the second in a series of PRs that introduces a new backend:
`pants.backend.npm.native_libs`
Initially, the backend will be available as:
`pants.backend.experimental.nfpm.native_libs`

I proposed this new backend (originally named `bindeps`) in discussion
#22396.

This backend will inspect ELF bin/lib files (like `lib*.so`) in packaged
contents (for this PR series, only in `pex_binary` targets) to identify
package dependency metadata and inject that metadata on the relevant
`nfpm_deb_package` or `nfpm_rpm_package` targets. Effectively, it will
provide an approximation of these native packager features:
- `rpm`: `rpmdeps` + `elfdeps`
- `deb`: `dh_shlibdeps` + `dpkg-shlibdeps` (These substitute
`${shlibs:Depends}` in debian control files have)

### Goal: Host-agnostic package builds

This pants backend is designed to be host-agnostic, like
[nFPM](https://nfpm.goreleaser.com/).

Native packaging tools are often restricted to a single release of a
single distro. Unlike native package builders, this new pants backend
does not use any of those distro-specific or distro-release-specific
utilities or local package databases. This new backend should be able to
run (help with building deb and rpm packages) anywhere that pants can
run (MacOS, rpm linux distros, deb linux distros, other linux distros,
docker, ...).

### Previous PRs in series

- #22873

## PR Overview

This PR adds rules in `nfpm.native_libs` to add package dependency
metadata to `nfpm_rpm_package`. The 2 new rules are:

- `inject_native_libs_dependencies_in_package_fields`:

    - An implementation of the polymorphic rule `inject_nfpm_package_fields`.
      This rule is low priority (`priority = 2`) so that in-repo plugins can
      override/augment what it injects. (See #22864)

    - Rule logic overview:
        - find any pex_binaries that will be packaged in an `nfpm_rpm_package`
          (using utility introduced in #22863)
        - Run new `rpm_depends_from_pex` rule (see below)
        - Inject identified SONAMEs in `nfpm_rpm_package` dependency fields
          (rpm accepts raw SONAMEs in these fields, so the SONAME does not need to
          be translated to a package name when building the package).
            - The `requires` field gets SONAMEs required by ELF binaries or
              libraries in the package contents
            - The `provides` field gets SONAMEs provided by ELF libraries in the
              package contents

    - How the rule outputs are used: The package dependency fields (like
      `requires` and `provides`) will be used when generating the config
      passed to `nFPM` so that `nFPM` includes the package dependency metadata
      in the built rpm package.

- `rpm_depends_from_pex`:
    - runs `elfdeps_analyze_pex` on a pex (added in #22873)
    - returns only the ELF metadata that can be injected in
      `nfpm_rpm_package` fields.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants