Improve chroot guessing on EL#586
Improve chroot guessing on EL#586ekohl wants to merge 5 commits intorpm-software-management:masterfrom
Conversation
The DNF repository download accepts the full chroot name and doesn't need the arch to be passed in via a separate parameter. Non-existing chroots will be reported as well. There's no need to have the chroot parts as an instance variable if it's only used once so this also passes it in as a parameter.
Copr can provide available chroots when a repository is not found. When a user hasn't provided the chroot, we can improve our guessing. The result is that it'll always prefer epel, but can also use the centos-stream or rhel buildroot when available. This makes it easier for projects that build without EPEL. This is an alternative take on 313470b.
|
Verified on a plain CentOS Stream 9. Reproducer of the problem: # dnf install -y -q dnf-plugins-core
Importing GPG key 0x8483C65D:
Userid : "CentOS (CentOS Official Signing Key) <[email protected]>"
Fingerprint: 99DB 70FA E1D7 CE22 7FB6 4882 05B5 55B3 8483 C65D
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
Installed:
dbus-libs-1:1.12.20-8.el9.x86_64 dnf-plugins-core-4.3.0-21.el9.noarch python3-dateutil-1:2.8.1-7.el9.noarch
python3-dbus-1.2.18-2.el9.x86_64 python3-dnf-plugins-core-4.3.0-21.el9.noarch python3-six-1.15.0-9.el9.noarch
python3-systemd-234-19.el9.x86_64 systemd-libs-252-53.el9.x86_64
# dnf copr enable -y @theforeman/foremanctl
Enabling a Copr repository. Please note that this repository is not part
of the main distribution, and quality may vary.
The Fedora Project does not exercise any power over the contents of
this repository beyond the rules outlined in the Copr FAQ at
<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,
and packages are not held to any quality or security level.
Please do not file bug reports about these packages in Fedora
Bugzilla. In case of problems, contact the owner of this repository.
Error: It wasn't possible to enable this project.
Repository 'epel-9-x86_64' does not exist in project '@theforeman/foremanctl'.
Available repositories: 'fedora-rawhide-x86_64', 'centos-stream-9-x86_64', 'fedora-42-x86_64', 'rhel-9-x86_64', 'fedora-41-x86_64'
If you want to enable a non-default repository, use the following command:
'dnf copr enable @theforeman/foremanctl <repository>'
But note that the installed repo file will likely need a manual modification.Now with the patch applied: # dnf install -y -q patch
Installed:
ed-1.14.2-12.el9.x86_64 info-6.7-15.el9.x86_64 patch-2.7.6-16.el9.x86_64
# cd /usr/lib/python3.9/site-packages/dnf-plugins
# curl -L https://github.com/rpm-software-management/dnf-plugins-core/pull/586.patch | patch -p2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 13162 100 13162 0 0 27709 0 --:--:-- --:--:-- --:--:-- 55535
patching file copr.py
Hunk #1 succeeded at 244 (offset -19 lines).
Hunk #2 succeeded at 292 (offset -19 lines).
Hunk #3 succeeded at 472 (offset -21 lines).
Hunk #4 succeeded at 489 (offset -21 lines).
Hunk #5 succeeded at 684 (offset -21 lines).
patching file copr.py
Hunk #1 succeeded at 477 (offset -21 lines).
Hunk #2 succeeded at 503 (offset -21 lines).
patching file copr.py
Hunk #1 succeeded at 472 (offset -21 lines).
Hunk #2 succeeded at 503 (offset -21 lines).
patching file copr.py
Hunk #1 succeeded at 249 (offset -19 lines).
Hunk #2 succeeded at 505 (offset -21 lines).
patching file copr.py
Hunk #1 succeeded at 77 (offset -1 lines).
Hunk #2 succeeded at 442 (offset -19 lines).
Hunk #3 succeeded at 476 with fuzz 2 (offset -21 lines).
Hunk #4 succeeded at 508 (offset -21 lines).
Hunk #5 succeeded at 529 (offset -21 lines).
# dnf copr enable -y @theforeman/foremanctl
Enabling a Copr repository. Please note that this repository is not part
of the main distribution, and quality may vary.
The Fedora Project does not exercise any power over the contents of
this repository beyond the rules outlined in the Copr FAQ at
<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,
and packages are not held to any quality or security level.
Please do not file bug reports about these packages in Fedora
Bugzilla. In case of problems, contact the owner of this repository.
Repository successfully enabled.
# cat /etc/yum.repos.d/
_copr:copr.fedorainfracloud.org:group_theforeman:foremanctl.repo
centos-addons.repo
centos.repo
# cat /etc/yum.repos.d/_copr\:copr.fedorainfracloud.org\:group_theforeman\:foremanctl.repo
[copr:copr.fedorainfracloud.org:group_theforeman:foremanctl]
name=Copr repo for foremanctl owned by @theforeman
baseurl=https://download.copr.fedorainfracloud.org/results/@theforeman/foremanctl/centos-stream-$releasever-$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/@theforeman/foremanctl/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1Then on AlmaLinux 9 with the patch applied: # dnf copr enable -y @theforeman/foremanctl
Enabling a Copr repository. Please note that this repository is not part
of the main distribution, and quality may vary.
The Fedora Project does not exercise any power over the contents of
this repository beyond the rules outlined in the Copr FAQ at
<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,
and packages are not held to any quality or security level.
Please do not file bug reports about these packages in Fedora
Bugzilla. In case of problems, contact the owner of this repository.
Repository successfully enabled.
# cat /etc/yum.repos.d/_copr\:copr.fedorainfracloud.org\:group_theforeman\:foremanctl.repo
[copr:copr.fedorainfracloud.org:group_theforeman:foremanctl]
name=Copr repo for foremanctl owned by @theforeman
baseurl=https://download.copr.fedorainfracloud.org/results/@theforeman/foremanctl/rhel-9-$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/@theforeman/foremanctl/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1Note how it chose the correct buildroot: |
| guesses = ['epel', 'rhel'] | ||
| if "CentOS Stream" in dist: | ||
| guesses.insert(1, 'centos-stream') |
There was a problem hiding this comment.
#525 (comment) showed me there's also the centos-stream+epel-next chroot. Should we support that too?
| guesses = ['epel', 'rhel'] | |
| if "CentOS Stream" in dist: | |
| guesses.insert(1, 'centos-stream') | |
| if "CentOS Stream" in dist: | |
| guesses = ['centos-stream+epel-next', 'epel', 'centos-stream', 'rhel'] | |
| else: | |
| guesses = ['epel', 'rhel'] |
There was a problem hiding this comment.
The centos-stream+epel-next chroots are tricky. For versions 8 and 9 (the only ones we have EPEL Next), real systems should be one of:
- RHEL + EPEL
- CentOS Stream + EPEL + EPEL Next
But in mock-core-configs we have all of these:
rhel+epelcentos-stream+epelcentos-stream+epel-next
For versions 8 and 9, EPEL on CentOS Stream means both EPEL and EPEL Next. Using only EPEL without EPEL Next on CentOS Stream is not valid and will result in uninstallable packages (albeit rarely). I think the centos-stream+epel-next chroots should be retired and centos-stream+epel chroots should be updated to also include the epel-next template (matching what centos-stream+epel-next is now). If we do that, there won't be any need to include logic in the copr plugin for centos-stream+epel-next.
Also, EPEL 8 Next is already EOL, EPEL 9 Next goes EOL in about two years, and EPEL 10 uses a totally different structure that obsoletes EPEL Next.
|
Thanks for working on this! I'd definitely like to see something like this implemented. The current logic makes sense for EPEL 8/9, but I think it's not quite right for EPEL 10. The
I don't know if it's worth the effort to assume the first option for each distro and then check for alternatives, or if we should just fetch the available chroots first and then pick the best one. |
|
@carlwgeorge thanks for the vision piece. I think you clearly identified I was still firmly in the EPEL 8/9 mindset.
There could be some COPR service load impact, though it's probably quite cheap. Still, perhaps worth checking with the COPR maintainers? I would still like to see this implemented, but am currently busy. If anyone wants to pick this up and address @carlwgeorge's concerns then please. |
I added this PR on team's kanban, we should get to the review soon. |
|
Not a review, just a note. Have you looked at the api for DNF5? Example call: https://copr.fedorainfracloud.org/api_3/rpmrepo/@copr/copr-pull-requests/fedora-rawhide-x86_64 That gives you a full introspection into given Copr repository via a single API query. The problem is that the query isn't actually generating repofile (that is a task for the client). My assumption was that it's too late to implement the same thing in dnf-plugins-core, because Fedora has already moved to DNF5 and getting such a huge change into Enterprise Linux packages would be complicated. |
|
Hi @ ekohl thanks for working on this. The main logic change in 9caeff1 looks good to me. If it increase user friendliness for folks using the plugin on centos +1. I also afree with what @ carlwgeorge mentioned about epel next, that would be good for e.g. follow-up PR. But one thing from my side as a Copr maintainer: I don't like the refactoring commits. The current code might not be perfect, but it works, and from a perspective of someone who should maintain this code, but doesn't actively participate in this repo: if it works, no touchy! :D But if something will go wrong with it and it would be fixed by someone else (I doubt that) , then I don't mind, so I'd lean towards keeping the changes minimal.
I don't think that's an issue, copr can handle it, the impact will be minimal I think.
I agree with Pavel here, all the info can be found via one API call here if it is necesarry for implementing the more complex part with the epel next. So overall from me, +1 to the 9caeff1 and small -1 to the refactoring unless it won't backfire on us :) |
4 participants
When a project is not built in the
epelbuildroot but instead incentos-streamorrhelthe commandcopr enable $coprwill fail.Copr can provide available chroots when a repository is not found and this information is already shown to the user. we can improve our guessing with this information when a user hasn't provided the chroot.
The result is that it'll always prefer
epel, but can also use thecentos-streamor therhelbuildroot when available. This makes it easier for projects that build without EPEL.The PR is organized into some refactoring commits that shouldn't affect anything. The result is that the final commit actually making the change is relatively small. I've chosen to first attempt the primary guess instead of probing for available repos first. This should result in lower server load for copr because in the majority of cases the guess is still valid.
This is an alternative take on #525.
Fixes #524