Add experimental oci repo plugin#591
Draft
lcarva wants to merge 1 commit intorpm-software-management:masterfrom
Draft
Add experimental oci repo plugin#591lcarva wants to merge 1 commit intorpm-software-management:masterfrom
lcarva wants to merge 1 commit intorpm-software-management:masterfrom
Conversation
This plugin allows using a YUM repo that is represented as an OCI Artifact by adding support to the `oci://` protocol. Example repo file: ``` [oci-repo-test] name=oci-repo-test baseurl=oci://quay.io/lucarval/yum-repo:latest enabled=1 gpgcheck=1 ``` When enabled, the plugin is activated for any enabled repo that uses the `oci://` protocol in its baseurl. The baseurl reference is expected to point to an OCI Artifact. During the `config` phase, it downloads the `repodata` from the OCI Artifact into a local temporary directory. This allows DNF to determine which packages can be installed from that repo. During the `resolved` phase, the plugin then downloads each RPM package that is marked for installation. It places those files alongside the repodata in the same temporary directory. This allows DNF to install those packages. Finally, during the `transaction` phase, the local temporary directory is removed as a clean up step. If you're interested in trying out this plugin locally: 1. Copy the `oci_repo.py` file to `/usr/lib/python3.12/site-packages/dnf-plugins/`, python version should match your system's python version. 2. Enable the plugin by creating `/etc/dnf/plugins/oci_repo.conf` with the following contents: ``` [main] enabled=1 ``` 3. Create yum repo file under `/etc/yum.repos.d/` that uses the `oci://` protocol. The example above should work. 4. Install the `python3-oras` RPM package. 5. Run `dnf install` as you would normally. NOTE: If using the example YUM repo above, you can install `cowsay` and `lolcat`. Be sure to disable other repos, e.g. `dnf install --disablerepo '*' --enablerepo oci-test-repo`. If you want to create your own OCI Artifact yum repo, start with an empty directory, copy/download all the RPMs you want to include into that directory, run `createrepo_c .` to create the repo metadata, then use `oras push <oci-ref> *` to create the OCI Artifact. The OCI Artifact is simply an OCI Image Manifest that contains each file as a separate blob/layer. Directories, i.e. `repodata`, are combined into a single blob/layer. The approach of downloading RPMs and repodata is not ideal since it does not leverage the existing DNF caching system. Also, for RPMs in particular, packages are downloaded if marked for installation, not if they are actually being installed. For example,if I choose to type "N" and not install them, they have already been downloaded. Signed-off-by: Luiz Carvalho <[email protected]>
cgwalters
reviewed
Sep 3, 2025
Comment on lines
+32
to
+33
| import oras.client # Install python3-oras RPM | ||
| import oras.defaults |
There was a problem hiding this comment.
Note that this stack does not share code with the podman stack, and in particular doesn't honor things in /etc/containers or handle auth files the same way.
That's not fatal, but https://github.com/containers/skopeo/blob/749370dd999e034d89227e1ca9e1391eb12ad58e/docs-experimental/skopeo-experimental-image-proxy.1.md#L4 was designed with use cases like this in mind and I'm of a mind to remove the experimental label from it.
| client = oras.client.OrasClient(hostname=hostname, insecure=insecure) | ||
|
|
||
| authfile = os.environ.get( | ||
| "REGSITRY_AUTH", os.path.expanduser("~/.docker/config.json") |
Comment on lines
+109
to
+112
| last_slash = url.rfind("/") | ||
| last_colon = url.rfind(":") | ||
| if last_colon > last_slash: | ||
| url = url[:last_colon] |
There was a problem hiding this comment.
I think there are subtle cases where this image parsing will fail
| tar_stream.seek(0) | ||
|
|
||
| with tarfile.open(fileobj=tar_stream, mode="r") as tar: | ||
| tar.extractall(path=dest, filter="data") |
There was a problem hiding this comment.
Is this python library robust against potentially malicious input? This would be a good thing to sandbox in a restricted subprocess (which is what podman/containers-storage does) or parse the tar to an in-memory VFS and defer regfiles to out of band (what composefs-rs does, see https://github.com/containers/composefs-rs/blob/main/crates/composefs-oci/src/tar.rs etc.)
Comment on lines
+160
to
+165
| if unpack: | ||
| tar_stream = io.BytesIO() | ||
| for chunk in r.iter_content(chunk_size=8192): | ||
| if chunk: | ||
| tar_stream.write(chunk) | ||
| tar_stream.seek(0) |
There was a problem hiding this comment.
Confused by streaming the whole thing to memory only to unpack to the fs, surely this can be done in a piped fashion?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This plugin allows using a YUM repo that is represented as an OCI Artifact by adding support to the
oci://protocol. Example repo file:When enabled, the plugin is activated for any enabled repo that uses the
oci://protocol in its baseurl. The baseurl reference is expected to point to an OCI Artifact.During the
configphase, it downloads therepodatafrom the OCI Artifact into a local temporary directory. This allows DNF to determine which packages can be installed from that repo.During the
resolvedphase, the plugin then downloads each RPM package that is marked for installation. It places those files alongside the repodata in the same temporary directory. This allows DNF to install those packages.Finally, during the
transactionphase, the local temporary directory is removed as a clean up step.If you're interested in trying out this plugin locally:
oci_repo.pyfile to/usr/lib/python3.12/site-packages/dnf-plugins/, python version should match your system's python version./etc/dnf/plugins/oci_repo.confwith the following contents:/etc/yum.repos.d/that uses theoci://protocol. The example above should work.python3-orasRPM package.dnf installas you would normally. NOTE: If using the example YUM repo above, you can installcowsayandlolcat. Be sure to disable other repos, e.g.dnf install --disablerepo '*' --enablerepo oci-test-repo.If you want to create your own OCI Artifact yum repo, start with an empty directory, copy/download all the RPMs you want to include into that directory, run
createrepo_c .to create the repo metadata, then useoras push <oci-ref> *to create the OCI Artifact. The OCI Artifact is simply an OCI Image Manifest that contains each file as a separate blob/layer. Directories, i.e.repodata, are combined into a single blob/layer.The approach of downloading RPMs and repodata is not ideal since it does not leverage the existing DNF caching system. Also, for RPMs in particular, packages are downloaded if marked for installation, not if they are actually being installed. For example,if I choose to type "N" and not install them, they have already been downloaded.