Skip to content

[converter] Add MergeManifest option to tell converter when manifests are merged#674

Merged
imeoer merged 1 commit intocontainerd:mainfrom
DataDog:fricounet/merge-manifest-config
Sep 9, 2025
Merged

[converter] Add MergeManifest option to tell converter when manifests are merged#674
imeoer merged 1 commit intocontainerd:mainfrom
DataDog:fricounet/merge-manifest-config

Conversation

@Fricounet
Copy link
Contributor

@Fricounet Fricounet commented Sep 9, 2025

Overview

Please briefly describe the changes your pull request makes.

When both the OCI and nydus manifests are merged with nydusify --merge-platform for instance, containerd will try to pull an unpack both images even if it doesn't have a nydus-snapshotter. Which results in the nydus image failing to be unpacked and a global image pull failure. One way to fix this issue is to use a config mediaType that containerd doesn't understand in the nydus image. This way it won't attempt to unpack it while the nydus snapshotter can still discover the image and pull it.

We use application/vnd.nydus.image.config.v1+json as the new config value only used if MergeManifest is set in the MergeOption.

Related Issues

Please link to the relevant issue. For example: Fix #123 or Related #456.

See #669 (comment) for context

Change Details

Please describe your changes in detail:

Test Results

If you have any relevant screenshots or videos that can help illustrate your changes, please add them here.

When propagating the change up to nydusify (see all the changes done in nydusify directly), this will result in an artifact as follow (note the config mediaType):

crane manifest localhost:5000/image:v1.3.0-nydus-platform-config@sha256:4aaffab81483a36fa808f16c0476c6968ce042e6ad7fd734b79f497a0591730d
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.nydus.image.config.v1+json",
    "digest": "sha256:3c374743d776cfe42e92a6795e788ec9d3bf2ec7a30dc3a263b37ee433045083",
    "size": 8712
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.nydus.blob.v1",
      "digest": "sha256:a2fec6caf440b76f26fa3366fa2f4e4b31af4e27572acc8d571afc244dd01c5d",
      "size": 8389,
      "annotations": {
        "containerd.io/snapshot/nydus-blob": "true"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.layer.nydus.blob.v1",
      "digest": "sha256:78b1c6220ef98ad2518018d11f35dd391e186f7efc48ac7fa1946278b9334aff",
      "size": 56627718,
      "annotations": {
        "containerd.io/snapshot/nydus-blob": "true"
      }
    },
...

Change Type

Please select the type of change your pull request relates to:

  • Bug Fix
  • Feature Addition
  • Documentation Update
  • Code Refactoring
  • Performance Improvement
  • Other (please describe)

Self-Checklist

Before submitting a pull request, please ensure you have completed the following:

  • I have run a code style check and addressed any warnings/errors.
  • I have added appropriate comments to my code (if applicable).
  • I have updated the documentation (if applicable).
  • I have written appropriate unit tests.

… are merged

When both the OCI and nydus manifests are merged with `nydusify --merge-platform` for instance, containerd will try to pull an unpack both images even if it doesn't have a nydus-snapshotter. Which results in the nydus image failing to be unpacked and a global image pull failure. One way to fix this issue is to use a config mediaType that containerd doesn't understand in the nydus image. This way it won't attempt to unpack it while the nydus snapshotter can still discover the image and pull it.

We use `application/vnd.nydus.image.config.v1+json` as the new config value only used if MergeManifest is set in the MergeOption.
@codecov
Copy link

codecov bot commented Sep 9, 2025

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.62%. Comparing base (d2c042a) to head (fe9ac56).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
pkg/converter/convert_unix.go 0.00% 3 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #674      +/-   ##
==========================================
- Coverage   20.62%   20.62%   -0.01%     
==========================================
  Files         122      122              
  Lines       13706    13709       +3     
==========================================
  Hits         2827     2827              
- Misses      10561    10564       +3     
  Partials      318      318              
Files with missing lines Coverage Δ
pkg/converter/types.go 0.00% <ø> (ø)
pkg/converter/convert_unix.go 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@imeoer imeoer merged commit e44c981 into containerd:main Sep 9, 2025
16 checks passed
@imeoer
Copy link
Collaborator

imeoer commented Sep 9, 2025

Thanks!

Fricounet added a commit to Fricounet/acceleration-service that referenced this pull request Sep 9, 2025
Use MergeOption.MergeManifest introduced in nydus-snapshotter v0.15.3 (containerd/nydus-snapshotter#674)
Fricounet added a commit to Fricounet/acceleration-service that referenced this pull request Sep 9, 2025
Use MergeOption.MergeManifest introduced in nydus-snapshotter v0.15.3 (containerd/nydus-snapshotter#674)

Signed-off-by: Baptiste Girard-Carrabin <baptiste.girardcarrabin@datadoghq.com>
imeoer pushed a commit to goharbor/acceleration-service that referenced this pull request Sep 9, 2025
Use MergeOption.MergeManifest introduced in nydus-snapshotter v0.15.3 (containerd/nydus-snapshotter#674)

Signed-off-by: Baptiste Girard-Carrabin <baptiste.girardcarrabin@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants