Buildah supports selective layer (rootfs.diff_ids) squashing with intermediate image retention in multistage builds #6257

Hello,

I’m currently designing functionality for our internal build and release tool Konflux, which relies heavily on Buildah for container image building.

As part of this work, I’ve encountered a potential limitation in Buildah that I’d like to discuss. I'm also prepared to contribute an implementation if we can agree on the best approach.

Problem

In Konflux, we need to access content from intermediate images (e.g., packages, binaries) after the build process of multistage-build images. The only way I’m currently aware of to preserve these intermediate images is by using the --layers flag within buildah build. This works well for retaining intermediate layers, which we can then inspect post-build.

However, enabling --layers changes the structure of the final image’s metadata. Specifically:

• When using --layers, the rootfs.diff_ids of the final image includes all intermediate layers (undesired).

• When not using --layers, the final image contains only two layers in rootfs.diff_ids (desired):

  • One for the base image.
  • One for the squashed final image layers (I assume).

  But of course, no intermediate images are stored (undesired).

• In both cases, with or without --layers, the final image preserves the full history (also desired) containing parent base image and final image history.

• Using --squash alongside --layers does not achieve the desired result either. This ends up fully squashing everything (including the parent base image), which removes necessary layering details (parent base image layer + all other squashed). In addition to that, history is capturing only the last layer of the final image, but we would like to preserve full history as it is produced without squash.

What I cannot do

• Unfortunately, performing two separate builds (one with --layers for analysis and one without for the final image) is not viable for performance and pipeline simplicity reasons.
• I cannot use external tools like docker-squash, as they are heavily dependent on Docker and we want to certainly avoid that.

Desired Outcome

I’m looking for a way to:

• Preserve intermediate images for inspection.
• Have only two layers in the final image’s rootfs.diff_ids (parent base image layer + squashed final image layers).
• Retain history metadata for the final image (parent base image and final image history)

Example Containerfile && buildah build examples and outcomes

FROM docker.io/library/golang:1.21-bullseye AS builder

WORKDIR /app
RUN echo 'package main; import "fmt"; func main() { fmt.Println("Hello from Go!") }' > main.go && \
  go build -o hello main.go

FROM registry.access.redhat.com/ubi9/ubi:latest

RUN yum install python3-pip -y
RUN pip3 install semver==2.13.0

COPY --from=builder /app/hello /usr/local/bin/hello

CMD ["echo"]

Potential Proposed Solution

Based on current functionality, I believe we could introduce a new alternative flag: e.g., --layers-stages-only which would store intermediate stages as images, but squashes all filesystem changes of the final image layer chain, effectively yielding the same layer structure as a buildah build without --layers (parent base image sha + sha containing all other changes from final image manifested in rootfs.diff_ids).

Of course, I don't insist on the name of the argument, this is just an example, up for discussion.

Questions

• Did I miss anything? Is there currently any supported combination of flags in Buildah that would achieve this behavior?
• Would such functionality (separating intermediate stage storage from final image layering) be considered a reasonable feature to add? If so, I’d be happy to submit a patch or collaborate on an implementation aligned with Buildah's design philosophy.

Thank you for your time and consideration. I’d appreciate any guidance or thoughts on how best to proceed.