Do clean up in cosa for bootc v1.6.0

[PeaceRebel]

As bootc v1.6.0 is expected to fix the reproducibility of ostree commit, these changes mainly contains changes to use ostree commit from meta.json

Closes: #4275

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[PeaceRebel]

=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- FAIL: fcos.upgrade.basic (50.43s)
    --- PASS: fcos.upgrade.basic/setup (2.71s)
    --- PASS: fcos.upgrade.basic/upgrade-from-previous (23.88s)
    --- FAIL: fcos.upgrade.basic/upgrade-from-current (0.08s)
            cluster.go:151: error: No such metadata object 61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11.commit
            cluster.go:173: "sudo ostree commit -b testupdate --tree=ref=61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11 --add-metadata-string version=44.20250825.dev.0.kola" failed: output , status Process exited with status 1
FAIL, output in tmp/kola

Looks like the issue still exist in the latest bootc version.

[jlebon]

=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- FAIL: fcos.upgrade.basic (50.43s)
    --- PASS: fcos.upgrade.basic/setup (2.71s)
    --- PASS: fcos.upgrade.basic/upgrade-from-previous (23.88s)
    --- FAIL: fcos.upgrade.basic/upgrade-from-current (0.08s)
            cluster.go:151: error: No such metadata object 61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11.commit
            cluster.go:173: "sudo ostree commit -b testupdate --tree=ref=61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11 --add-metadata-string version=44.20250825.dev.0.kola" failed: output , status Process exited with status 1
FAIL, output in tmp/kola

Looks like the issue still exist in the latest bootc version.

I think the issue there is that we needed new enough rpm-ostree (v2025.10, which pulls in the newer ostree-ext) in the starting disk image, i.e. the qcow2 that kola run-upgrade downloads. I think that's in all the streams we care about now, so it should work. Can you retry?

[PeaceRebel]

I think the issue there is that we needed new enough rpm-ostree (v2025.10, which pulls in the newer ostree-ext) in the starting disk image, i.e. the qcow2 that kola run-upgrade downloads. I think that's in all the streams we care about now, so it should work. Can you retry?

Using 44.20250908.91.0, the test fails with the same error.

EDIT: rpm-ostree version in the above image is rpm-ostree-2025.10-3.fc44.x86_64

[jlebon]

Hmm, and you're sure you're using new enough cosa that has the newer bootc as well?

[PeaceRebel]

Hmm, and you're sure you're using new enough cosa that has the newer bootc as well?

Yup, using local build from this branch.

[coreos-assembler]$ rpm -qa bootc
bootc-1.6.0-1.fc42.x86_64

[jlebon]

Can you dig into this more? E.g. compare the output of ostree show --raw $merge_commit between the two imports to show the raw commit objects (which will also include the commit metadata) and see where the difference is.

[jlebon]

I think bootc-dev/bootc#1637 could also be a factor here that breaks reproducibility.

[dustymabe]

@PeaceRebel what's the status of this?

[PeaceRebel]

@PeaceRebel what's the status of this?

This is parked for now due to the issue with bootc. I may be able to look at it after ignition subpackage work.