[ci] Add CLI tests and modernized CI to the release branch#4774
Merged
tobe2098 merged 11 commits intorelease/1.16from Apr 6, 2026
Merged
[ci] Add CLI tests and modernized CI to the release branch#4774tobe2098 merged 11 commits intorelease/1.16from
tobe2098 merged 11 commits intorelease/1.16from
Conversation
tobe2098
force-pushed
the
cli_tests_1.16
branch
2 times, most recently
from
dc10498 to
645b541
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## release/1.16 #4774 +/- ##
=============================================
Coverage 89.26% 89.26%
=============================================
Files 259 259
Lines 15798 15798
=============================================
Hits 14101 14101
Misses 1697 1697 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
tobe2098
changed the title
tobe2098
changed the title
tobe2098
force-pushed
the
cli_tests_1.16
branch
4 times, most recently
from
60da05b to
f1f47cd
Compare
tobe2098
changed the title
Contributor
Author
|
CLI tests are essentially untouched, so they do not require extensive review. |
Contributor
Author
|
I would outline that the main concerns are 1) the package versions are the same as on the 1.16.1 release 2) that the build system is not broken 3) that we publish the appropriate macos packages |
sharder996
requested changes
Apr 1, 2026
Contributor
Author
|
Added. The last one is empty because I had to manually add the changes in an earlier cherry-pick. |
This PR revamps how our GitHub workflows are invoked so that only the necessary workflows are run and we avoid situations where a change to the `README.md` gets blocked on an Apple runner failing to compile Multipass due to a spurious seg fault. This setup works by having a single orchestration workflow which filters the changed files with [dorny/paths-filter](https://github.com/dorny/paths-filter), defines relevant outputs, and then spawns the relevant workflows. A "Finalize" job in the orchestration workflow then checks the return statuses of the spawned workflows and sets it own return status accordingly. Regarding branch protection rules, the idea is that we would protect on `Dynamic-CI \ Filter changed files` and `Dynamic-CI \ Finalize` to make sure that the PR is properly analyzed and and errors from spawned workflows are propagated up, effectively gating the PR from being merged. Below are some example draft PRs demonstrating various scenarios and how the new CI setup reacts to them. 1. Documentation changes (#4136) - https://github.com/canonical/multipass/actions/runs/15478104252/job/43578947769?pr=4136 demonstrates a run that has been cancelled. 2. Documentation changes that fail to build (#4143) 3. Source code changes (#4137) 4. Changes that are neither documentation or source code (#4142) 5. Changing both documentation and code, but having one fail (#4144) Merge queues aren't officially supported in the `dorny/paths-filter` action, but there are several user submitted examples of people using the action in this manner. Because of repository permissions this has yet to have been tested. Another GitHub action; [tj-actions/changed-files](https://github.com/tj-actions/changed-files) was considered for added functionality of being able to spawn workflows. However, I chose not to go with it due to it's increased complexity and granularity of control that isn't strictly needed in our setup. One other thing, `docs/readthedocs.com:canonical-multipass` will continue to be built for every PR regardless of changed files. The trigger for this workflow is managed by [ReadtheDocs](https://app.readthedocs.com/dashboard/canonical-multipass/pull-requests/) which does not offer the granularity or control of being able to selectively control when the workflow is run. ### Future development One slight improvement that can be made to further optimize CI would be to bisect `windows-macos.yml` and further filter on files that only affect Windows or macOS builds. However, care would need to be taken in order to ensure that there is no unnoticed cross-platform dependencies in these filtered files. Closes #4136 Closes #4143 Closes #4137 Closes #4142 Closes #4144 --- MULTI-1932 ---
When running outside of a PR, `Linux` and `Windows/macOS` have the same concurrency group leading to them cancelling each other.
Contributor
Author
|
I re-did the cherry-picks, now it should be closer to actual commit history and conflict resolution was minimal. |
- Edited the schema for naming ccache dumps; `matrix.runs-on` is already a unique identifier so cleaned up some obtuse naming - Stopped throttling builds on ARM Mac - Removed cache size limit - Removed some outdated directories in `.gitignore` that are related to building with Travis CI - Removed wait for lint check; the output doesn't make a difference when gating a PR, so why gate compilation on it - Removed the `cla-check` workflow. A `cla-check` is required by ruleset on the Canonical org level, so this workflow was doing double duty
The arch wrapper is needed when cross-compiling, i.e. intel->arm and vice versa. The reason why the arch wrapper was needed in the beginning was there were no ARM based runners as described in this commit: > bbe964b Since then, the arch wrapper has basically became redundant. We now use the respective arch for producing intel and arm packages, hence making the wrapper redundant. Made several changes on the steps that depend on the "arch" variable: - Replaced the arch check in nuget shim replace step with a file type check. - Install nuget step now checks if nuget or mono is present or not. If not, then proceeds with installing them. MULTI-2355
According to the GH docs, macOS 15 will be the last version that will support Intel. The macos-15 runners will be available until Aug '27. This patch introduces the macos-15-intel runner to test-drive it. MULTI-2350
This PR adds the ability to build on the latest macOS runners while still targeting an older version of macOS. The `CMAKE_OSX_DEPLOYMENT_TARGET` is propagated to vcpkg so that all dependencies are also built for the correct version of macOS. Because OpenSSL was originally fetched via brew installation, its binary would inherently target the version of macOS that it was being installed on. To remedy this, OpenSSL is now fetched from vcpkg. `windows-macos.yml` is subsequently pruned to no longer spawn unneeded macOS runners. The deployment target can be queried from a Multipass binary with the command `otool -l multipassd | grep -A 3 LC_BUILD_VERSION`. Updated licence information in the installer with the appropriate licence as found in the [openssl repository](https://github.com/openssl/openssl). Also, updated the licence information for semver as its source recently changed in #4461. Verified on a macOS 14 virtual machine. Nested virtualization is only supported on M3 and newer, so you will need to remove the CPU checks in `preinstall-multipassd.sh` and `CPack.distribution.dist.in`. Closes #4023 Closes canonical/multipass-private#739 --- MULTI-1920 MULTI-2272
Set the minimum macOS build target for QEMU and libssh. QEMU still dynamically links against some brew dependencies which causes some warnings during the build process. Please let me know what the CLI tests reveal. --- MULTI-2396
Now that we don't have to target the macOS version that Multipass is being built on we simplify things a bit and target the same macOS version regardless of architecture. We're probably moving up to macOS 14 soon, but for now I went with the lower of 13.3 and 14.0.
Accurately set the build target for the Flutter GUI when on macOS. Do the same for dependencies installed with `pod`.
CLI tests aim to test the high-level, user-facing Multipass functionality via the CLI commands. The tests are written in Python, and the suite uses `pytest` as the test executor. See `README.md` for more details. Signed-off-by: Mustafa Kemal Gilor <[email protected]>
Some of the tests fail for legitimate reasons intermittently, and right now, we don't want to make the calling workflow fail if CLI tests fail.
sharder996
approved these changes
Apr 2, 2026
Collaborator
sharder996
left a comment
sharder996
left a comment
There was a problem hiding this comment.
CLI tests where entirely encapsulated in #4220 which I see in the commit history. As the title of the PR suggests, my approval encompasses the inclusion of the CLI tests and CI changes. Not necessarily approving the state of the release branch.
I will let @tobe2098 handle the merging of this into release/1.16.
xmkg
approved these changes
Apr 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.