Embed Podman and drop Docker support

[apyrgio]

This PR introduces a very core change to Dangerzone, that is required for the Independent Container Updates feature (#1006). It adds first class support for Podman, by embedding it into our Windows / macOS packages, and using it internally to create VMs in these platforms where we can run containers. As a result, we drop Docker support, since it's no longer needed, and because it's not compatible with the format of the image we use in the Independent Containers Update feature.

This PR also introduces some visual changes. First, it allows users to select documents without waiting for the container installation to finish, or the Podman machine to start. We have measured that after the first time that Dangerzone runs, where initialization takes a while, the rest of the times require a few seconds to get Dangerzone starting, so we figued it's fine to let the user select documents, and do some work in the background. Second, we now have a widget that shows all the logs of our background tasks, which will help when debugging startup issues.

Finally, the integration of Podman allows us to do end-to-end CI tests on Windows and macOS platforms, which was previously highly difficult. This means that each PR, and more critically all of our Podman machine logic, will be tested on the two main platforms we are supporting, and we will catch errors sooner.

Here is a list of open issues that this PR affects:

• Closes Download and update Podman's machine-os images on Windows and macOS #1170
• Closes Create a module for Podman commands #1172
• Refs Introduce PodmanCommand containers/podman-py#558
• Closes Internals: make container_utils.Runtime and settings.Settings singletons #1168
• Fixes Remove Docker Desktop dependency for Windows and Mac #118
• Fixes Issue with Docker Desktop on Mac #1045
• Fixes Embed Podman as standalone binary #1145
• Fixes Let Dangerzone control the lifecycle of a Podman machine #1171
• Fixes Install container updates right away #1184
• Closes Invalid JSON return from container: See 'docker run --help' #253
• Closes Unable to find docker image (macOS, homebrew) #470
• Closes Check Docker Desktop version once we are sure that Docker is installed and running #1132
• Fixes Run the macOS docker-related tests on the CI #1009
• Closes Run Unsafe (development-only) Dangerzone version in MacOS and Windows for CI purposes #668

[almet]

Thanks for the work on this, @apyrgio, nicely done!

Here are some feedback after poking with the current implementation on a M1 machine:

1. Detection that podman is not available locally and needs to be installed.

When starting from a dev build, I've been granted with an error, saying that podman is not available locally, and needs to be installed. That's indeed the case, but I believe the message could be improved to mention using mazette install in order to get all the dependencies rather than "instaling podman", which indeed, won't do any difference now.

2. Update the colors used in the status bar text

At least when on dark mode, they aren't great, see the screenshot here. The red and green should be updated from my point of view. Using https://webaim.org/resources/contrastchecker/ can help ensure the contrast is good enough.

3. Error messages are duplicated, maybe due to how exceptions are handled

Maybe not directly related to your work, but I believe it's worth fixing here, see the screenshot:

4. When installing, I didn't see any wheel.

I'm not sure if it's because of the night mode or because of something else, but that was missing to me :-)

5. Update the dangerzone-image script to use the embedded podman

The scripts in dangerzone-image command should be updated to use a local connection to the embedded podman. Currently I receive (<Stderr: b'Cannot connect to Podman. Please verify your connection to the Linux system using podman system connection list, or try podman machine initandpodman machine start to manage a new Linux VM\nError: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:50723: connect: connection refused\n'>) errors.

6. Make it easier to download the signed container.tar archive

One thing that seem missing is the documentation about how to download the signed container.tar. In fact, since we don't have any released package, I'm not sure it's currently feasible. We have a dangerzone-testing package, but no packages are currently published as latest for now.

My way to solve this has been to fallback on the packages published on my own repositories, but that's not viable on the long run, obviously. One simple way would be to always tag the latest published package as latest.

[almet]

7. When debugging, knowning what the commands are can be useful

I did hack something together, but I believe the way to go would be to replace the subprocess_run function to add debug information for all commands. Here is what I did:

diff --git a/dangerzone/updater/cosign.py b/dangerzone/updater/cosign.py
index c2d7eaee..c83bfd94 100644
--- a/dangerzone/updater/cosign.py
+++ b/dangerzone/updater/cosign.py
@@ -1,3 +1,4 @@
+import shlex
 import subprocess
 from pathlib import Path
 from tempfile import NamedTemporaryFile
@@ -53,13 +54,15 @@ def verify_blob(pubkey: Path, bundle: str, payload: str) -> None:

 def download_signature(image: str, digest: str) -> list[str]:
     try:
+        args = [
+            _COSIGN_BINARY,
+            "download",
+            "signature",
+            f"{image}@sha256:{digest}",
+        ]
+        log.debug(shlex.join(args))
         process = subprocess_run(
-            [
-                _COSIGN_BINARY,
-                "download",
-                "signature",
-                f"{image}@sha256:{digest}",
-            ],
+            args,
             capture_output=True,
             check=True,
         )
@@ -71,8 +74,10 @@ def download_signature(image: str, digest: str) -> list[str]:


 def save(arch_image: str, destination: Path) -> None:
+    args = [_COSIGN_BINARY, "save", arch_image, "--dir", str(destination.absolute())]
+    log.debug(shlex.join(args))
     process = subprocess_run(
-        [_COSIGN_BINARY, "save", arch_image, "--dir", str(destination.absolute())],
+        args,
         capture_output=True,
     )
     if process.returncode != 0:

[almet]

dangerzone-machine remove command is not working for me:

When trying to remove the machine with poetry run ./dev_scripts/dangerzone-machine remove, I see the following error message:

Are you sure you want to remove machine 'dz-internal-0.9.1'? [y/N]: y
INFO:dangerzone.podman.machine:Removing Podman machine: dz-internal-0.9.1
❌ The Podman process failed with the following error: Command '['/opt/podman/bin/podman', 'machine', 'rm', 'dz-internal-0.9.1']' returned non-zero exit status 125.
Stdout: b'The following files will be deleted:\n\n\n/Users/alexis/.config/containers/podman/machine/applehv/dz-internal-0.9.1.json\n/var/folders/88/3q7jndss7yl_jsvlfmjh12z00000gn/T/podman/dz-internal-0.9.1.sock\n/var/folders/88/3q7jndss7yl_jsvlfmjh12z00000gn/T/podman/dz-internal-0.9.1-gvproxy.sock\n/var/folders/88/3q7jndss7yl_jsvlfmjh12z00000gn/T/podman/dz-internal-0.9.1-api.sock\n/var/folders/88/3q7jndss7yl_jsvlfmjh12z00000gn/T/podman/dz-internal-0.9.1.log\nAre you sure you want to continue? [y/N] '
Stderr: b'Error: EOF\n'
Aborted!

[apyrgio]

Nice catch! That was because the Podman wrapper I wrote did not always run podman machine remove with the --force flag. It doesn't make sense to prompt users in a low-level utility, and combined with the fact that stdin was closed, it triggered this error. Fixed in e2d81d3.

[apyrgio]

1. Detection that podman is not available locally and needs to be installed.

Fair point, we need to craft the message in a way that we mention mazette, but also be useful to end-users, in case their installation is borked somehow.

2. Update the colors used in the status bar text

At least when on dark mode, they aren't great, see the screenshot here. The red and green should be updated from my point of view. Using https://webaim.org/resources/contrastchecker/ can help ensure the contrast is good enough.

Damn, I didn't know this. Does this mean we need different GIF as well? Also, do we need some soft of palette project-wide for consistency? Pinging @harrislapiroff for this as well.

3. Error messages are duplicated, maybe due to how exceptions are handled

Maybe not directly related to your work, but I believe it's worth fixing here, see the screenshot:

It's directly related to this PR. I'm currently logging errors in the task level, and the startup manager level. This is just in case something we lose an exception in any of those two levels, but in the common path, we end up duplicating them. I think it's something we can fix later on, unless you have a smooth way to log once.

4. When installing, I didn't see any wheel.

I'm not sure if it's because of the night mode or because of something else, but that was missing to me :-)

Hm, the spinner looks like this:

It's kinda grey, so it may not bode well with dark mode. I think we need a white variant as well, as mentioned in point (2).

5. Update the dangerzone-image script to use the embedded podman

This is what should happen behind the scenes, so let's figure out what's going on here.

6. Make it easier to download the signed container.tar archive

We really need to publish our first container officially. Once we have this, it will work as a bootstrap.

7. When debugging, knowning what the commands are can be useful

Yup, we have discussed this need before. I'm capturing it here (#1236) and we can expand on it.

[almet]

Fair point, we need to craft the message in a way that we mention mazette, but also be useful to end-users, in case their installation is borked somehow.

We can actually detect if we're in dev mode and only mention mazette if that's the case.

Does this mean we need different GIF as well?

I'm not sure where I should see this GIF, and so I'm not sure if the reason I'm not seeing it is because of the dark mode or because of something else.

In case it's supposed to be shown where I expect it to be, e.g. next to the status bar, then I think we might need to have two GIFs. If we were in a web application, we would do that easily with CSS.

With Qt, I'm not sure how the dark mode switch is done. Do we need to listen to some kind of events, in case we could react to these accordingly, or maybe we could just decide on the GIF to load based on the selected theme at the time the status bar is shown to the user, which I think should be enough.

Another (simpler) way is to have a progress bar that works well for white and dark mode alike.

Do we need some soft of palette project-wide for consistency? Pinging @harrislapiroff for this as well.

If we don't have this already, yes that would be a good thing to have. I'm not sure how that would play with the redesign to come, and so I would advocate for something as simple as possible, with variants for light and dark mode.

I'm currently logging errors in the task level, and the startup manager level. This is just in case something we lose an exception in any of those two levels, but in the common path, we end up duplicating them. I think it's something we can fix later on, unless you have a smooth way to log once.

I see two classes of errors: a) errors at the startup manager level (failed to start a task for instance) and b) errors inside a task itself.

If we are catching the errors with a try/except block for the tasks we could log them there, and that should be enough. In case we want to make the fact that an error was caught known to the startup manager, then we could re-raise another exception, but without the details in it.

[almet]

I'm not sure where I should see this GIF, and so I'm not sure if the reason I'm not seeing it is because of the dark mode or because of something else.

Using the light mode made it clear to me where the GIF should be, and clearly it's not showing in dark mode.

It made me realize that our "dark mode" support is actually not updating properly the application, leading to half the UI being updated (e.g. the background but not the inputs change, leading to an ugly interface), see #1238 for this.

Another (simpler) way is to have a progress bar that works well for white and dark mode alike.

I've tried this approach and it works, switching to a color like orange, like this:

diff --git a/share/spinner.svg b/share/spinner.svg
index 91a2ee52..8587e34a 100644
--- a/share/spinner.svg
+++ b/share/spinner.svg
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" style="margin: auto; background: rgb(255, 255, 255); display: block; shape-rendering: auto;" width="200px" height="200px" viewBox="0 0 100 100" preserveAspectRatio="xMidYMid">
-  <circle cx="50" cy="50" fill="none" stroke="#333333" stroke-width="10" r="35" stroke-dasharray="164.93361431346415 56.97787143782138">
+  <circle cx="50" cy="50" fill="none" stroke="#FFAA33" stroke-width="10" r="35" stroke-dasharray="164.93361431346415 56.97787143782138">
     <animateTransform attributeName="transform" type="rotate" repeatCount="indefinite" dur="1s" values="0 50 50;360 50 50" keyTimes="0;1"></animateTransform>
   </circle>
-</svg>
\ No newline at end of file
+</svg>

[apyrgio]

Hey @almet. I believe I've addressed all your comments, both the ones you mentioned here, and some we had discussed live. I'm afraid I don't have the time right now to pinpoint the discussions above to specific commits, but if you take a look at the commits from d3c4dcb and onward, you'll see the changes there. Thanks again for your input 🙂