Skip to content

Switch autostart event in favor of only start#758

Closed
ben-grande wants to merge 1 commit intoQubesOS:mainfrom
ben-grande:preload-autostart-refill
Closed

Switch autostart event in favor of only start#758
ben-grande wants to merge 1 commit intoQubesOS:mainfrom
ben-grande:preload-autostart-refill

Conversation

@ben-grande
Copy link
Contributor

@ben-grande ben-grande commented Dec 3, 2025

The autostart made a refresh, which cleaned all preloaded disposables, served to:

  • Invalidate previous incomplete remnants
  • Get fresh ones after a change was made to a template in chain

Both of these issues were handled, "domain-load" of the disposable template invalidates incomplete ones while "domain-shutdown" of a template in chain triggers a refresh.

For: QubesOS/qubes-issues#1512

@codecov
Copy link

codecov bot commented Dec 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.54%. Comparing base (1186794) to head (48e3c99).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #758      +/-   ##
==========================================
- Coverage   70.74%   70.54%   -0.20%     
==========================================
  Files          61       61              
  Lines       13809    13807       -2     
==========================================
- Hits         9769     9740      -29     
- Misses       4040     4067      +27     
Flag Coverage Δ
unittests 70.54% <100.00%> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@qubesos-bot
Copy link

qubesos-bot commented Dec 5, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025121419-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025111104-4.3&flavor=update

  • system_tests_dispvm

    • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
      qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
  • system_tests_qwt_win10_seamless@hw13

  • system_tests_gui_interactive_preloaded

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/Pw7Zf-\d+-/...

    • collect_logs: Failed (test died + timed out)
      # Test died: command 'curl --form [email protected] --form upn...

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/fQRNV-\d+-/...

  • system_tests_dispvm_perf@hw7

    • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

    • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
      raise TimeoutError from exc_val... TimeoutError
  • system_tests_basic_vm_qrexec_gui_xfs

Failed tests

12 failures
  • system_tests_dispvm

    • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
      qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: Failed (test died)
      # Test died: Install failed with code 1 at qubesos/tests/windows_in...
  • system_tests_gui_interactive_preloaded

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/Pw7Zf-\d+-/...

    • collect_logs: Failed (test died + timed out)
      # Test died: command 'curl --form [email protected] --form upn...

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/fQRNV-\d+-/...

  • system_tests_dispvm_perf@hw7

    • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

    • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

    • TC_00_DispVMPerf_whonix-workstation-18: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
      raise TimeoutError from exc_val... TimeoutError
  • system_tests_basic_vm_qrexec_gui_xfs

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/158999#dependencies

18 fixed
  • system_tests_extra

    • system_tests: Fail (unknown)
      Tests qubes.tests.extra failed (exit code 1), details reported sepa...

    • system_tests: Failed (test died)
      # Test died: Some tests failed at qubesos/tests/system_tests.pm lin...

    • TC_00_QVCTest_debian-13-xfce: test_010_screenshare (failure + cleanup)
      AssertionError: 2.4614345149565264 not less than 2.0

    • TC_00_QVCTest_whonix-gateway-18: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

    • TC_00_QVCTest_whonix-workstation-18: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • system_tests_guivm_gui_interactive

    • gui_keyboard_layout: unnamed test (unknown)
    • gui_keyboard_layout: Failed (test died)
      # Test died: no candidate needle with tag(s) 'work-xterm, work-xter...
  • system_tests_dispvm

    • TC_20_DispVM_whonix-workstation-18: test_030_edit_file (failure)
      AssertionError: Timeout waiting for editor window

    • TC_20_DispVM_whonix-workstation-18: test_100_open_in_dispvm (failure)
      AssertionError: './open-file test.txt' failed with ./open-file test...

  • system_tests_guivm_vnc_gui_interactive

    • gui_filecopy: unnamed test (unknown)
    • gui_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'disp-text-editor' mat...
  • system_tests_qwt_win10@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/kauF4-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_qwt_win10_seamless@hw13

  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/iDVvW-\d+-/...
  • system_tests_dispvm_perf@hw7

Unstable tests

Details

Performance Tests

Performance degradation:

11 performance degradations
  • debian-13-xfce_exec: 7.99 🔻 ( previous job: 6.42, degradation: 124.42%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 307771.00 🔻 ( previous job: 491827.00, degradation: 62.58%)
  • fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 119277.00 🔻 ( previous job: 137220.00, degradation: 86.92%)
  • fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 307500.00 🔻 ( previous job: 355690.00, degradation: 86.45%)
  • fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 1992.00 🔻 ( previous job: 3964.00, degradation: 50.25%)
  • fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 33249.00 🔻 ( previous job: 49472.00, degradation: 67.21%)
  • fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 17538.00 🔻 ( previous job: 75326.00, degradation: 23.28%)
  • fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 119920.00 🔻 ( previous job: 140939.00, degradation: 85.09%)
  • fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 271511.00 🔻 ( previous job: 349176.00, degradation: 77.76%)
  • fedora-42-xfce_dom0-vm-api (mean:0.037): 0.45 🔻 ( previous job: 0.39, degradation: 114.29%)
  • fedora-42-xfce_dom0-vm-gui-api (mean:0.056): 0.67 🔻 ( previous job: 0.54, degradation: 123.70%)

Remaining performance tests:

84 tests
  • debian-13-xfce_exec-root: 27.12 🟢 ( previous job: 27.48, improvement: 98.68%)
  • debian-13-xfce_socket: 8.15 🟢 ( previous job: 8.68, improvement: 93.91%)
  • debian-13-xfce_socket-root: 8.24 🟢 ( previous job: 8.40, improvement: 98.05%)
  • debian-13-xfce_exec-data-simplex: 68.10 🔻 ( previous job: 67.70, degradation: 100.60%)
  • debian-13-xfce_exec-data-duplex: 69.13 🟢 ( previous job: 69.20, improvement: 99.90%)
  • debian-13-xfce_exec-data-duplex-root: 79.25 🟢 ( previous job: 86.64, improvement: 91.46%)
  • debian-13-xfce_socket-data-duplex: 133.83 🟢 ( previous job: 136.31, improvement: 98.18%)
  • fedora-42-xfce_exec: 9.17 🟢 ( previous job: 9.23, improvement: 99.33%)
  • fedora-42-xfce_exec-root: 59.89 🟢 ( previous job: 59.89, improvement: 100.00%)
  • fedora-42-xfce_socket: 8.07 🟢 ( previous job: 8.23, improvement: 98.02%)
  • fedora-42-xfce_socket-root: 8.48 🔻 ( previous job: 8.31, degradation: 102.00%)
  • fedora-42-xfce_exec-data-simplex: 69.42 🔻 ( previous job: 66.18, degradation: 104.90%)
  • fedora-42-xfce_exec-data-duplex: 65.14 🟢 ( previous job: 71.51, improvement: 91.10%)
  • fedora-42-xfce_exec-data-duplex-root: 99.91 🔻 ( previous job: 96.52, degradation: 103.51%)
  • fedora-42-xfce_socket-data-duplex: 138.43 🔻 ( previous job: 137.78, degradation: 100.47%)
  • whonix-gateway-18_exec: 8.51 🟢 ( previous job: 8.52, improvement: 99.93%)
  • whonix-gateway-18_exec-root: 131.84 🔻 ( previous job: 129.05, degradation: 102.16%)
  • whonix-gateway-18_socket: 8.05 🟢 ( previous job: 8.26, improvement: 97.46%)
  • whonix-gateway-18_socket-root: 8.26 🔻 ( previous job: 7.76, degradation: 106.54%)
  • whonix-gateway-18_exec-data-simplex: 69.13 🟢 ( previous job: 73.13, improvement: 94.54%)
  • whonix-gateway-18_exec-data-duplex: 70.60 🟢 ( previous job: 71.98, improvement: 98.08%)
  • whonix-gateway-18_exec-data-duplex-root: 147.92 🔻 ( previous job: 146.68, degradation: 100.85%)
  • whonix-gateway-18_socket-data-duplex: 141.12 🟢 ( previous job: 142.71, improvement: 98.89%)
  • whonix-workstation-18_exec: 8.93 🔻 ( previous job: 8.85, degradation: 100.86%)
  • whonix-workstation-18_exec-root: 140.56 🟢 ( previous job: 149.84, improvement: 93.80%)
  • whonix-workstation-18_socket: 9.17 🔻 ( previous job: 9.12, degradation: 100.53%)
  • whonix-workstation-18_socket-root: 8.09 🟢 ( previous job: 8.75, improvement: 92.49%)
  • whonix-workstation-18_exec-data-simplex: 66.99 🟢 ( previous job: 67.02, improvement: 99.96%)
  • whonix-workstation-18_exec-data-duplex: 71.06 🔻 ( previous job: 69.25, degradation: 102.61%)
  • whonix-workstation-18_exec-data-duplex-root: 140.98 🟢 ( previous job: 146.32, improvement: 96.35%)
  • whonix-workstation-18_socket-data-duplex: 132.74 🟢 ( previous job: 139.15, improvement: 95.39%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 439470.00 🟢 ( previous job: 365230.00, improvement: 120.33%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 220335.00 🔻 ( previous job: 231269.00, degradation: 95.27%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 103750.00 🔻 ( previous job: 110619.00, degradation: 93.79%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 123452.00 🔻 ( previous job: 130397.00, degradation: 94.67%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 43982.00 🟢 ( previous job: 39923.00, improvement: 110.17%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 7643.00 🟢 ( previous job: 3264.00, improvement: 234.16%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 12138.00 🟢 ( previous job: 12000.00, improvement: 101.15%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 3874.00 🟢 ( previous job: 2099.00, improvement: 184.56%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 326354.00 🟢 ( previous job: 103611.00, improvement: 314.98%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 433295.00 🟢 ( previous job: 432938.00, improvement: 100.08%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 186794.00 🟢 ( previous job: 143368.00, improvement: 130.29%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 105175.00 🟢 ( previous job: 103977.00, improvement: 101.15%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 9359.00 🔻 ( previous job: 10236.00, degradation: 91.43%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7828.00 🔻 ( previous job: 8300.00, degradation: 94.31%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 5154.00 🟢 ( previous job: 3936.00, improvement: 130.95%)
  • fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 393609.00 🟢 ( previous job: 346866.00, improvement: 113.48%)
  • fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 53519.00 🟢 ( previous job: 26931.00, improvement: 198.73%)
  • fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 77268.00 🔻 ( previous job: 85360.00, degradation: 90.52%)
  • fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9402.00 🟢 ( previous job: 8945.00, improvement: 105.11%)
  • fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 757.00 🟢 ( previous job: 457.00, improvement: 165.65%)
  • fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 370783.00 🔻 ( previous job: 371835.00, degradation: 99.72%)
  • fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 129102.00 🟢 ( previous job: 108858.00, improvement: 118.60%)
  • fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 371308.00 🟢 ( previous job: 351871.00, improvement: 105.52%)
  • fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 2033.00 🟢 ( previous job: 1909.00, improvement: 106.50%)
  • fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 7790.00 🔻 ( previous job: 8365.00, degradation: 93.13%)
  • fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1050.00 🟢 ( previous job: 517.00, improvement: 203.09%)
  • fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 364215.00 🟢 ( previous job: 361328.00, improvement: 100.80%)
  • fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 52199.00 🟢 ( previous job: 46842.00, improvement: 111.44%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 85067.00 🟢 ( previous job: 79581.00, improvement: 106.89%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2409.00 🟢 ( previous job: 2110.00, improvement: 114.17%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8603.00 🟢 ( previous job: 7708.00, improvement: 111.61%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1150.00 🟢 ( previous job: 861.00, improvement: 133.57%)
  • debian-13-xfce_dom0-dispvm-api (mean:6.966): 83.59 🟢 ( previous job: 84.68, improvement: 98.72%)
  • debian-13-xfce_dom0-dispvm-gui-api (mean:7.833): 94.00 🟢 ( previous job: 98.62, improvement: 95.31%)
  • debian-13-xfce_dom0-dispvm-preload-2-api (mean:4.105): 49.26
  • debian-13-xfce_dom0-dispvm-preload-4-api (mean:3.781): 45.37
  • debian-13-xfce_dom0-dispvm-preload-2-gui-api (mean:4.169): 50.02
  • debian-13-xfce_dom0-dispvm-preload-4-gui-api (mean:3.893): 46.72
  • debian-13-xfce_dom0-vm-api (mean:0.033): 0.40 🟢 ( previous job: 0.59, improvement: 67.01%)
  • debian-13-xfce_dom0-vm-gui-api (mean:0.043): 0.51 🟢 ( previous job: 0.58, improvement: 88.60%)
  • fedora-42-xfce_dom0-dispvm-gui-api (mean:8.345): 100.14 🟢 ( previous job: 107.00, improvement: 93.59%)
  • fedora-42-xfce_dom0-dispvm-preload-2-api (mean:4.55): 54.59
  • fedora-42-xfce_dom0-dispvm-preload-4-api (mean:4.062): 48.74
  • fedora-42-xfce_dom0-dispvm-preload-2-gui-api (mean:4.693): 56.32
  • fedora-42-xfce_dom0-dispvm-preload-4-gui-api (mean:4.113): 49.36
  • whonix-workstation-18_dom0-dispvm-api (mean:9.005): 108.06 🟢 ( previous job: 117.52, improvement: 91.95%)
  • whonix-workstation-18_dom0-dispvm-gui-api (mean:9.589): 115.07 🟢 ( previous job: 130.38, improvement: 88.26%)
  • whonix-workstation-18_dom0-dispvm-preload-2-api (mean:5.419): 65.03
  • whonix-workstation-18_dom0-dispvm-preload-4-api (mean:4.765): 57.18
  • whonix-workstation-18_dom0-dispvm-preload-2-gui-api (mean:5.567): 66.80
  • whonix-workstation-18_dom0-dispvm-preload-4-gui-api (mean:4.657): 55.88
  • whonix-workstation-18_dom0-vm-api (mean:0.038): 0.46 🟢 ( previous job: 0.60, improvement: 76.49%)
  • whonix-workstation-18_dom0-vm-gui-api (mean:0.041): 0.49 🔻 ( previous job: 0.45, degradation: 108.46%)

@ben-grande
Copy link
Contributor Author

Untested.

Now it is tested... and working.

@ben-grande
Copy link
Contributor Author

Why fail with me OpenQA... I don't know, the preloads are taking too much time to get ready for that test. I'd have to see the machine live or get the output of systemd-analyze blame|plot and journalctl with and without --user.

Increased the timeout from 40s to 60s.

@ben-grande
Copy link
Contributor Author

I'd have to see the machine live or get the output of

I saw openqa disp* logs, but didn't identify the problem.

@marmarek
Copy link
Member

marmarek commented Dec 8, 2025

systemd-analyze blame|plot

I'm not sure if feasible, but maybe you can extend test with collecting such info on failure? Or is relevant VM killed already at that point?

@ben-grande ben-grande force-pushed the preload-autostart-refill branch from 7a4e1ad to 4d5728e Compare December 9, 2025 09:19
@ben-grande
Copy link
Contributor Author

I'm not sure if feasible, but maybe you can extend test with collecting such info on failure? Or is relevant VM killed already at that point?

Tests passed when timeout increased. VM is not killed at that point. The performance tests run fine with the expected values, just the CI one that times out. The issue this PR attempted to solve was that a 2 preload switch was too much, but in some way, I ended up producing a similar test when cleaning up manually 2 preloaded disposables and then running the autostart script to preload 2.

Looking again at the logs, WaitForRunningSystem completes but WaitForSession doesn't complete.

@ben-grande ben-grande force-pushed the preload-autostart-refill branch 2 times, most recently from c21f900 to 20de5d8 Compare December 9, 2025 09:28
The autostart made a refresh, which cleaned all preloaded disposables,
served to:

- Invalidate previous incomplete remnants
- Get fresh ones after a change was made to a template in chain

Both of these issues were handled, "domain-load" of the disposable
template invalidates incomplete ones while "domain-shutdown" of a
template in chain triggers a refresh.

For: QubesOS/qubes-issues#1512
@ben-grande
Copy link
Contributor Author

Superseded by #760

@ben-grande ben-grande closed this Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants