tests: update for Thunderbird version/config in Whonix 17 #103

marmarek · 2025-03-10T21:03:50Z

No description provided.

qubesos-bot · 2025-03-11T06:42:32Z

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025031404-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025021804-4.3&flavor=update

system_tests_whonix
- whonixcheck: unnamed test (unknown)
- whonixcheck: unnamed test (unknown)
system_tests_qrexec
- TC_00_Qrexec_fedora-41-xfce: test_065_qrexec_exit_code_vm (failure)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^... AssertionError: b'125\n' != b'0\n'
system_tests_dispvm
- TC_20_DispVM_whonix-workstation-17: test_100_open_in_dispvm (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_audio
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_222_audio_rec_unmuted_pulseaudio (failure)
  AssertionError: too short audio, expected 10s, got 9.47804988662131...
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_260_audio_mic_enabled_switch_audiovm (failure)
  AssertionError: too short audio, expected 10s, got 9.46775510204081...
system_tests_whonix@hw7
- whonixcheck: unnamed test (unknown)
- whonixcheck: unnamed test (unknown)
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/GrZKv-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'qvm-create-windows-qube -i Win...
system_tests_basic_vm_qrexec_gui_btrfs
- TC_20_NonAudio_whonix-workstation-17-pool: test_010_run_xterm (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError
- TC_20_NonAudio_whonix-workstation-17-pool: test_011_run_gnome_terminal (skipped + cleanup)
system_tests_qwt_win10_seamless@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...
system_tests_qwt_win10@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'clipboard-paste-notif...

Failed tests

25 failures

system_tests_whonix
- whonixcheck: fail (unknown)
  Whonixcheck for anon-whonix failed...
- whonixcheck: unnamed test (unknown)
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)
system_tests_qrexec
- TC_00_Qrexec_fedora-41-xfce: test_065_qrexec_exit_code_vm (failure)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^... AssertionError: b'125\n' != b'0\n'
system_tests_dispvm
- TC_20_DispVM_whonix-workstation-17: test_100_open_in_dispvm (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_audio
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_222_audio_rec_unmuted_pulseaudio (failure)
  AssertionError: too short audio, expected 10s, got 9.47804988662131...
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_251_audio_playback_audiovm_pipewire_late_start (failure)
  AssertionError: too short audio, expected 10s, got 7.37337868480725...
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_260_audio_mic_enabled_switch_audiovm (failure)
  AssertionError: too short audio, expected 10s, got 9.46775510204081...
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for anon-whonix failed...
- whonixcheck: unnamed test (unknown)
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/GrZKv-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'qvm-create-windows-qube -i Win...
system_tests_basic_vm_qrexec_gui_btrfs
- TC_20_NonAudio_whonix-workstation-17-pool: test_010_run_xterm (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError
- TC_20_NonAudio_whonix-workstation-17-pool: test_011_run_gnome_terminal (skipped + cleanup)
system_tests_suspend@hw1
- suspend: unnamed test (unknown)
- suspend: Failed (test died)
  # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
system_tests_qwt_win10_seamless@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...
system_tests_qwt_win10@hw13
- windows_clipboard_and_filecopy: unnamed test (unknown)
- windows_clipboard_and_filecopy: Failed (test died)
  # Test died: no candidate needle with tag(s) 'clipboard-paste-notif...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/129058#dependencies

10 fixed

system_tests_whonix
- whonixcheck: unnamed test (unknown)
system_tests_suspend
- suspend: unnamed test (unknown)
- suspend: Failed (test died)
  # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
system_tests_qrexec
- TC_00_Qrexec_whonix-workstation-17: test_070_qrexec_vm_simultaneous_write (failure)
  exited with non-zero code 125; stderr: b"dd: error writing 'standa...
system_tests_kde_gui_interactive
- clipboard_and_web: unnamed test (unknown)
- clipboard_and_web: Failed (test died)
  # Test died: no candidate needle with tag(s) 'clipboard-paste-notif...
system_tests_audio
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_223_audio_play_hvm (failure)
  AssertionError: too short audio, expected 10s, got 7.33528344671201...
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_226_audio_playback_pipewire (failure)
  AssertionError: too short audio, expected 10s, got 9.19027210884353...
system_tests_whonix@hw7
- whonixcheck: unnamed test (unknown)
system_tests_basic_vm_qrexec_gui_btrfs
- TC_03_QvmRevertTemplateChanges: test_000_revert_linux (error)
  subprocess.CalledProcessError: Command '['sha1sum', '/var/lib/qubes...

Unstable tests

Performance Tests

Performance degradation:

32 performance degradations

debian-12-xfce_exec-data-simplex: 72.89 :small_red_triangle_up: ( previous job: 48.93, degradation: 148.97%)
debian-12-xfce_exec-data-duplex: 63.68 :small_red_triangle_up: ( previous job: 50.76, degradation: 125.45%)
debian-12-xfce_exec-data-duplex-root: 80.12 :small_red_triangle_up: ( previous job: 64.91, degradation: 123.44%)
debian-12-xfce_socket-data-duplex: 165.57 :small_red_triangle_up: ( previous job: 81.49, degradation: 203.18%)
fedora-41-xfce_exec-data-simplex: 61.99 :small_red_triangle_up: ( previous job: 49.65, degradation: 124.85%)
fedora-41-xfce_exec-data-duplex: 65.16 :small_red_triangle_up: ( previous job: 49.08, degradation: 132.76%)
fedora-41-xfce_exec-data-duplex-root: 101.31 :small_red_triangle_up: ( previous job: 81.65, degradation: 124.08%)
fedora-41-xfce_socket-data-duplex: 138.12 :small_red_triangle_up: ( previous job: 78.62, degradation: 175.68%)
whonix-gateway-17_exec-data-simplex: 75.79 :small_red_triangle_up: ( previous job: 48.76, degradation: 155.42%)
whonix-gateway-17_exec-data-duplex: 74.65 :small_red_triangle_up: ( previous job: 48.55, degradation: 153.75%)
whonix-gateway-17_exec-data-duplex-root: 92.61 :small_red_triangle_up: ( previous job: 70.13, degradation: 132.07%)
whonix-gateway-17_socket-data-duplex: 158.12 :small_red_triangle_up: ( previous job: 82.74, degradation: 191.11%)
whonix-workstation-17_exec-data-simplex: 75.85 :small_red_triangle_up: ( previous job: 47.01, degradation: 161.33%)
whonix-workstation-17_exec-data-duplex: 71.56 :small_red_triangle_up: ( previous job: 49.48, degradation: 144.64%)
whonix-workstation-17_exec-data-duplex-root: 93.03 :small_red_triangle_up: ( previous job: 79.93, degradation: 116.39%)
whonix-workstation-17_socket-data-duplex: 151.07 :small_red_triangle_up: ( previous job: 81.71, degradation: 184.88%)
dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 398092.00 :small_red_triangle_up: ( previous job: 486352.00, degradation: 81.85%)
dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 144951.00 :small_red_triangle_up: ( previous job: 276742.00, degradation: 52.38%)
dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 46095.00 :small_red_triangle_up: ( previous job: 423495.00, degradation: 10.88%)
dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 74275.00 :small_red_triangle_up: ( previous job: 185030.00, degradation: 40.14%)
dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 11197.00 :small_red_triangle_up: ( previous job: 100699.00, degradation: 11.12%)
dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 752.00 :small_red_triangle_up: ( previous job: 3277.00, degradation: 22.95%)
dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 424181.00 :small_red_triangle_up: ( previous job: 475329.00, degradation: 89.24%)
dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 5802.00 :small_red_triangle_up: ( previous job: 8767.00, degradation: 66.18%)
fedora-41-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 305262.00 :small_red_triangle_up: ( previous job: 343795.00, degradation: 88.79%)
fedora-41-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 903.00 :small_red_triangle_up: ( previous job: 1126.00, degradation: 80.20%)
fedora-41-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 68830.00 :small_red_triangle_up: ( previous job: 87999.00, degradation: 78.22%)
fedora-41-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3360.00 :small_red_triangle_up: ( previous job: 3885.00, degradation: 86.49%)
fedora-41-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 274928.00 :small_red_triangle_up: ( previous job: 392725.00, degradation: 70.01%)
fedora-41-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 50641.00 :small_red_triangle_up: ( previous job: 71108.00, degradation: 71.22%)
fedora-41-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 3157.00 :small_red_triangle_up: ( previous job: 3959.00, degradation: 79.74%)
fedora-41-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 963.00 :small_red_triangle_up: ( previous job: 2693.00, degradation: 35.76%)

Remaining performance tests:

40 tests

debian-12-xfce_exec: 6.45 🟢 ( previous job: 7.15, improvement: 90.19%)
debian-12-xfce_exec-root: 28.87 :small_red_triangle_up: ( previous job: 27.97, degradation: 103.24%)
debian-12-xfce_socket: 8.62 :small_red_triangle_up: ( previous job: 8.33, degradation: 103.53%)
debian-12-xfce_socket-root: 8.63 :small_red_triangle_up: ( previous job: 8.20, degradation: 105.16%)
fedora-41-xfce_exec: 9.18 :small_red_triangle_up: ( previous job: 9.13, degradation: 100.46%)
fedora-41-xfce_exec-root: 60.13 🟢 ( previous job: 61.17, improvement: 98.30%)
fedora-41-xfce_socket: 8.22 🟢 ( previous job: 8.66, improvement: 94.91%)
fedora-41-xfce_socket-root: 8.58 🟢 ( previous job: 8.61, improvement: 99.64%)
whonix-gateway-17_exec: 7.92 :small_red_triangle_up: ( previous job: 7.87, degradation: 100.66%)
whonix-gateway-17_exec-root: 41.59 :small_red_triangle_up: ( previous job: 38.36, degradation: 108.40%)
whonix-gateway-17_socket: 8.21 :small_red_triangle_up: ( previous job: 7.54, degradation: 108.81%)
whonix-gateway-17_socket-root: 7.14 🟢 ( previous job: 8.27, improvement: 86.35%)
whonix-workstation-17_exec: 7.42 🟢 ( previous job: 8.23, improvement: 90.11%)
whonix-workstation-17_exec-root: 53.80 :small_red_triangle_up: ( previous job: 52.56, degradation: 102.36%)
whonix-workstation-17_socket: 8.66 :small_red_triangle_up: ( previous job: 8.21, degradation: 105.50%)
whonix-workstation-17_socket-root: 8.30 :small_red_triangle_up: ( previous job: 8.20, degradation: 101.25%)
dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9388.00 :small_red_triangle_up: ( previous job: 10163.00, degradation: 92.37%)
dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 286.00 :green_circle: ( previous job: 282.00, improvement: 101.42%)
dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 115744.00 :green_circle: ( previous job: 95209.00, improvement: 121.57%)
dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 439102.00 :green_circle: ( previous job: 433474.00, improvement: 101.30%)
dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 161813.00 :small_red_triangle_up: ( previous job: 164133.00, degradation: 98.59%)
dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 111470.00 :green_circle: ( previous job: 99808.00, improvement: 111.68%)
dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7684.00 :green_circle: ( previous job: 7053.00, improvement: 108.95%)
dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 3871.00 :green_circle: ( previous job: 3868.00, improvement: 100.08%)
fedora-41-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 368050.00 :small_red_triangle_up: ( previous job: 396586.00, degradation: 92.80%)
fedora-41-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 240996.00 :green_circle: ( previous job: 99783.00, improvement: 241.52%)
fedora-41-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 118751.00 :green_circle: ( previous job: 44770.00, improvement: 265.25%)
fedora-41-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 79061.00 :small_red_triangle_up: ( previous job: 86742.00, degradation: 91.15%)
fedora-41-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 4227.00 :green_circle: ( previous job: 3785.00, improvement: 111.68%)
fedora-41-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 8180.00 :small_red_triangle_up: ( previous job: 8623.00, degradation: 94.86%)
fedora-41-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 362829.00 :small_red_triangle_up: ( previous job: 401907.00, degradation: 90.28%)
fedora-41-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 249008.00 :green_circle: ( previous job: 116848.00, improvement: 213.10%)
fedora-41-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 338032.00 :small_red_triangle_up: ( previous job: 357875.00, degradation: 94.46%)
fedora-41-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 95260.00 :green_circle: ( previous job: 41375.00, improvement: 230.24%)
fedora-41-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8779.00 :green_circle: ( previous job: 8744.00, improvement: 100.40%)
fedora-41-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1805.00 :green_circle: ( previous job: 1613.00, improvement: 111.90%)
fedora-41-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 251155.00 :green_circle: ( previous job: 139933.00, improvement: 179.48%)
fedora-41-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 337488.00 :green_circle: ( previous job: 294875.00, improvement: 114.45%)
fedora-41-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 123456.00 :green_circle: ( previous job: 78093.00, improvement: 158.09%)
fedora-41-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 7568.00 :small_red_triangle_up: ( previous job: 8408.00, degradation: 90.01%)

Only press 'Get Messages' button to trigger the password prompt. In Whonix, at this point the mail folders list is not available yet, so cannot click 'Inbox' before entering the password.

marmarek added the openqa-pending label Mar 10, 2025

This was referenced Mar 11, 2025

Add boot mode support QubesOS/qubes-manager#407

Merged

Add Arch packaging QubesOS/qubes-core-admin-client#332

Open

tests: update for Thunderbird version/config in Whonix 17

046667f

Only press 'Get Messages' button to trigger the password prompt. In Whonix, at this point the mail folders list is not available yet, so cannot click 'Inbox' before entering the password.

marmarek force-pushed the whonix17 branch from 7214d8a to 046667f Compare March 11, 2025 14:02

marmarek merged commit 046667f into QubesOS:main Mar 14, 2025
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: update for Thunderbird version/config in Whonix 17 #103

tests: update for Thunderbird version/config in Whonix 17 #103

marmarek commented Mar 10, 2025

qubesos-bot commented Mar 11, 2025 •

edited

Loading

tests: update for Thunderbird version/config in Whonix 17 #103

tests: update for Thunderbird version/config in Whonix 17 #103

Conversation

marmarek commented Mar 10, 2025

qubesos-bot commented Mar 11, 2025 • edited Loading

OpenQA test summary

New failures, excluding unstable

Failed tests

Fixed failures

Unstable tests

Performance Tests

Performance degradation:

Remaining performance tests:

qubesos-bot commented Mar 11, 2025 •

edited

Loading