qemu_rx failing kernel.workqueue.api inconsistently

[duynguyenxa]

Describe the bug

kernel.workqueue.api inconsistently on github CI for qemu_rx target
The test cases is failing randomly on qemu_rx, attempt to rerun the job sometime make the test pass.

Regression

• This is a regression.

Steps to reproduce

Not sure

Relevant log output

===================================================================
RunID: 57ff46f14078f4789931ca77c9c74cd2
PROJECT EXECUTION FAILED

INFO    - /__w/zephyr/zephyr/twister-out/qemu_rx_r5f562n8/zephyr/tests/kernel/workq/work/kernel.workqueue.api/handler.log
INFO    - 4 Iteration:
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue

INFO    - 997/997 qemu_rx/r5f562n8          kernel.workqueue.api                               FAILED Testsuite failed (qemu 1.063s <zephyr>)
INFO    - /__w/zephyr/zephyr/twister-out/qemu_rx_r5f562n8/zephyr/tests/kernel/workq/work/kernel.workqueue.api/handler.log
ERROR   - *** Booting Zephyr OS build v4.1.0-6770-g714797d088aa ***
Running TESTSUITE work
===================================================================
START - test_drain_empty
 PASS - test_drain_empty in 0.001 seconds
===================================================================
START - test_nop
 SKIP - test_nop in 0.001 seconds
===================================================================
START - test_null_queue
 PASS - test_null_queue in 0.001 seconds
===================================================================
START - test_smp_running_cancel
 SKIP - test_smp_running_cancel in 0.001 seconds
===================================================================
START - test_smp_simple_queue
 SKIP - test_smp_simple_queue in 0.001 seconds
===================================================================
START - test_unstarted
 PASS - test_unstarted in 0.001 seconds
===================================================================
TESTSUITE work succeeded
Running TESTSUITE work_1cpu
===================================================================
START - test_1cpu_basic_reschedule

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/workq/work/src/main.c:1189: work_1cpu_test_1cpu_basic_reschedule: (elapsed_ms <= max_ms is false)
long 120 > 110

 FAIL - test_1cpu_basic_reschedule in 0.139 seconds
===================================================================
START - test_1cpu_basic_schedule

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/workq/work/src/main.c:1035: work_1cpu_test_1cpu_basic_schedule: (elapsed_ms <= max_ms is false)
long 120 > 110

 FAIL - test_1cpu_basic_schedule in 0.150 seconds
===================================================================
START - test_1cpu_basic_schedule_running
 PASS - test_1cpu_basic_schedule_running in 0.240 seconds
===================================================================
START - test_1cpu_delayed_cancel
 PASS - test_1cpu_delayed_cancel in 0.001 seconds
===================================================================
START - test_1cpu_delayed_cancel_sync
 PASS - test_1cpu_delayed_cancel_sync in 0.001 seconds
===================================================================
START - test_1cpu_delayed_cancel_sync_wait
 PASS - test_1cpu_delayed_cancel_sync_wait in 0.050 seconds
===================================================================
START - test_1cpu_delayed_flush
 PASS - test_1cpu_delayed_flush in 0.030 seconds
===================================================================
START - test_1cpu_drain_wait
 PASS - test_1cpu_drain_wait in 0.240 seconds
===================================================================
START - test_1cpu_immed_reschedule
 PASS - test_1cpu_immed_reschedule in 0.470 seconds
===================================================================
START - test_1cpu_immed_schedule
 PASS - test_1cpu_immed_schedule in 0.030 seconds
===================================================================
START - test_1cpu_plugged_drain
 PASS - test_1cpu_plugged_drain in 0.120 seconds
===================================================================
START - test_1cpu_system_reschedule

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/workq/work/src/main.c:1456: work_1cpu_test_1cpu_system_reschedule: (elapsed_ms <= max_ms is false)
long 120 > 110

 FAIL - test_1cpu_system_reschedule in 0.151 seconds
===================================================================
START - test_1cpu_system_schedule

    Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/workq/work/src/main.c:1407: work_1cpu_test_1cpu_system_schedule: (elapsed_ms <= max_ms is false)
long 120 > 110

 FAIL - test_1cpu_system_schedule in 0.150 seconds
===================================================================
TESTSUITE work_1cpu failed.

------ TESTSUITE SUMMARY START ------

SUITE PASS - 100.00% [work]: pass = 3, fail = 0, skip = 3, total = 6 duration = 0.006 seconds
 - PASS - [work.test_drain_empty] duration = 0.001 seconds
 - SKIP - [work.test_nop] duration = 0.001 seconds
 - PASS - [work.test_null_queue] duration = 0.001 seconds
 - SKIP - [work.test_smp_running_cancel] duration = 0.001 seconds
 - SKIP - [work.test_smp_simple_queue] duration = 0.001 seconds
 - PASS - [work.test_unstarted] duration = 0.001 seconds

SUITE FAIL -  62.50% [work_1cpu]: pass = 15, fail = 9, skip = 0, total = 24 duration = 2.205 seconds
 - FAIL - [work_1cpu.test_1cpu_basic_reschedule] duration = 0.139 seconds
 - FAIL - [work_1cpu.test_1cpu_basic_schedule] duration = 0.150 seconds
 - PASS - [work_1cpu.test_1cpu_basic_schedule_running] duration = 0.240 seconds
 - PASS - [work_1cpu.test_1cpu_delayed_cancel] duration = 0.001 seconds
 - PASS - [work_1cpu.test_1cpu_delayed_cancel_sync] duration = 0.001 seconds
 - PASS - [work_1cpu.test_1cpu_delayed_cancel_sync_wait] duration = 0.050 seconds
 - PASS - [work_1cpu.test_1cpu_delayed_flush] duration = 0.030 seconds
 - PASS - [work_1cpu.test_1cpu_drain_wait] duration = 0.240 seconds
 - PASS - [work_1cpu.test_1cpu_immed_reschedule] duration = 0.470 seconds
 - PASS - [work_1cpu.test_1cpu_immed_schedule] duration = 0.030 seconds
 - PASS - [work_1cpu.test_1cpu_plugged_drain] duration = 0.120 seconds
 - PASS - [work_1cpu.test_1cpu_queue_no_yield] duration = 0.001 seconds
 - PASS - [work_1cpu.test_1cpu_queued_cancel] duration = 0.001 seconds
 - PASS - [work_1cpu.test_1cpu_queued_cancel_sync] duration = 0.001 seconds
 - PASS - [work_1cpu.test_1cpu_queued_flush] duration = 0.239 seconds
 - PASS - [work_1cpu.test_1cpu_reentrant_queue] duration = 0.030 seconds
 - FAIL - [work_1cpu.test_1cpu_running_cancel] duration = 0.041 seconds
 - FAIL - [work_1cpu.test_1cpu_running_cancel_sync] duration = 0.029 seconds
 - FAIL - [work_1cpu.test_1cpu_running_flush] duration = 0.001 seconds
 - FAIL - [work_1cpu.test_1cpu_simple_queue] duration = 0.030 seconds
 - FAIL - [work_1cpu.test_1cpu_sync_queue] duration = 0.030 seconds
 - PASS - [work_1cpu.test_1cpu_system_queue] duration = 0.030 seconds
 - FAIL - [work_1cpu.test_1cpu_system_reschedule] duration = 0.151 seconds
 - FAIL - [work_1cpu.test_1cpu_system_schedule] duration = 0.150 seconds

Impact

Intermittent – Occurs occasionally; hard to reproduce.

Environment

OS: Linux
Toolchain: Zephyr SDK

Additional Context

No response