GH Actions E2E tests runner instability

[artus9033]

Before submitting a new issue

• I tested using the latest version of the library, as the bug might be already fixed.
• I tested using a supported version of react native.
• I checked for possible duplicate issues, with possible answers.

Bug summary

The CI E2E tests runner is 'unstable' in the sense that it randomly fails to start the emulator. Usually one / two runs in the E2E matrix would fail, but succeed in a subsequent re-run, such as this job: https://github.com/callstackincubator/react-native-legal/actions/runs/15728794209/job/44324791688 which succeeded here: https://github.com/callstackincubator/react-native-legal/actions/runs/15728794209

The critical logs are as follows:

> Task :app:installRelease
[EmulatorConsole]: Failed to start Emulator console for 5554
Installing APK 'app-release.apk, app-release.dm' on 'emulator-5554 - 15' for :app:release
Installed on 1 device.

BUILD SUCCESSFUL in 5m
97 actionable tasks: 97 executed
info Connecting to the development server...
8081
info Starting the app on "emulator-5554"...
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.reactnativelegalbareexample/.MainActivity }
CI detected and MAESTRO_CLI_NO_ANALYTICS environment variable set, analytics disabled.

Waiting for flows to complete...
Error: Exception in thread "pool-4-thread-1" java.io.IOException: Command failed (host:transport:emulator-5554): device offline
	at dadb.adbserver.AdbServer.send$dadb(AdbServer.kt:103)
	at dadb.adbserver.AdbServerDadb.open(AdbServer.kt:148)
	at dadb.forwarding.TcpForwarder.handleForwarding$lambda-1(TcpForwarder.kt:64)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
[Failed] [Android] Check React Native entry in OSS libraries list (2m 1s) (Unable to launch app com.reactnativelegalbareexample: am force-stop com.reactnativelegalbareexample)

At the same time, I'm almost sure that it always happens for Android only and just for Android API level 35 emulator (e.g. another such run is here: https://github.com/callstackincubator/react-native-legal/actions/runs/15704990994/attempts/1).

Maybe we should add some condition to retry a failed matrix job once if it fails with such output (Failed to start Emulator console) or consider changing the runner (not sure how this would impact the quota, though).

CC @wjaszczuk , @mateusz1913

Library version

N/A

Environment info

Github Actions

Steps to reproduce

1. Create any PR
2. Observe E2E status

Reproducible sample code

N/A

[artus9033]

Re-opening after https://github.com/callstackincubator/react-native-legal/actions/runs/15878510061/attempts/1 🤕

[wjaszczuk]

TL;DR;
I shortened the time between emulator start and test run.

Longer explanation:

@artus9033 @mateusz1913 I've found a bunch of issues in this topic, and I've linked them here:

mobile-dev-inc/Maestro#1525
mobile-dev-inc/Maestro#1289
mobile-dev-inc/Maestro#1570
mobile-dev-inc/Maestro#1573

https://github.com/DavidREntwistle added lots of context there. It appears that flaky tests on GitHub are caused by the emulator itself.

In the current scenario, the Android artefact build runs after the emulator. In the new approach, I split the build and tests into two phases, using RNEF and caching.

The E2E test on Android:

• downloads the APK from the previous step
• runs the emulator (with caching recipe this time)
• Installs the APK on a running emulator
• runs the e2e tests.

I also noticed that the API 34 emulator hangs up due to a Pixel app break. I watched a video from tests and they run, but there is an issue with the crash modal window at the first plan (everything is visible at the second plan, in the background), and it seems Maestro cannot find the underlying elements. I changed v34 to v33.

I also upgraded RN to 0.80.1 and used RNEF in the Bare Example app.