add display test harness

[hakehuang]

introduce a display test harness.

1. using a camera to capture the display content.
2. calculate fingerprint for given display at customer time
3. in CI, compare the fingerprint with the runtime capture video frames.

please check the readme for more details

[josuah]

Interesting approach, very comprehensive and could help testing Zephyr-based UVC cameras themselves as well maybe some day.

Out of curiosity, is it resilient to the DUT being rotated and moved slightly by accident? While looking quickly, I did not find anything to try to crop the view to the display rectangle only.

A few comments left to help move this forward.

[Copilot]

Pull Request Overview

This PR introduces a display test harness that uses a camera to capture display output, computes fingerprints at customer runtime, and compares them against reference frames in CI.

• Added board configuration files and overlay settings for display tests
• Extended Twister schema and harness to support a new display_capture test mode
• Introduced a standalone display-twister-harness Python package with camera capture and fingerprint plugins

Reviewed Changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/drivers/display/display_check/boards/*.conf/.overlay	Added per-board display test memory and overlay configs
tests/drivers/display/display_check/CMakeLists.txt	Defined Zephyr test project sources
scripts/schemas/twister/testsuite-schema.yaml	Extended schema with optional display_capture_config
scripts/pylib/twister/twisterlib/testinstance.py	Whitelisted display_capture harness
scripts/pylib/twister/twisterlib/harness.py	Implemented Display_capture harness subclass
scripts/requirements-run-test.txt	Added OpenCV and NumPy dependencies
scripts/pylib/display-twister-harness/{test_display.py,…}	New pytest fixtures, tests, and camera capture code
scripts/pylib/display-twister-harness/camera_shield/**	Core camera controller, plugin base, and plugin API
scripts/pylib/display-twister-harness/camera_shield/README.rst	Usage and setup documentation

Comments suppressed due to low confidence (7)

scripts/pylib/display-twister-harness/README.rst:12

• Typo in comment: 'reslution' should be 'resolution'.

  rex_x: 1280   # x reslution

scripts/pylib/display-twister-harness/README.rst:30

• Typo in comment: 'direcory' should be 'directory'.

  #      directory: "./fingerprints" # fingerprints direcory to compare with not used in generate mode

scripts/pylib/display-twister-harness/README.rst:118

• Typo in comment above: 'mutliply fingure prints' should be 'multiply fingerprint files'.

   but will help to check other defects.

scripts/pylib/display-twister-harness/camera_shield/config.yaml:1

• The key 'rex_x' appears to be a typo; it should be 'res_x' to match the rest of the code.

case_config: {device_id: 0, fps: 30, res_y: 720, rex_x: 1280, run_time: 20}

scripts/pylib/twister/twisterlib/harness.py:632

• [nitpick] Class names typically follow PascalCase without underscores; consider renaming to DisplayCapture for consistency.

class Display_capture(Pytest):

scripts/pylib/display-twister-harness/test_display.py:19

• [nitpick] The docstring for get_prompt is very generic; consider describing the structure of the returned dict and possible keys.

def get_prompt(config):

tests/drivers/display/display_check/boards/mimxrt1170_evk_mimxrt1176_cm7.conf:7

• The comment uses 'panelwidth' twice but the example calculation uses height; clarify which dimensions are used in the formula.

# Sample will allocate buffer equal to: (panelwidth / 8) * (panelwidth / 4) * pixel depth. For a

[hakehuang]

Out of curiosity, is it resilient to the DUT being rotated and moved slightly by accident? While looking quickly, I did not find anything to try to crop the view to the display rectangle only.

No by current algorithm, as this could be a problem the display is wrongly goes to upside down. but it is extendable.

[josuah]

It looks like this project can be used for testing a lot of things that are difficult to test automatically on Zephyr:

• Motors
• LEDs
• Displays
• Cameras (by replacing the UVC camera of the setup by a Zephyr camera, or by sending the camera output to a display)
• GUIs plotted on display

Right away or in the future, it can be interesting to rename scripts/pylib/display-twister-harness into something more generic. Maybe scripts/pylib/webcam-twister-harness.

For what it currently does, scripts/pylib/display-twister-harness seems fit, however.

[sonarqubecloud]

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
20.7% Duplication on New Code

See analysis details on SonarQube Cloud

[hakehuang]

@nashif please take a look, and need your comments on whether it is ok to move test harness code in tree.

[nashif]

i can't test and verify it myself, but I like the idea in general.

[hakehuang]

@josuah , can help advice how to proceed with this PR, As this function is design for zephyr, so the scripts is better in tree. Thanks

[josuah]

The last question I had was "how to let people know who maintains it?" and it is already answered in the MAINTAINERS.yml file:

zephyr/MAINTAINERS.yml

Line 3444 in e22ca6b

- hakehuang

Thank you for all the work on testing!

[hakehuang]

@anasnashif can you help to approve this? @josuah has verify the process, am I right?

[josuah]

@josuah has verify the process, am I right?

I did not get a chance to test it on my desk, but did my best to spot issues in what the code does (AOI with twister!), the documentation, the configuration, and found no issue.