Supporting Vendor-Specific Display Controllers

[ioannis-karachalios]

Hi community,

I am a SW engineer from Renesas Electronics, and we are currently evaluating the possibility to support the display controller integrated in our Bluetooth Low Energy, high-end devices. For starters, we would be interested in demonstrating the display controller of the DA1469x SoC which is currently supported by Zephyr. We have done some research on this topic and our understanding is that there is no dedicated driver framework when it comes to interfacing with display drivers. Instead, a more generic approach is adopted using a 4-wire or 3-wire SPI interface. Thing is that our display controller is what differentiates us from other competitors in the market when it comes to developing low-power, wireless, MCU-based wearable applications. Following is some of the advantages for one to utilize our display controller instead of adopting the mentioned generic approach.

• Timing generator capable of supporting several different serial and parallel display interfaces for instance, MIPI DPI (VSYNC/HSYNC), MIPI DBI-C (SPI3/4), serial Sharp and serial/parallel JDI, to name a few.

• DMA accelerator with geometry awareness. One would argue that this accelerator could be replaced by a generic DMA channel, but this should not work for all the cases. For example, when the frame buffer is stored in an external medium, for instance PSRAM/FLASH, and the output pixel rate is higher than the input pixel rate then a mechanism, namely pixels pre-fetching, must be employed. Otherwise, the display controller will fail to execute a frame update.

• Pixel formatter used to perform on-the-fly and zero-latency color formats conversions while data are being fetched from the frame buffer. For instance, frame buffer's pixels data with RGB565 color format could be converted to RGB888 at the output.

• Layer overlay circuitry capable of performing geometry tasks for instance moving the frame buffers onto the X/Y Cartesian coordinate system, performing partial updates, color blending etc.

Having mentioned all the above, one can conclude that our display controller should offload the main processor (ARM Cortex-M33) and contribute to lower power consumption required for battery operated wearable products.

Our main purpose of this first communication is to kick off discussions on the possibilities to port our display controller to the Zephyr project. For instance, could we create an intermediate interface between Zephyr's generic display framework and our device-specific display accelerator, or could we initiate discussions on exploring a new accelerator-oriented display driver framework?

Thanks in advance.

Best regards,
Ioannis Karachalios

[DerekSnell]

Hi @ioannis-karachalios ,
NXP is working on a similar framework for Zephyr. Please see [RFC] 2D Accelerator API Proposal, and we look forward to your feedback there.

Best regards

[ioannis-karachalios]

Hi @DerekSnell,

We thoroughly studied the mentioned framework and we concluded that the proposed solution should not meet our controller's peculiarities simply because our display accelerator does not exhibit any rendering capabilities as opposed to vector accelerators.

Best regards

[danieldegrasse]

Hi @ioannis-karachalios- thank you for the initial feedback. I would appreciate you engaging on the RFC, if you are willing. NXP has similar 2D accelerators which are only capable of operations such as pixel format conversion or display rotation. The current state of that RFC is that several community members have expressed concern that a generic API will limit performance of some accelerators, so we may explore device-specific interfaces for each accelerator

[ioannis-karachalios]

Hi @danieldegrasse,
Our concern is that these common functionalities, e.g. pixel format conversion, can only be performed while data pixels are being fetched from a frame buffer and outputted through the selected display interface e.g. 4-wire SPI. It's not that our accelerator supports read-write-modify or memory-to-memory operations. Nevertheless, I will also engage on the RFC and provide a first feedback.

Best regards

[danieldegrasse]

Understood- we do have accelerators like this as well, that can only convert data while it in flight to the display. This is what the accel2d_write_display API is intended to cover- if the source buffer format did not match the display format, the color format conversion would be performed by the accelerator on the fly.

[ioannis-karachalios]

Indeed, this should accommodate our accelerator's capabilities. I was wandering whether the proposed RFC, and apart from dealing with rendering operations (on-the-fly or in-memory) will also take care of display interfaces. Will there be any interface layer where one can employ standardized or even proprietary display interface protocols? Typically this should include I/O definitions, bus definitions and driver settings definitions.

[ioannis-karachalios]

Hi @danieldegrasse could you please answer to my last question on the display support? It's not clear to me how display_write will actually work. Thanks in advance.