Support for Espressif chipsets via ESP-IDF #2142

vikramdattu · 2025-05-23T14:52:58Z

*** This is a placeholder PR: Not to be merged ****

This PR is meant to showcase the work which will bring the support for ESP-IDF build support and hence the Espressif chipsets.

Structure:

Changes specific to ESP-IDF are put under esp_port directory.
Go to the example webrtc_classic and build it as ESP-IDF example.

Setup ESP-IDF

Setup ESP_IDF with release/v5.4 branch: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/linux-macos-setup.html

BUILD

cd esp_port/examples/webrtc_classic
idf.py set-target esp32s3 # Or esp32p4 etc
idf.py build

Flash the code on chip and run monitor

idf.py flash
idf.py monitor

The work is under progress, will keep rebasing as and when common functionality code keeps getting into main repository.

@sirknightj @unicornss

We have two options:

Upstream and merge all the changes to this repo: Not as-is of course and will have reviews from structure and code point of view
The esp_port maintained by Espressif with some suitable name under espressif/ organisation. It will use amazon-kinesis-video-sdk-c as a submodule or need developer to set path of cloned repo.

We can keep this PR for discussion ground continue discussion on other aspects of how best we can go about this. I prefer the option 1, simply because it remains maintained under single roof without needing to constant rebasing.

vikramdattu · 2025-06-03T05:08:04Z

@sirknightj @unicornss do you have comments on this?
If you look at my solution, I have added a esp_port directory containing extra needed components and examples specific to ESP-IDF build system. (There are a few, very few, changes in the common source files specific to ESP-IDF under macro checks)
I would like to know if you want to suggest adding a generic port layer which can be leveraged by other platforms to build solutions based on this SDK? For example, this might have port directory and different platform ports can stay in the directory etc.

I would like to speed up the porting process as per your comments, so that the repo can then be used in-time for ESP-IDF based open-source solutions, viz., esp-rainmaker and esp-matter etc.

aggarg · 2025-06-05T11:33:10Z

You may want to review the reference integration for the Realtek Ameba Pro2 Mini board at: https://github.com/awslabs/freertos-webrtc-reference-on-amebapro-for-amazon-kinesis-video-streams.

This reference integration utilizes our OS-agnostic libraries. You can adopt a similar approach to develop an integration for the ESP platform.

vikramdattu · 2025-06-05T12:43:48Z

You may want to review the reference integration for the Realtek Ameba Pro2 Mini board at: https://github.com/awslabs/freertos-webrtc-reference-on-amebapro-for-amazon-kinesis-video-streams.

This reference integration utilizes our OS-agnostic libraries. You can adopt a similar approach to develop an integration for the ESP platform.

@aggarg

This solution in my particular PR is the working one. Only requiring the discussion on the final structure of the code to make it acceptable.

The approach you have shared is completely different take from what I've achieved. The particular ameba implementation does not use this repo, and hence I cannot go yet another implementation approach using multiple components. My goal is to get this repo adopted on ESP platforms and make the existing sources/APIs generic enough as is to be able to be used for ESP chipsets.

- Avoids libwebsockets taking control over using raw fread/nvs APIs - The SDK, hence, do not need to rely on libwebsockets method

- Handle partial writes by sending data in multiple iterations - Use retries when message send fails - Track and handle message receive if in parts using `receiveMessage` var

- Retries increased from 3 to 10 - Consider ENOMEM or PBUF errors also for retries These changes help to retain sockets longer for embedded devices

- Use dynamically allocated WSS URL - Use dynamic allocations for singaling payload - Use required only allocations in LwsApiCalls instead large static arrays

- Added ifaddr altenate for ESP-IDF - Created abstracted API for IDF_CMAKE platform to query local IP - Do not treat SO_SNDBUF not available as fatal

- Use custom thread implementation for receiveLwsMessageWrapper - Custom thread implementation for connectionListener

…xamples

- esp_hosted + esp_wifi_remote enabled chipsets not having wifi/BT to use it via n/w adapter - network_coprocessor component can be use with network_adapter

…_video

- Cleanly added the support under esp_port directory - Added ESP specific components, submodules and an webrtc_classic example - Added `app_common` needed as a common functionality for webrtc example/s. - Added network_adapter example: Needed when we want to use the webrtc_classic on P4+C6 board

- Added esp_websocket_client over libwebsockets for ESP-IDF - Using this as default for ESP port as it provides better flexibility for ESP-IDF

- Refactored webrtc_message re-assembly logic to message utils - Any componenent can include message_utils.h to leverage the same - Use message_utils in webrtc_bridge and register for network_coprocessor messages

- This component is specifically designed to exchange messages between signaling and streaming planes of the WebRTC - For example, if we want to implement only the streaming part of the webrtc and get signaling messages relayed to it, another device can implement signaling and relay those messages to streaming plane

- It helps packing and unpacking of signaling messages - These serialized messages then can be transfered on any transport

- Added streaming_only and signaling_only examples - streaming_only can be flashed on P4 and signaling only on C6 - The message exchange between the two happens over webrtc_bridge

- esp_port/README.md: Added complete description and setup guide for ESP chipsets - esp_port/README.md: Describe two solutions: webrtc_classic and split mode - Added puml diagrams under esp_port/docs showcasing the workflow in both modes - Added a README under example/app_common/spiffs/certs/

- Patch1: Support LWIP in split_mode - Patch2: Work-around for STATUS define

sirknightj · 2025-06-19T02:06:46Z

esp_port/README.md

+### Install pkg-config
+
+```bash
+sudo apt-get install pkg-config


Would recommend adding the sudo apt-get update step just in case.

sirknightj · 2025-06-19T02:09:02Z

esp_port/README.md

+  - AWS_KVS_LOG_LEVEL
+
+- You may override the above settings in `main/app_main.c` file of the example also.
+


Suggest also adding the commands to use the idf.py to:

Connect the board to your computer (micro usb)

Build the project

Flash project onto the ESP board (and any buttons that need to be pressed or held down during the flash, any LED indicators that we can look for)

Perhaps also some links to troubleshooting documentation in case there is something blocking USB access in the computer (the board is not reachable)

sirknightj · 2025-06-19T02:10:20Z

esp_port/README.md

+
+  - ESP_WIFI_SSID
+  - ESP_WIFI_PASSWORD
+  - ESP_MAXIMUM_RETRY


I think the majority of the fields here are self explanatory, except ESP_MAXIMUM_RETRY.

Perhaps we can make a table with the options along with a description.

sirknightj · 2025-06-19T02:18:59Z

esp_port/docs/camera_device.puml

@@ -0,0 +1,92 @@
+@startuml


I saw this which you might want to look at -- to get these displayed in one of the readmes: https://github.com/Zingam/UML-in-Markdown

Alternatively, GitHub supports mermaid diagrams - https://github.blog/developer-skills/github/include-diagrams-markdown-files-mermaid/

sirknightj · 2025-06-19T02:21:12Z

esp_port/README.md

+  - AWS_SECRET_ACCESS_KEY
+  - AWS_DEFAULT_REGION
+  - AWS_KVS_CHANNEL
+  - AWS_KVS_LOG_LEVEL


I saw some sample frames in one of the folders, would it make sense to have an option to use the camera sensor vs. pre-generated frames?

sirknightj · 2025-06-19T02:24:03Z

esp_port/examples/app_common/src/app_media.c

+        frame.frameData = pSampleConfiguration->pVideoFrameBuffer;
+        frame.size = frameSize;
+
+        CHK_STATUS(app_media_read_frame_from_disk(frame.frameData, &frameSize, filePath));


I see the frames being read from here

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 2 times, most recently from d9ab49b to b3e9d05 Compare May 28, 2025 11:57

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 4 times, most recently from 4ff89b5 to b96ece0 Compare June 4, 2025 13:55

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 4 times, most recently from 6ad13b1 to 7f787a0 Compare June 12, 2025 05:45

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 2 times, most recently from 39f8a4a to 278e573 Compare June 17, 2025 14:32

vikramdattu added 15 commits June 18, 2025 10:59

Read and pass cacert buffer to lws_config instead of path

795b841

- Avoids libwebsockets taking control over using raw fread/nvs APIs - The SDK, hence, do not need to rely on libwebsockets method

Increase robustness of LwsApiCall implementation

d1d4107

- Handle partial writes by sending data in multiple iterations - Use retries when message send fails - Track and handle message receive if in parts using `receiveMessage` var

Increase socket connection resilience to memory errors

540b34c

- Retries increased from 3 to 10 - Consider ENOMEM or PBUF errors also for retries These changes help to retain sockets longer for embedded devices

Prefer dynamic allocations over large STATIC arrays

20a9f78

- Use dynamically allocated WSS URL - Use dynamic allocations for singaling payload - Use required only allocations in LwsApiCalls instead large static arrays

Network.c: Changes to support ESP-IDF

502e54f

- Added ifaddr altenate for ESP-IDF - Created abstracted API for IDF_CMAKE platform to query local IP - Do not treat SO_SNDBUF not available as fatal

ESP32 specific change for memory fix

6c40098

Timeout increase tests for ESP

1ec3595

Use custom thread implementation for ESP platforms to save memory

dfe03d3

- Use custom thread implementation for receiveLwsMessageWrapper - Custom thread implementation for connectionListener

Created esp_port directory to contain ESP-IDF ported components and e…

fbdba11

…xamples

Added esp_hosted, esp_wifi_remote and network_coprocessor components

2421773

- esp_hosted + esp_wifi_remote enabled chipsets not having wifi/BT to use it via n/w adapter - network_coprocessor component can be use with network_adapter

Added esp32_p4_Function_ev_board component

3bf69b3

Added media_stream component which would be used to record/play audio…

1f593c2

…_video

ESP port: Use esp_websocket_client as an alternate implementation

ea6cbd1

- Added esp_websocket_client over libwebsockets for ESP-IDF - Using this as default for ESP port as it provides better flexibility for ESP-IDF

webrtc message utils

d71ef19

- Refactored webrtc_message re-assembly logic to message utils - Any componenent can include message_utils.h to leverage the same - Use message_utils in webrtc_bridge and register for network_coprocessor messages

vikramdattu added 5 commits June 18, 2025 11:05

Added signaling_serializer component

bf6ee37

- It helps packing and unpacking of signaling messages - These serialized messages then can be transfered on any transport

Added WebRTC split functionality

e53b739

- Added streaming_only and signaling_only examples - streaming_only can be flashed on P4 and signaling only on C6 - The message exchange between the two happens over webrtc_bridge

Added IDF patches

ea74bf0

- Patch1: Support LWIP in split_mode - Patch2: Work-around for STATUS define

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch from 278e573 to ea74bf0 Compare June 18, 2025 15:26

sirknightj reviewed Jun 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support for Espressif chipsets via ESP-IDF #2142

Support for Espressif chipsets via ESP-IDF #2142

Uh oh!

vikramdattu commented May 23, 2025 •

edited

Loading

Uh oh!

vikramdattu commented Jun 3, 2025

Uh oh!

aggarg commented Jun 5, 2025 •

edited

Loading

Uh oh!

vikramdattu commented Jun 5, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

Uh oh!

		- AWS_KVS_LOG_LEVEL

		- You may override the above settings in `main/app_main.c` file of the example also.

Support for Espressif chipsets via ESP-IDF #2142

Are you sure you want to change the base?

Support for Espressif chipsets via ESP-IDF #2142

Uh oh!

Conversation

vikramdattu commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Structure:

Setup ESP-IDF

BUILD

Flash the code on chip and run monitor

Uh oh!

vikramdattu commented Jun 3, 2025

Uh oh!

aggarg commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vikramdattu commented Jun 5, 2025

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vikramdattu commented May 23, 2025 •

edited

Loading

aggarg commented Jun 5, 2025 •

edited

Loading