Support for Espressif chipsets via ESP-IDF #2142

vikramdattu · 2025-05-23T14:52:58Z

*** This is a placeholder PR: Not to be merged ****

This PR is meant to showcase the work which will bring the support for ESP-IDF build support and hence the Espressif chipsets.

Structure:

Changes specific to ESP-IDF are put under esp_port directory.
Go to the example webrtc_classic and build it as ESP-IDF example.

Setup ESP-IDF

Setup ESP_IDF with release/v5.4 branch: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/linux-macos-setup.html

BUILD

cd esp_port/examples/webrtc_classic
idf.py set-target esp32s3 # Or esp32p4 etc
idf.py build

Flash the code on chip and run monitor

idf.py flash
idf.py monitor

The work is under progress, will keep rebasing as and when common functionality code keeps getting into main repository.

@sirknightj @unicornss

We have two options:

Upstream and merge all the changes to this repo: Not as-is of course and will have reviews from structure and code point of view
The esp_port maintained by Espressif with some suitable name under espressif/ organisation. It will use amazon-kinesis-video-sdk-c as a submodule or need developer to set path of cloned repo.

We can keep this PR for discussion ground continue discussion on other aspects of how best we can go about this. I prefer the option 1, simply because it remains maintained under single roof without needing to constant rebasing.

vikramdattu · 2025-06-03T05:08:04Z

@sirknightj @unicornss do you have comments on this?
If you look at my solution, I have added a esp_port directory containing extra needed components and examples specific to ESP-IDF build system. (There are a few, very few, changes in the common source files specific to ESP-IDF under macro checks)
I would like to know if you want to suggest adding a generic port layer which can be leveraged by other platforms to build solutions based on this SDK? For example, this might have port directory and different platform ports can stay in the directory etc.

I would like to speed up the porting process as per your comments, so that the repo can then be used in-time for ESP-IDF based open-source solutions, viz., esp-rainmaker and esp-matter etc.

aggarg · 2025-06-05T11:33:10Z

You may want to review the reference integration for the Realtek Ameba Pro2 Mini board at: https://github.com/awslabs/freertos-webrtc-reference-on-amebapro-for-amazon-kinesis-video-streams.

This reference integration utilizes our OS-agnostic libraries. You can adopt a similar approach to develop an integration for the ESP platform.

vikramdattu · 2025-06-05T12:43:48Z

You may want to review the reference integration for the Realtek Ameba Pro2 Mini board at: https://github.com/awslabs/freertos-webrtc-reference-on-amebapro-for-amazon-kinesis-video-streams.

This reference integration utilizes our OS-agnostic libraries. You can adopt a similar approach to develop an integration for the ESP platform.

@aggarg

This solution in my particular PR is the working one. Only requiring the discussion on the final structure of the code to make it acceptable.

The approach you have shared is completely different take from what I've achieved. The particular ameba implementation does not use this repo, and hence I cannot go yet another implementation approach using multiple components. My goal is to get this repo adopted on ESP platforms and make the existing sources/APIs generic enough as is to be able to be used for ESP chipsets.

sirknightj · 2025-06-19T02:06:46Z

esp_port/README.md

+### Install pkg-config
+
+```bash
+sudo apt-get install pkg-config


Would recommend adding the sudo apt-get update step just in case.

sirknightj · 2025-06-19T02:09:02Z

esp_port/README.md

+  - AWS_KVS_LOG_LEVEL
+
+- You may override the above settings in `main/app_main.c` file of the example also.
+


Suggest also adding the commands to use the idf.py to:

Connect the board to your computer (micro usb)

Build the project

Flash project onto the ESP board (and any buttons that need to be pressed or held down during the flash, any LED indicators that we can look for)

Perhaps also some links to troubleshooting documentation in case there is something blocking USB access in the computer (the board is not reachable)

sirknightj · 2025-06-19T02:10:20Z

esp_port/README.md

+
+  - ESP_WIFI_SSID
+  - ESP_WIFI_PASSWORD
+  - ESP_MAXIMUM_RETRY


I think the majority of the fields here are self explanatory, except ESP_MAXIMUM_RETRY.

Perhaps we can make a table with the options along with a description.

sirknightj · 2025-06-19T02:18:59Z

esp_port/docs/camera_device.puml

@@ -0,0 +1,92 @@
+@startuml


I saw this which you might want to look at -- to get these displayed in one of the readmes: https://github.com/Zingam/UML-in-Markdown

Alternatively, GitHub supports mermaid diagrams - https://github.blog/developer-skills/github/include-diagrams-markdown-files-mermaid/

sirknightj · 2025-06-19T02:21:12Z

esp_port/README.md

+  - AWS_SECRET_ACCESS_KEY
+  - AWS_DEFAULT_REGION
+  - AWS_KVS_CHANNEL
+  - AWS_KVS_LOG_LEVEL


I saw some sample frames in one of the folders, would it make sense to have an option to use the camera sensor vs. pre-generated frames?

sirknightj · 2025-06-19T02:24:03Z

esp_port/examples/app_common/src/app_media.c

+        frame.frameData = pSampleConfiguration->pVideoFrameBuffer;
+        frame.size = frameSize;
+
+        CHK_STATUS(app_media_read_frame_from_disk(frame.frameData, &frameSize, filePath));


I see the frames being read from here

- Handle partial writes by sending data in multiple iterations - Use retries when message send fails - Track and handle message receive if in parts using `receiveMessage` var

- Retries increased from 3 to 10 - Consider ENOMEM or PBUF errors also for retries These changes help to retain sockets longer for embedded devices

- Use dynamically allocated WSS URL - Use dynamic allocations for singaling payload - Use required only allocations in LwsApiCalls instead large static arrays

- Added ifaddr altenate for ESP-IDF - Created abstracted API for IDF_CMAKE platform to query local IP - Do not treat SO_SNDBUF not available as fatal

- Use custom thread implementation for receiveLwsMessageWrapper - Custom thread implementation for connectionListener

…xamples

- esp_hosted + esp_wifi_remote enabled chipsets not having wifi/BT to use it via n/w adapter - network_coprocessor component can be use with network_adapter

…_video

- Cleanly added the support under esp_port directory - Added ESP specific components, submodules and an webrtc_classic example - Added `app_common` needed as a common functionality for webrtc example/s. - Added network_adapter example: Needed when we want to use the webrtc_classic on P4+C6 board

- Added esp_websocket_client over libwebsockets for ESP-IDF - Using this as default for ESP port as it provides better flexibility for ESP-IDF

- It helps packing and unpacking of signaling messages - These serialized messages then can be transfered on any transport

- Added streaming_only and signaling_only examples - streaming_only can be flashed on P4 and signaling only on C6 - The message exchange between the two happens over webrtc_bridge

- esp_port/README.md: Added complete description and setup guide for ESP chipsets - esp_port/README.md: Describe two solutions: webrtc_classic and split mode - Added puml diagrams under esp_port/docs showcasing the workflow in both modes - Added a README under example/app_common/spiffs/certs/

- Patch1: Support LWIP in split_mode. Allows port filtering in lwip_opts.h - Patch2: Work-around for STATUS define. KVS too uses `STATUS` for return values. Guard the IDF STATUS under `#ifndef STATUS`

- e.g., config structure can be inited with most common config and fine tuned thereon ```c WebRtcAppConfig webrtcConfig = WEBRTC_APP_CONFIG_DEFAULT(); ``` - Bugfix: make a copy of `awsCredentialOptions` so that it can be re-used out of scope

…webrtcAppRun - This allows us keep webrtcAppInit lighter and move webrtcAppRun to different task

… examples as well

- Application should not need to take care of fitting the message received from webrtc_bridge into SDK's structure format - The message will simply be deserialized and passed to SDK using `webrtcAppSignalingMessageReceived` API - Messages from SDK are received via callback and we can send to webrtc_bridge from application - hosted: Fixed build for IDF release/v5.5

…leio - This is helpful where our cert files are actually in NVS - Also updated the README for esp_webrtc_utils component - app_webrtc.c: Use fileIO wrapper functions instead of direct file ops

…ient - We simply store the server time globally when we do http requests - Use that gloabal time for aws signature and Date header while websocket connect

- Can be useful for provisioning workflows OR any other use cases where host needs BT

- Aggressive timeout makes board reboot prematurely. - Wait for max 10 seconds before SDIO transport is ready

- New API `webrtcAppCreateAndSendOffer`, can be called to initiate the offer - Also, spin the mediaSender thread in handlerAnswer

- Works in Viewer mode of KVS WebRTC - The camera registers to a room and initiates the sdp offer - The client can join the room(this sends the sdp answer to camera) - peer connection is established and feed can be viewed

- TURN servers are not `MUST` for the peer connection - The device can still continue with the STUN server - In cases where, server is configured to not allow IceServers, we can continue

- Application should not need to take care of fitting the message received from webrtc_bridge into SDK's structure format - The message will simply be deserialized and passed to SDK using `webrtcAppSignalingMessageReceived` API - Messages from SDK are received via callback and we can send to webrtc_bridge from application

- New API `webrtcAppCreateAndSendOffer`, can be called to initiate the offer - Also, spin the mediaSender thread in handlerAnswer

- Attempt reconnections from webrtc task when disconnections happen - Reconnections are handled with new task in LwsApiCalls.c - Workflow was broken with LwsApiCallsESP.c: We now handle it from webrtc_task itself - Do not reboot the device when using wifi-set CLI command

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 2 times, most recently from d9ab49b to b3e9d05 Compare May 28, 2025 11:57

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 4 times, most recently from 4ff89b5 to b96ece0 Compare June 4, 2025 13:55

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 4 times, most recently from 6ad13b1 to 7f787a0 Compare June 12, 2025 05:45

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch 3 times, most recently from 278e573 to ea74bf0 Compare June 18, 2025 15:26

sirknightj reviewed Jun 19, 2025

View reviewed changes

vikramdattu added 13 commits July 10, 2025 11:00

Increase robustness of LwsApiCall implementation

eae7eff

- Handle partial writes by sending data in multiple iterations - Use retries when message send fails - Track and handle message receive if in parts using `receiveMessage` var

Increase socket connection resilience to memory errors

d8e956e

- Retries increased from 3 to 10 - Consider ENOMEM or PBUF errors also for retries These changes help to retain sockets longer for embedded devices

Prefer dynamic allocations over large STATIC arrays

37353d3

- Use dynamically allocated WSS URL - Use dynamic allocations for singaling payload - Use required only allocations in LwsApiCalls instead large static arrays

Network.c: Changes to support ESP-IDF

e2d2ece

- Added ifaddr altenate for ESP-IDF - Created abstracted API for IDF_CMAKE platform to query local IP - Do not treat SO_SNDBUF not available as fatal

ESP32 specific change for memory fix

d34d6d0

Timeout increase tests for ESP

7c13ff5

Use custom thread implementation for ESP platforms to save memory

5a136c5

- Use custom thread implementation for receiveLwsMessageWrapper - Custom thread implementation for connectionListener

Created esp_port directory to contain ESP-IDF ported components and e…

a15a15d

…xamples

Added esp_hosted, esp_wifi_remote and network_coprocessor components

681863c

- esp_hosted + esp_wifi_remote enabled chipsets not having wifi/BT to use it via n/w adapter - network_coprocessor component can be use with network_adapter

Added esp32_p4_Function_ev_board component

c2f43ef

Added media_stream component which would be used to record/play audio…

2ba034d

…_video

ESP port: Use esp_websocket_client as an alternate implementation

53d0645

- Added esp_websocket_client over libwebsockets for ESP-IDF - Using this as default for ESP port as it provides better flexibility for ESP-IDF

vikramdattu added 29 commits July 30, 2025 19:40

Added signaling_serializer component

e5e87a7

- It helps packing and unpacking of signaling messages - These serialized messages then can be transfered on any transport

Added WebRTC split functionality

e6a027e

- Added streaming_only and signaling_only examples - streaming_only can be flashed on P4 and signaling only on C6 - The message exchange between the two happens over webrtc_bridge

Added IDF patches for lwip split functioning

f5b09dd

- Patch1: Support LWIP in split_mode. Allows port filtering in lwip_opts.h - Patch2: Work-around for STATUS define. KVS too uses `STATUS` for return values. Guard the IDF STATUS under `#ifndef STATUS`

Updated lwip port filtering patch

b23b99a

Use config option for setting max concurrent streaming sessions

582ec03

Restruct: defined a default config

01ac74b

- e.g., config structure can be inited with most common config and fine tuned thereon ```c WebRtcAppConfig webrtcConfig = WEBRTC_APP_CONFIG_DEFAULT(); ``` - Bugfix: make a copy of `awsCredentialOptions` so that it can be re-used out of scope

Leave Wi-Fi configuration for ESP32-C6 to handle

72ad3b3

Refactor: Move logging into separate WebRtcLogging.c

f5c1a22

Refactor(app_webrtc): Move credential provider from webrtcAppInit to …

8f91abb

…webrtcAppRun - This allows us keep webrtcAppInit lighter and move webrtcAppRun to different task

Restruct: Major changes for media_stream, should go in app_webrtc and…

626457d

… examples as well

webrtc_classic: remove unused embedded files

520f319

Create a new task for webrtc handling and unblock the caller

e0ad0f4

LwsApiCalls.c: Handling of time_skew

f37f17c

Move app_webrtc.c/.h to kvs_webrtc component

b0c1b9a

Updated usersctp to the threadsafe version

7b03f10

esp_work_queue: New API to init workq with config

b4eef12

Improve flash_wrapper functionality to handle NVS ops disguised as fi…

a1e78ae

…leio - This is helpful where our cert files are actually in NVS - Also updated the README for esp_webrtc_utils component - app_webrtc.c: Use fileIO wrapper functions instead of direct file ops

TimeSkew (2/n): remove SNTP time sync dependency for esp_websocket_cl…

0134e0d

…ient - We simply store the server time globally when we do http requests - Use that gloabal time for aws signature and Date header while websocket connect

esp_hosted: Fix BT functionality

6ec339d

- Can be useful for provisioning workflows OR any other use cases where host needs BT

esp_hosted: Modify init timer timeout 1s --> 10s

4080789

- Aggressive timeout makes board reboot prematurely. - Wait for max 10 seconds before SDIO transport is ready

app_webrtc: Added and API to trigger offer

e95b702

- New API `webrtcAppCreateAndSendOffer`, can be called to initiate the offer - Also, spin the mediaSender thread in handlerAnswer

Example show-casing apprtc based esp camera

151f464

- Works in Viewer mode of KVS WebRTC - The camera registers to a room and initiates the sdp offer - The client can join the room(this sends the sdp answer to camera) - peer connection is established and feed can be viewed

Continue with the peer connection even if getIceServerConfig failed

87e5cdf

- TURN servers are not `MUST` for the peer connection - The device can still continue with the STUN server - In cases where, server is configured to not allow IceServers, we can continue

app_webrtc: Added and API to trigger offer

a5c28ec

- New API `webrtcAppCreateAndSendOffer`, can be called to initiate the offer - Also, spin the mediaSender thread in handlerAnswer

Restructure app_webrtc.c to move out signaling to separate file

e4a847b

vikramdattu force-pushed the esp_amazon_kvs_sdk_c branch from ea74bf0 to f65eeca Compare August 1, 2025 15:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support for Espressif chipsets via ESP-IDF #2142

Support for Espressif chipsets via ESP-IDF #2142

Uh oh!

vikramdattu commented May 23, 2025 •

edited

Loading

Uh oh!

vikramdattu commented Jun 3, 2025

Uh oh!

aggarg commented Jun 5, 2025 •

edited

Loading

Uh oh!

vikramdattu commented Jun 5, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

sirknightj Jun 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		- AWS_KVS_LOG_LEVEL

		- You may override the above settings in `main/app_main.c` file of the example also.

Support for Espressif chipsets via ESP-IDF #2142

Are you sure you want to change the base?

Support for Espressif chipsets via ESP-IDF #2142

Uh oh!

Conversation

vikramdattu commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Structure:

Setup ESP-IDF

BUILD

Flash the code on chip and run monitor

Uh oh!

vikramdattu commented Jun 3, 2025

Uh oh!

aggarg commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vikramdattu commented Jun 5, 2025

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

sirknightj Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vikramdattu commented May 23, 2025 •

edited

Loading

aggarg commented Jun 5, 2025 •

edited

Loading