Skip to content

Support for Espressif chipsets via ESP-IDF #2142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 20 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
795b841
Read and pass cacert buffer to lws_config instead of path
vikramdattu Jun 5, 2025
d1d4107
Increase robustness of LwsApiCall implementation
vikramdattu May 14, 2025
540b34c
Increase socket connection resilience to memory errors
vikramdattu Dec 10, 2024
20a9f78
Prefer dynamic allocations over large STATIC arrays
vikramdattu May 13, 2025
502e54f
Network.c: Changes to support ESP-IDF
vikramdattu Dec 10, 2024
6c40098
ESP32 specific change for memory fix
vikramdattu Dec 10, 2024
1ec3595
Timeout increase tests for ESP
vikramdattu Dec 10, 2024
dfe03d3
Use custom thread implementation for ESP platforms to save memory
vikramdattu Dec 10, 2024
fbdba11
Created esp_port directory to contain ESP-IDF ported components and e…
vikramdattu May 23, 2025
2421773
Added esp_hosted, esp_wifi_remote and network_coprocessor components
vikramdattu May 23, 2025
3bf69b3
Added esp32_p4_Function_ev_board component
vikramdattu Jun 10, 2025
1f593c2
Added media_stream component which would be used to record/play audio…
vikramdattu May 23, 2025
295fd0f
ESP port: Added needed components and examples
vikramdattu Apr 28, 2025
ea6cbd1
ESP port: Use esp_websocket_client as an alternate implementation
vikramdattu May 14, 2025
d71ef19
webrtc message utils
vikramdattu May 14, 2025
27a4d24
WebRTC bridge component
vikramdattu May 23, 2025
bf6ee37
Added signaling_serializer component
vikramdattu May 26, 2025
e53b739
Added WebRTC split functionality
vikramdattu May 6, 2025
25acfc9
Added documentation for ESP port
vikramdattu May 30, 2025
ea74bf0
Added IDF patches
vikramdattu Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,7 @@ tags
CMakeLists.txt.user
*.vscode
*.swp

sdkconfig
sdkconfig.old
dependencies.lock
9 changes: 9 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,12 @@
[submodule "open-source/amazon-kinesis-video-streams-producer-c"]
path = open-source/amazon-kinesis-video-streams-producer-c
url = https://github.com/awslabs/amazon-kinesis-video-streams-producer-c.git
[submodule "esp_port/components/esp_usrsctp/usrsctp"]
path = esp_port/components/esp_usrsctp/usrsctp
url = https://github.com/vikramdattu/usrsctp.git
[submodule "esp_port/components/libsrtp2/libsrtp"]
path = esp_port/components/libsrtp2/libsrtp
url = https://github.com/cisco/libsrtp.git
[submodule "esp_port/components/libwebsockets/libwebsockets"]
path = esp_port/components/libwebsockets/libwebsockets
url = https://github.com/warmcat/libwebsockets.git
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ option(INSTRUMENTED_ALLOCATORS "Enable memory instrumentation" OFF)
option(ENABLE_AWS_SDK_IN_TESTS "Enable support for compiling AWS SDKs for tests" ON)
option(ENABLE_STATS_CALCULATION_CONTROL "Enable support for runtime control of ice agent stat calculations." OFF)
option(BUILD_OLD_MBEDTLS_VERSION "Use MbedTLS version 2.28.8." OFF)
option(PREFER_DYNAMIC_ALLOCS "Prefer dynamic allocations for payloads and URLs" OFF)

# Developer Flags
option(BUILD_TEST "Build the testing tree." OFF)
Expand Down Expand Up @@ -147,6 +148,10 @@ if (ENABLE_STATS_CALCULATION_CONTROL)
add_definitions(-DENABLE_STATS_CALCULATION_CONTROL)
endif()

if (PREFER_DYNAMIC_ALLOCS)
add_definitions(-DPREFER_DYNAMIC_ALLOCS=1)
endif()

if(USE_OPENSSL)
add_definitions(-DKVS_USE_OPENSSL)
elseif(USE_MBEDTLS)
Expand Down
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ Please refer to the release notes in [Releases](https://github.com/awslabs/amazo
- Sub 200k library size
- OpenSSL, libsrtp, libjsmn, libusrsctp and libwebsockets dependencies.

**For Espressif ESP32/ESP-IDF users:**
If you are building for ESP32 or using ESP-IDF, please see the [ESP-IDF/ESP32 Port README](./esp_port/README.md) for hardware-specific setup, configuration, and usage instructions.

## Build
### Download
To download run the following command:
Expand All @@ -76,7 +79,7 @@ GStreamer is installed on your system.

On Ubuntu and Raspberry Pi OS you can get the libraries by running
```shell
sudo apt-get install cmake m4 pkg-config libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools
sudo apt-get install cmake m4 pkg-config libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools
```

By default we download all the libraries from GitHub and build them locally, so should require nothing to be installed ahead of time. If you do wish to link to existing libraries you can use the following flags to customize your build.
Expand Down Expand Up @@ -184,7 +187,7 @@ To build the library and the provided samples run make in the build directory yo

### Building with dependencies off

In addition to the dependencies already installed, install the dependencies using the appropriate package manager.
In addition to the dependencies already installed, install the dependencies using the appropriate package manager.

On Ubuntu:
`sudo apt-get install libsrtp2-dev libusrsctp-dev libwebsockets-dev`
Expand Down Expand Up @@ -244,7 +247,7 @@ export AWS_KVS_LOG_LEVEL=<LOG_LEVEL>

For example, the following command switches on `DEBUG` level logs while runnning the samples.
```shell
export AWS_KVS_LOG_LEVEL=2
export AWS_KVS_LOG_LEVEL=2
```

Note: The default log level is `LOG_LEVEL_WARN`.
Expand All @@ -262,7 +265,7 @@ Note: This log level is extremely VERBOSE and could flood the files if using fil

<details>
<summary>Time-to-first-frame breakdown metrics</summary>

There is a flag in the sample application which (pSampleConfiguration->enableSendingMetricsToViewerViaDc) can be set to TRUE to send metrics from the master to the [JS viewer](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html). This helps get a detailed breakdown of time-to-first-frame and all the processes and API calls on master and the viewer both. This is intended to be used with the KVS WebRTC C SDK running as the master and the JS SDK as the viewer. The master sends peer, ice-agent, signaling and data-channel metrics to the viewer which are plotted ~ 20 seconds after the viewer is started. Since the timeline plot is intended to understand the time-to-first-frame, the sample web page needs to be refreshed and the master needs to be restarted if a new / updated plot is needed. While using the SDK in this mode, it is expected that all datachannel messages are JSON messages. This feature is only meant to be used for a single viewer at a time.
</details>

Expand Down Expand Up @@ -291,7 +294,7 @@ This application sends sample H264/Opus frames (path: `/samples/h264SampleFrames
./samples/kvsWebrtcClientMaster <channelName> <storage-option> <audio-codec> <video-codec>
```

To use the **Storage for WebRTC** feature, run the same command as above but with an additional command line arg to enable the feature.
To use the **Storage for WebRTC** feature, run the same command as above but with an additional command line arg to enable the feature.

```shell
./samples/kvsWebrtcClientMaster <channelName> 1 <audio-codec> <video-codec>
Expand Down Expand Up @@ -381,7 +384,7 @@ If using the WebRTC SDK Test Page, set the following values using the same AWS c
* Secret access key
* Signaling channel name
* Client ID (optional)

Then choose Start Viewer to start live video streaming of the sample H264/Opus frames.

## Memory optimization switches
Expand Down Expand Up @@ -477,7 +480,7 @@ Set the environment variables for IoT Core credentials:

```shell
export AWS_IOT_CORE_CREDENTIAL_ENDPOINT=xxxxx.credentials.iot.xxxxx.amazonaws.com
export AWS_IOT_CORE_PRIVATE_KEY=xxxxxxxx-private.pem.key
export AWS_IOT_CORE_PRIVATE_KEY=xxxxxxxx-private.pem.key
export AWS_IOT_CORE_ROLE_ALIAS=xxxxxx
export AWS_IOT_CORE_THING_NAME=xxxxxx
export AWS_IOT_CORE_CERT=xxxxx-certificate.pem.crt
Expand Down Expand Up @@ -575,7 +578,7 @@ More information about what environment variables you can configure can be found

### Filtering network interfaces

This is useful to reduce candidate gathering time when it is known for certain network interfaces to not work well. A sample callback is available in `Common.c`. The `iceSetInterfaceFilterFunc` in `KvsRtcConfiguration` must be set to the required callback. In the sample, it can be done this way in `initializePeerConnection()`:
This is useful to reduce candidate gathering time when it is known for certain network interfaces to not work well. A sample callback is available in `Common.c`. The `iceSetInterfaceFilterFunc` in `KvsRtcConfiguration` must be set to the required callback. In the sample, it can be done this way in `initializePeerConnection()`:
`configuration.kvsRtcConfiguration.iceSetInterfaceFilterFunc = sampleFilterNetworkInterfaces`

### Building on MacOS M1
Expand All @@ -602,7 +605,7 @@ The default thread stack size in the KVS WebRTC SDK is determined by the system'

### Setting ICE related timeouts

There are some default timeout values set for different steps in ICE in the [KvsRtcConfiguration](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/structKvsRtcConfiguration.html). These are configurable in the application. While the defaults are generous, there could be applications that might need more flexibility to improve chances of connection establishment because of poor network.
There are some default timeout values set for different steps in ICE in the [KvsRtcConfiguration](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/structKvsRtcConfiguration.html). These are configurable in the application. While the defaults are generous, there could be applications that might need more flexibility to improve chances of connection establishment because of poor network.

You can find the default setting in the logs:
```
Expand All @@ -615,15 +618,15 @@ You can find the default setting in the logs:
Let us look into when each of these could be changed:
1. `iceCandidateNominationTimeout`: Say the connection with host/srflx could not be established and TURN seems to be the only resort. Let us assume it takes about 15 seconds to gather the first local relay candidate, the application could set the timeout to a value more than 15 seconds to ensure candidate pairs with the local relay candidate are tried for success. If the value is set to less than 15 seconds in this case, the SDK would lose out on trying a potential candidate pair leading to connection establishment failure
2. `iceLocalCandidateGatheringTimeout`: Say the host candidates would not work and srflx/relay candidates need to be tried. Due to poor network, it is anticipated the candidates are gathered slowly and the application does not want to spend more than 20 seconds on this step. The goal is to try all possible candidate pairs. Increasing the timeout helps in giving some more time to gather more potential candidates to try for connection. Also note, this parameter increase would not make a difference in the situation unless `iceCandidateNominationTimeout` > `iceLocalCandidateGatheringTimeout` since nomination step should also be given time to work with the new candidates
3. `iceConnectionCheckTimeout`: It is useful to increase this timeout in unstable/slow network where the packet exchange takes time and hence the binding request/response. Essentially, increasing it will allow atleast one candidate pair to be tried for nomination by the other peer.
3. `iceConnectionCheckTimeout`: It is useful to increase this timeout in unstable/slow network where the packet exchange takes time and hence the binding request/response. Essentially, increasing it will allow atleast one candidate pair to be tried for nomination by the other peer.
4. `iceConnectionCheckPollingInterval`: This value is set to a default of 50 ms per [spec](https://datatracker.ietf.org/doc/html/rfc8445#section-14.2). Changing this would change the frequency of connectivity checks and essentially, the ICE state machine transitions. Decreasing the value could help in faster connection establishment in a reliable high performant network setting with good system resources. Increasing the value could help in reducing the network load, however, the connection establishment could slow down. Unless there is a strong reasoning, it is **NOT** recommended to deviate from spec/default.

### Enable ICE agent stats

The SDK calculates 4 different stats:
1. ICE server stats - stats for ICE servers the SDK is using
2. [Local candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-local-candidate) - stats for the selected local candidate
3. [Remote candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-remote-candidate) - stats for the selected remote candidate
3. [Remote candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-remote-candidate) - stats for the selected remote candidate
4. [Candidate pair stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-candidate-pair) - stats for the selected candidate pair

For more information on these stats, refer to [AWS Docs](https://docs.aws.amazon.com/kinesisvideostreams-webrtc-dg/latest/devguide/kvswebrtc-reference.html)
Expand Down
7 changes: 7 additions & 0 deletions esp_port/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
sdkconfig
sdkconfig.old
dependencies.lock
managed_components
build/
*.key
*.pem
Loading