Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
eae7eff
Increase robustness of LwsApiCall implementation
vikramdattu May 14, 2025
d8e956e
Increase socket connection resilience to memory errors
vikramdattu Dec 10, 2024
37353d3
Prefer dynamic allocations over large STATIC arrays
vikramdattu May 13, 2025
e2d2ece
Network.c: Changes to support ESP-IDF
vikramdattu Dec 10, 2024
d34d6d0
ESP32 specific change for memory fix
vikramdattu Dec 10, 2024
7c13ff5
Timeout increase tests for ESP
vikramdattu Dec 10, 2024
5a136c5
Use custom thread implementation for ESP platforms to save memory
vikramdattu Dec 10, 2024
a15a15d
Created esp_port directory to contain ESP-IDF ported components and e…
vikramdattu May 23, 2025
681863c
Added esp_hosted, esp_wifi_remote and network_coprocessor components
vikramdattu May 23, 2025
c2f43ef
Added esp32_p4_Function_ev_board component
vikramdattu Jun 10, 2025
2ba034d
Added media_stream component which would be used to record/play audio…
vikramdattu May 23, 2025
c2f9b9d
ESP port: Added needed components and examples
vikramdattu Apr 28, 2025
53d0645
ESP port: Use esp_websocket_client as an alternate implementation
vikramdattu May 14, 2025
4b4788c
webrtc message utils
vikramdattu May 14, 2025
3030218
WebRTC bridge component
vikramdattu May 23, 2025
e5e87a7
Added signaling_serializer component
vikramdattu May 26, 2025
e6a027e
Added WebRTC split functionality
vikramdattu May 6, 2025
9db19d5
Added documentation for ESP port
vikramdattu May 30, 2025
f5b09dd
Added IDF patches for lwip split functioning
vikramdattu Jun 3, 2025
b23b99a
Updated lwip port filtering patch
vikramdattu Jul 10, 2025
582ec03
Use config option for setting max concurrent streaming sessions
vikramdattu Jun 25, 2025
01ac74b
Restruct: defined a default config
vikramdattu Jul 7, 2025
72ad3b3
Leave Wi-Fi configuration for ESP32-C6 to handle
vikramdattu Jul 7, 2025
f5c1a22
Refactor: Move logging into separate WebRtcLogging.c
vikramdattu Jun 25, 2025
8f91abb
Refactor(app_webrtc): Move credential provider from webrtcAppInit to …
vikramdattu Jun 25, 2025
626457d
Restruct: Major changes for media_stream, should go in app_webrtc and…
vikramdattu Jun 27, 2025
520f319
webrtc_classic: remove unused embedded files
vikramdattu Jun 25, 2025
e0ad0f4
Create a new task for webrtc handling and unblock the caller
vikramdattu Jun 25, 2025
f37f17c
LwsApiCalls.c: Handling of time_skew
vikramdattu Jun 30, 2025
b0c1b9a
Move app_webrtc.c/.h to kvs_webrtc component
vikramdattu Jul 8, 2025
2a548b7
webrtc_split: move SignalingMessage logic to the app_webrtc.c
vikramdattu Jul 7, 2025
7b03f10
Updated usersctp to the threadsafe version
vikramdattu Jul 10, 2025
b4eef12
esp_work_queue: New API to init workq with config
vikramdattu Jul 10, 2025
a1e78ae
Improve flash_wrapper functionality to handle NVS ops disguised as fi…
vikramdattu Jul 10, 2025
0134e0d
TimeSkew (2/n): remove SNTP time sync dependency for esp_websocket_cl…
vikramdattu Jul 11, 2025
6ec339d
esp_hosted: Fix BT functionality
vikramdattu Jul 11, 2025
4080789
esp_hosted: Modify init timer timeout 1s --> 10s
vikramdattu Jul 14, 2025
e95b702
app_webrtc: Added and API to trigger offer
vikramdattu Jul 15, 2025
151f464
Example show-casing apprtc based esp camera
vikramdattu Jul 15, 2025
87e5cdf
Continue with the peer connection even if getIceServerConfig failed
vikramdattu Jul 18, 2025
2ebb6da
webrtc_split: move SignalingMessage logic to the app_webrtc.c
vikramdattu Jul 7, 2025
a5c28ec
app_webrtc: Added and API to trigger offer
vikramdattu Jul 15, 2025
e4a847b
Restructure app_webrtc.c to move out signaling to separate file
vikramdattu Jul 18, 2025
f65eeca
Improve connection robustness
vikramdattu Aug 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,7 @@ tags
CMakeLists.txt.user
*.vscode
*.swp

sdkconfig
sdkconfig.old
dependencies.lock
9 changes: 9 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,12 @@
[submodule "open-source/amazon-kinesis-video-streams-producer-c"]
path = open-source/amazon-kinesis-video-streams-producer-c
url = https://github.com/awslabs/amazon-kinesis-video-streams-producer-c.git
[submodule "esp_port/components/esp_usrsctp/usrsctp"]
path = esp_port/components/esp_usrsctp/usrsctp
url = https://github.com/vikramdattu/usrsctp.git
[submodule "esp_port/components/libsrtp2/libsrtp"]
path = esp_port/components/libsrtp2/libsrtp
url = https://github.com/cisco/libsrtp.git
[submodule "esp_port/components/libwebsockets/libwebsockets"]
path = esp_port/components/libwebsockets/libwebsockets
url = https://github.com/warmcat/libwebsockets.git
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ option(INSTRUMENTED_ALLOCATORS "Enable memory instrumentation" OFF)
option(ENABLE_AWS_SDK_IN_TESTS "Enable support for compiling AWS SDKs for tests" ON)
option(ENABLE_STATS_CALCULATION_CONTROL "Enable support for runtime control of ice agent stat calculations." OFF)
option(BUILD_OLD_MBEDTLS_VERSION "Use MbedTLS version 2.28.8." OFF)
option(PREFER_DYNAMIC_ALLOCS "Prefer dynamic allocations for payloads and URLs" OFF)

# Developer Flags
option(BUILD_TEST "Build the testing tree." OFF)
Expand Down Expand Up @@ -147,6 +148,10 @@ if (ENABLE_STATS_CALCULATION_CONTROL)
add_definitions(-DENABLE_STATS_CALCULATION_CONTROL)
endif()

if (PREFER_DYNAMIC_ALLOCS)
add_definitions(-DPREFER_DYNAMIC_ALLOCS=1)
endif()

if(USE_OPENSSL)
add_definitions(-DKVS_USE_OPENSSL)
elseif(USE_MBEDTLS)
Expand Down
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ Please refer to the release notes in [Releases](https://github.com/awslabs/amazo
- Sub 200k library size
- OpenSSL, libsrtp, libjsmn, libusrsctp and libwebsockets dependencies.

**For Espressif ESP32/ESP-IDF users:**
If you are building for ESP32 or using ESP-IDF, please see the [ESP-IDF/ESP32 Port README](./esp_port/README.md) for hardware-specific setup, configuration, and usage instructions.

## Build
### Download
To download run the following command:
Expand All @@ -76,7 +79,7 @@ GStreamer is installed on your system.

On Ubuntu and Raspberry Pi OS you can get the libraries by running
```shell
sudo apt-get install cmake m4 pkg-config libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools
sudo apt-get install cmake m4 pkg-config libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools
```

By default we download all the libraries from GitHub and build them locally, so should require nothing to be installed ahead of time. If you do wish to link to existing libraries you can use the following flags to customize your build.
Expand Down Expand Up @@ -184,7 +187,7 @@ To build the library and the provided samples run make in the build directory yo

### Building with dependencies off

In addition to the dependencies already installed, install the dependencies using the appropriate package manager.
In addition to the dependencies already installed, install the dependencies using the appropriate package manager.

On Ubuntu:
`sudo apt-get install libsrtp2-dev libusrsctp-dev libwebsockets-dev`
Expand Down Expand Up @@ -244,7 +247,7 @@ export AWS_KVS_LOG_LEVEL=<LOG_LEVEL>

For example, the following command switches on `DEBUG` level logs while runnning the samples.
```shell
export AWS_KVS_LOG_LEVEL=2
export AWS_KVS_LOG_LEVEL=2
```

Note: The default log level is `LOG_LEVEL_WARN`.
Expand All @@ -262,7 +265,7 @@ Note: This log level is extremely VERBOSE and could flood the files if using fil

<details>
<summary>Time-to-first-frame breakdown metrics</summary>

There is a flag in the sample application which (pSampleConfiguration->enableSendingMetricsToViewerViaDc) can be set to TRUE to send metrics from the master to the [JS viewer](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html). This helps get a detailed breakdown of time-to-first-frame and all the processes and API calls on master and the viewer both. This is intended to be used with the KVS WebRTC C SDK running as the master and the JS SDK as the viewer. The master sends peer, ice-agent, signaling and data-channel metrics to the viewer which are plotted ~ 20 seconds after the viewer is started. Since the timeline plot is intended to understand the time-to-first-frame, the sample web page needs to be refreshed and the master needs to be restarted if a new / updated plot is needed. While using the SDK in this mode, it is expected that all datachannel messages are JSON messages. This feature is only meant to be used for a single viewer at a time.
</details>

Expand Down Expand Up @@ -291,7 +294,7 @@ This application sends sample H264/Opus frames (path: `/samples/h264SampleFrames
./samples/kvsWebrtcClientMaster <channelName> <storage-option> <audio-codec> <video-codec>
```

To use the **Storage for WebRTC** feature, run the same command as above but with an additional command line arg to enable the feature.
To use the **Storage for WebRTC** feature, run the same command as above but with an additional command line arg to enable the feature.

```shell
./samples/kvsWebrtcClientMaster <channelName> 1 <audio-codec> <video-codec>
Expand Down Expand Up @@ -381,7 +384,7 @@ If using the WebRTC SDK Test Page, set the following values using the same AWS c
* Secret access key
* Signaling channel name
* Client ID (optional)

Then choose Start Viewer to start live video streaming of the sample H264/Opus frames.

## Memory optimization switches
Expand Down Expand Up @@ -477,7 +480,7 @@ Set the environment variables for IoT Core credentials:

```shell
export AWS_IOT_CORE_CREDENTIAL_ENDPOINT=xxxxx.credentials.iot.xxxxx.amazonaws.com
export AWS_IOT_CORE_PRIVATE_KEY=xxxxxxxx-private.pem.key
export AWS_IOT_CORE_PRIVATE_KEY=xxxxxxxx-private.pem.key
export AWS_IOT_CORE_ROLE_ALIAS=xxxxxx
export AWS_IOT_CORE_THING_NAME=xxxxxx
export AWS_IOT_CORE_CERT=xxxxx-certificate.pem.crt
Expand Down Expand Up @@ -575,7 +578,7 @@ More information about what environment variables you can configure can be found

### Filtering network interfaces

This is useful to reduce candidate gathering time when it is known for certain network interfaces to not work well. A sample callback is available in `Common.c`. The `iceSetInterfaceFilterFunc` in `KvsRtcConfiguration` must be set to the required callback. In the sample, it can be done this way in `initializePeerConnection()`:
This is useful to reduce candidate gathering time when it is known for certain network interfaces to not work well. A sample callback is available in `Common.c`. The `iceSetInterfaceFilterFunc` in `KvsRtcConfiguration` must be set to the required callback. In the sample, it can be done this way in `initializePeerConnection()`:
`configuration.kvsRtcConfiguration.iceSetInterfaceFilterFunc = sampleFilterNetworkInterfaces`

### Building on MacOS M1
Expand All @@ -602,7 +605,7 @@ The default thread stack size in the KVS WebRTC SDK is determined by the system'

### Setting ICE related timeouts

There are some default timeout values set for different steps in ICE in the [KvsRtcConfiguration](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/structKvsRtcConfiguration.html). These are configurable in the application. While the defaults are generous, there could be applications that might need more flexibility to improve chances of connection establishment because of poor network.
There are some default timeout values set for different steps in ICE in the [KvsRtcConfiguration](https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-c/structKvsRtcConfiguration.html). These are configurable in the application. While the defaults are generous, there could be applications that might need more flexibility to improve chances of connection establishment because of poor network.

You can find the default setting in the logs:
```
Expand All @@ -615,15 +618,15 @@ You can find the default setting in the logs:
Let us look into when each of these could be changed:
1. `iceCandidateNominationTimeout`: Say the connection with host/srflx could not be established and TURN seems to be the only resort. Let us assume it takes about 15 seconds to gather the first local relay candidate, the application could set the timeout to a value more than 15 seconds to ensure candidate pairs with the local relay candidate are tried for success. If the value is set to less than 15 seconds in this case, the SDK would lose out on trying a potential candidate pair leading to connection establishment failure
2. `iceLocalCandidateGatheringTimeout`: Say the host candidates would not work and srflx/relay candidates need to be tried. Due to poor network, it is anticipated the candidates are gathered slowly and the application does not want to spend more than 20 seconds on this step. The goal is to try all possible candidate pairs. Increasing the timeout helps in giving some more time to gather more potential candidates to try for connection. Also note, this parameter increase would not make a difference in the situation unless `iceCandidateNominationTimeout` > `iceLocalCandidateGatheringTimeout` since nomination step should also be given time to work with the new candidates
3. `iceConnectionCheckTimeout`: It is useful to increase this timeout in unstable/slow network where the packet exchange takes time and hence the binding request/response. Essentially, increasing it will allow atleast one candidate pair to be tried for nomination by the other peer.
3. `iceConnectionCheckTimeout`: It is useful to increase this timeout in unstable/slow network where the packet exchange takes time and hence the binding request/response. Essentially, increasing it will allow atleast one candidate pair to be tried for nomination by the other peer.
4. `iceConnectionCheckPollingInterval`: This value is set to a default of 50 ms per [spec](https://datatracker.ietf.org/doc/html/rfc8445#section-14.2). Changing this would change the frequency of connectivity checks and essentially, the ICE state machine transitions. Decreasing the value could help in faster connection establishment in a reliable high performant network setting with good system resources. Increasing the value could help in reducing the network load, however, the connection establishment could slow down. Unless there is a strong reasoning, it is **NOT** recommended to deviate from spec/default.

### Enable ICE agent stats

The SDK calculates 4 different stats:
1. ICE server stats - stats for ICE servers the SDK is using
2. [Local candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-local-candidate) - stats for the selected local candidate
3. [Remote candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-remote-candidate) - stats for the selected remote candidate
3. [Remote candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-remote-candidate) - stats for the selected remote candidate
4. [Candidate pair stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcstatstype-candidate-pair) - stats for the selected candidate pair

For more information on these stats, refer to [AWS Docs](https://docs.aws.amazon.com/kinesisvideostreams-webrtc-dg/latest/devguide/kvswebrtc-reference.html)
Expand Down
7 changes: 7 additions & 0 deletions esp_port/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
sdkconfig
sdkconfig.old
dependencies.lock
managed_components
build/
*.key
*.pem
Loading