Skip to content

Commit 7921ab1

Browse files
vikramdattuvikramdattu
committed
WIP: Documentation
 - esp_port/README.md: Describe two solutions: webrtc_classic and split mode - Added puml diagrams under esp_port/docs showcasing the workflow in both modes
1 parent 9e4c97e commit 7921ab1

File tree

5 files changed

+429
-3
lines changed

5 files changed

+429
-3
lines changed

‎esp_port/README.md

Lines changed: 38 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ esp_port/
1919
│ ├── libsrtp2/ # SRTP (Secure Real-time Transport Protocol) library
2020
│ ├── libwebsockets/ # WebSocket library
2121
│ ├── network_coprocessor/ # Network coprocessor support
22-
│ `── state_machine/ # State machine implementation
22+
│ └── state_machine/ # State machine implementation
2323
└── examples/ # Example applications
2424
```
2525

@@ -35,12 +35,47 @@ The `libsrtp2` component is a port of the Cisco libSRTP library, which provides
3535
- `libwebsockets`: WebSocket implementation for signaling
3636
- Additional components provide supporting functionality for the WebRTC stack
3737

38+
## Operational Modes
39+
40+
The WebRTC SDK supports two operational modes for different hardware configurations:
41+
42+
### Classic Mode
43+
In classic mode, signaling and streaming are performed on the same chip. This mode can be implemented as:
44+
- **Single Chip Solution**: All WebRTC functionality runs on a single ESP chip (e.g., ESP32-WROVER-KIT, ESP32-S3-EYE)
45+
- **Dual Chip Solution**:
46+
- Host processor (e.g., ESP32-P4) handles all signaling and streaming
47+
- Wi-Fi coprocessor (e.g., ESP32-C6) transparently forwards network traffic to the main processor
48+
49+
The `webrtc_classic` example demonstrates this functionality.
50+
51+
### Split Mode
52+
Split mode is designed for dual chip solutions, dividing responsibilities between processors:
53+
- **Signaling Processor**: Network coprocessor (ESP32-C6) handles signaling with KVS
54+
- **Streaming Processor**: Main processor (ESP32-P4) handles media streaming
55+
56+
This configuration offers significant advantages:
57+
- **Power Efficiency**: The main processor remains in deep sleep until streaming is required
58+
- **Instant Wake-up**: Network coprocessor maintains network connectivity, allowing immediate streaming when needed
59+
60+
To implement split mode:
61+
1. Flash the `signalling_only` application to the network coprocessor (ESP32-C6)
62+
2. Flash the `streaming_only` application to the main processor (ESP32-P4)
63+
64+
The network coprocessor connects to the signaling server and relays necessary messages to the main processor, which establishes direct RTP and SCTP sessions for streaming.
65+
3866
## Configuration
3967

40-
Each component may have its own configuration options that can be set through `menuconfig`. Refer to the individual component documentation for specific configuration options.
68+
Each component has its own configuration options that can be set through `menuconfig`. Refer to the individual component documentation for specific configuration options.
69+
70+
To use the WebRTC SDK in your project:
71+
72+
1. Add the required components to your project's `CMakeLists.txt`
73+
2. Configure parameters including AWS credentials, Wi-Fi settings, and WebRTC options through `menuconfig`
74+
3. For split mode, ensure proper communication between the signaling and streaming processors
4175

4276
## Notes
4377

4478
- The port uses mbedTLS for cryptographic operations
4579
- Some components have specific ESP-IDF version requirements
46-
- Check the examples directory for implementation references
80+
- For detailed information on component dependencies, refer to each component's documentation
81+
- Check the examples directory for implementation references and usage patterns

‎esp_port/docs/camera_device.puml

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
@startuml
2+
skinparam roundCorner 20
3+
skinparam BoxPadding 50
4+
5+
actor Client #Green
6+
7+
participant KVS [
8+
= Kinesis Video
9+
= Streams Service
10+
]
11+
12+
participant StunTurn [
13+
= STUN/TURN
14+
= Server
15+
]
16+
17+
box "<font size = 20><b><i>Camera Device</i>" #LightBlue
18+
19+
participant Streamer [
20+
= WebRTC Engine
21+
= (Network & Media)
22+
]
23+
24+
participant Camera [
25+
= Camera
26+
= Hardware
27+
]
28+
end box
29+
30+
== 1. Initialization ==
31+
Client -> KVS: Create signaling channel
32+
note over Streamer
33+
Device boots up and
34+
initializes WebRTC engine
35+
end note
36+
Streamer -> Camera: Initialize camera hardware
37+
Streamer -> Streamer: Initialize network stack
38+
Streamer -> Streamer: Initialize WebRTC components
39+
40+
== 2. KVS Signaling Connection ==
41+
Streamer -> KVS: Get IoT credentials
42+
KVS --> Streamer: Return access token
43+
Streamer -> KVS: Describe channel
44+
KVS --> Streamer: Channel info (Ingestion/Viewer)
45+
Streamer -> KVS: Get channel endpoint
46+
KVS --> Streamer: WSS and HTTPS endpoints
47+
Streamer -> KVS: Get ICE server configuration
48+
KVS --> Streamer: TURN server credentials
49+
Streamer -> KVS: Connect to signaling channel (WSS)
50+
KVS --> Streamer: Connection established
51+
note over Streamer: WSS ping-pong heartbeat starts
52+
53+
== 3. Call Initiation ==
54+
Client -> KVS: Initiate call (SDP Offer)
55+
KVS -> Streamer: Forward SDP Offer
56+
Streamer -> Streamer: Process SDP Offer
57+
Streamer -> Camera: Configure media settings
58+
Streamer -> KVS: Send SDP Answer
59+
KVS -> Client: Forward SDP Answer
60+
61+
== 4. ICE Connection Setup ==
62+
Streamer -> Streamer: Generate local ICE candidates
63+
Client -> Client: Generate local ICE candidates
64+
Streamer <--> StunTurn: Gather reflexive/relay candidates
65+
Client <--> StunTurn: Gather reflexive/relay candidates
66+
Streamer -> KVS: Send ICE candidates
67+
KVS -> Client: Forward ICE candidates
68+
Client -> KVS: Send ICE candidates
69+
KVS -> Streamer: Forward ICE candidates
70+
Streamer -> Streamer: ICE connectivity checks
71+
Client -> Client: ICE connectivity checks
72+
note over Streamer, Client: ICE connection established
73+
74+
== 5. Media Streaming ==
75+
activate Camera
76+
Streamer -> Camera: Start capture
77+
Camera --> Streamer: Video frames
78+
Streamer -> Streamer: Encode media
79+
Streamer -> Client: RTP/SRTP packets over DTLS
80+
note over Client: Media playback starts
81+
Client -> Streamer: RTCP feedback
82+
83+
== 6. Termination ==
84+
Client -> KVS: End call signal
85+
KVS -> Streamer: Forward end call
86+
Streamer -> Camera: Stop capture
87+
deactivate Camera
88+
Streamer -> Client: Close media connections
89+
Streamer -> KVS: Close signaling connection
90+
note over Streamer: Return to idle state
91+
92+
@enduml

‎esp_port/docs/connection_steps.md

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# WebRTC SDK Connection Process
2+
3+
This document outlines the sequential steps that occur during the WebRTC connection process with Kinesis Video Streams, based on detailed logs from a successful connection.
4+
5+
## 1. Initial Setup
6+
- Application initializes with channel name `demo-channel`
7+
- Loads certificates from specified paths (`certificate.pem`, `private.key`, `cacert.pem`)
8+
9+
## 2. Credential Setup
10+
- Creates IoT credential provider with endpoint: `c3mh3f5xs9l81c.credentials.iot.us-east-1.amazonaws.com`
11+
- Uses role alias `webrtc_iot_role_alias` and thing name `webrtc_iot_thing`
12+
- Fetches IoT credentials via HTTPS request to the credentials endpoint
13+
- Getting IoT credential with URL: `https://c3mh3f5xs9l81c.credentials.iot.us-east-1.amazonaws.com/role-aliases/webrtc_iot_role_alias/credentials`
14+
- Receives credentials with expiration time (valid for 1 hour)
15+
16+
## 3. Media Source Initialization
17+
- Discovers sample H.264 video frames from the samples directory
18+
- Initializes WebRTC and SRTP security components
19+
20+
## 4. Signaling Client Creation
21+
- Initializes signaling client in "New" state
22+
- Transitions to "Get Security Credentials" state using the IoT credentials fetched earlier
23+
- Control plane URL is generated: `https://kinesisvideo.us-east-1.amazonaws.com`
24+
25+
## 5. Channel Description
26+
- Transitions to "Describe Channel" state
27+
- Sends API request to: `https://kinesisvideo.us-east-1.amazonaws.com/describeSignalingChannel`
28+
- Request body:
29+
```json
30+
{
31+
"ChannelName": "demo-channel"
32+
}
33+
```
34+
- API Response:
35+
```json
36+
{
37+
"ChannelInfo": {
38+
"ChannelARN": "arn:aws:kinesisvideo:us-east-1:995678934610:channel/demo-channel/1712136149494",
39+
"ChannelName": "demo-channel",
40+
"ChannelStatus": "ACTIVE",
41+
"ChannelType": "SINGLE_MASTER",
42+
"CreationTime": 1.712136149494E9,
43+
"FullMeshConfiguration": null,
44+
"SingleMasterConfiguration": {
45+
"MessageTtlSeconds": 60
46+
},
47+
"Version": "3Ikjs9sOSk6Q4Alisl0d"
48+
}
49+
}
50+
```
51+
52+
## 6. Channel Endpoint Retrieval
53+
- Transitions to "Get Channel Endpoint" state
54+
- Sends API request to: `https://kinesisvideo.us-east-1.amazonaws.com/getSignalingChannelEndpoint`
55+
- Request body:
56+
```json
57+
{
58+
"ChannelARN": "arn:aws:kinesisvideo:us-east-1:995678934610:channel/demo-channel/1712136149494",
59+
"SingleMasterChannelEndpointConfiguration": {
60+
"Protocols": ["WSS", "HTTPS"],
61+
"Role": "MASTER"
62+
}
63+
}
64+
```
65+
- API Response:
66+
```json
67+
{
68+
"ResourceEndpointList": [
69+
{
70+
"Protocol": "HTTPS",
71+
"ResourceEndpoint": "https://r-c50d6457.kinesisvideo.us-east-1.amazonaws.com"
72+
},
73+
{
74+
"Protocol": "WSS",
75+
"ResourceEndpoint": "wss://m-b94e17e0.kinesisvideo.us-east-1.amazonaws.com"
76+
}
77+
]
78+
}
79+
```
80+
81+
## 7. ICE Server Configuration
82+
- Transitions to "Get ICE Server Configuration" state
83+
- Sends API request to: `https://r-c50d6457.kinesisvideo.us-east-1.amazonaws.com/v1/get-ice-server-config`
84+
- Request body:
85+
```json
86+
{
87+
"ChannelARN": "arn:aws:kinesisvideo:us-east-1:995678934610:channel/demo-channel/1712136149494",
88+
"ClientId": "ProducerMaster",
89+
"Service": "TURN"
90+
}
91+
```
92+
- API Response:
93+
```json
94+
{
95+
"IceServerList": [
96+
{
97+
"Password": "nhgUYYY0TKW5knkHAU92JI9whmOO058qbmxRvyIY2Rg=",
98+
"Ttl": 300,
99+
"Uris": [
100+
"turn:13-217-27-141.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=udp",
101+
"turns:13-217-27-141.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=udp",
102+
"turns:13-217-27-141.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=tcp"
103+
],
104+
"Username": "1743506463:djE6YXJuOmF3czpraW5lc2lzdmlkZW86dXMtZWFzdC0xOjk5NTY3ODkzNDYxMDpjaGFubmVsL2RlbW8tY2hhbm5lbC8xNzEyMTM2MTQ5NDk0"
105+
},
106+
{
107+
"Password": "O1ABsajTCFta+wj/9tTp8yF5aOFoiG6k/sWL3F+QT6w=",
108+
"Ttl": 300,
109+
"Uris": [
110+
"turn:54-159-217-207.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=udp",
111+
"turns:54-159-217-207.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=udp",
112+
"turns:54-159-217-207.t-99577840.kinesisvideo.us-east-1.amazonaws.com:443?transport=tcp"
113+
],
114+
"Username": "1743506463:djE6YXJuOmF3czpraW5lc2lzdmlkZW86dXMtZWFzdC0xOjk5NTY3ODkzNDYxMDpjaGFubmVsL2RlbW8tY2hhbm5lbC8xNzEyMTM2MTQ5NDk0"
115+
}
116+
]
117+
}
118+
```
119+
120+
## 8. Signaling Connection
121+
- Transitions to "Ready" state once all configurations are obtained
122+
- Begins connection to the WSS endpoint: `wss://m-b94e17e0.kinesisvideo.us-east-1.amazonaws.com`
123+
- Fully qualified URL example: `wss://m-07e4d66a.kinesisvideo.us-east-1.amazonaws.com?X-Amz-ChannelARN=arn:aws:kinesisvideo:us-east-1:995678934610:channel/demo-channel/1689337250341`
124+
- Successfully establishes WebSocket connection
125+
- Transitions to "Connected" state when the connection is established
126+
127+
## 9. WebRTC Peer Connection (After Connected State)
128+
- Once connected, the signaling client can exchange SDP offers/answers and ICE candidates
129+
- WebRTC peer connection is established using the acquired ICE server configurations
130+
- Media streaming begins after successful connection

‎esp_port/docs/webrtc_classic.puml

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
@startuml
2+
skinparam roundCorner 20
3+
skinparam BoxPadding 50
4+
5+
actor Client #Green
6+
7+
participant SignallingServer [
8+
= Signalling
9+
= Server
10+
]
11+
12+
participant StunTurn [
13+
= STUN/TURN
14+
= Server
15+
]
16+
17+
box "<font size = 20><b><i>Camera Device</i>" #LightBlue
18+
19+
participant Streamer [
20+
= Main
21+
= Processor
22+
]
23+
24+
participant Communicator [
25+
= Network
26+
= Coprocessor
27+
]
28+
end box
29+
30+
== 1. Call Initialisation ==
31+
32+
note over Client
33+
Client Initiates the call
34+
end note
35+
36+
Client -> SignallingServer: SDP Offer (Streaming Request)
37+
SignallingServer -> Streamer: SDP Offer (Streaming Request)
38+
39+
note over Streamer
40+
Handles Signalling
41+
and Streaming
42+
end note
43+
Activate Streamer #LightGreen
44+
45+
Streamer -> Streamer: Initialization
46+
Streamer -> SignallingServer: SDP Answer (Accept the connection)
47+
SignallingServer -> Client: SDP Answer (Accept the connection)
48+
49+
== 2. STUN/TURN and CandidateGeneration ==
50+
Client -> Client: LocalCandidate
51+
Client -> StunTurn: RequestCandidates
52+
StunTurn -> Client: Reflex/Relay candidates
53+
54+
Streamer -> Streamer: LocalCandidate
55+
Streamer -> StunTurn: RequestCandidates
56+
StunTurn -> Streamer: Reflex/Relay candidates
57+
58+
== 3. ICE Negotiation ==
59+
60+
Client -> SignallingServer : ICE candidates
61+
SignallingServer -> Streamer : ICE candidates
62+
Streamer -> SignallingServer: ICE candidates
63+
SignallingServer -> Client: ICE candidates
64+
65+
Client -> Client: Candidate selection
66+
Streamer -> Streamer: Candidate selection
67+
68+
|||
69+
70+
== 4. RTP Stream ==
71+
Client <--> Streamer: Video/Audio/Data over DTLS
72+
73+
Client -> Streamer: Call End
74+
75+
Deactivate Streamer
76+
77+
@enduml

0 commit comments

Comments
 (0)