[QUESTION] How to detecting receiver bandwidth for adaptive bitrate (transceiverOnBandwidthEstimation, REMB, onSenderBandwidthEstimation)

[roxlu]

When we have a single sender (video, audio) and only a viewer, how can the sender detect the bandwidth of the receiver?

I've set the transceiverOnBandwidthEstimation() on the videeo transceiver but I'm not receiving any REMB packets. I've added some extra logging in PeerConnection/Rtcp.c and noticed that the onRtcpRembPacket() function is never called. I am receiving TWCC packets though.

[benwkim]

onRtcpRembPacket is called by onInboundPacket which is a callback function set here. Can you check your callback setup for onInboundPacket?

[roxlu]

@benwkim sorry for the late reply. I've checked and the onRtcpPacket in peerconnection/Rtcp.c is called. The onRtcpPacket is called, however the RTCP_PACKET_TYPE_SPECIFIC_FEEDBACK case is never triggered. This is the case which calls onRtcpRembPacket. I've added some more info below that might explain why I'm not receiving REMB packets.

To test this I've done the following:

• Clone this repository using commit b820e2e
• Added some printfs in Rtcp.c
• Configure and build with cmake
• Run kvsWebRtcClientMaster [channel]
• Open https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html
• Press Start Viewer

Offer SDP

o=- 3310153261655664093 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 9 0 8 13 110 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:fyov
a=ice-pwd:4dh1frHrW3tRNzhkdmD58/Q4
a=ice-options:trickle
a=fingerprint:sha-256 82:A5:09:83:6A:65:AA:D4:AF:7B:6C:84:00:7C:A9:71:40:96:34:A1:2C:B2:5A:E5:45:2D:48:C0:7F:63:C3:18
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:126 telephone-event/8000
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 102 103 104 105 106 107 108 109 127 125 39 40 41 42 43 44 45 46 47 48 112 113 114 49
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:fyov
a=ice-pwd:
a=ice-options:trickle
a=fingerprint:sha-256 82:A5:09:83:6A:65:AA:D4:AF:7B:6C:84:00:7C:A9:71:40:96:34:A1:2C:B2:5A:E5:45:2D:48:C0:7F:63:C3:18
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:35 VP9/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 profile-id=1
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:37 VP9/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=rtcp-fb:37 ccm fir
a=rtcp-fb:37 nack
a=rtcp-fb:37 nack pli
a=fmtp:37 profile-id=3
a=rtpmap:38 rtx/90000
a=fmtp:38 apt=37
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=102
a=rtpmap:104 H264/90000
a=rtcp-fb:104 goog-remb
a=rtcp-fb:104 transport-cc
a=rtcp-fb:104 ccm fir
a=rtcp-fb:104 nack
a=rtcp-fb:104 nack pli
a=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:105 rtx/90000
a=fmtp:105 apt=104
a=rtpmap:106 H264/90000
a=rtcp-fb:106 goog-remb
a=rtcp-fb:106 transport-cc
a=rtcp-fb:106 ccm fir
a=rtcp-fb:106 nack
a=rtcp-fb:106 nack pli
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=106
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:125 rtx/90000
a=fmtp:125 apt=127
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:41 H264/90000
a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:43 H264/90000
a=rtcp-fb:43 goog-remb
a=rtcp-fb:43 transport-cc
a=rtcp-fb:43 ccm fir
a=rtcp-fb:43 nack
a=rtcp-fb:43 nack pli
a=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f
a=rtpmap:44 rtx/90000
a=fmtp:44 apt=43
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:47 AV1/90000
a=rtcp-fb:47 goog-remb
a=rtcp-fb:47 transport-cc
a=rtcp-fb:47 ccm fir
a=rtcp-fb:47 nack
a=rtcp-fb:47 nack pli
a=fmtp:47 profile=1
a=rtpmap:48 rtx/90000
a=fmtp:48 apt=47
a=rtpmap:112 red/90000
a=rtpmap:113 rtx/90000
a=fmtp:113 apt=112
a=rtpmap:114 ulpfec/90000
a=rtpmap:49 flexfec-03/90000
a=rtcp-fb:49 goog-remb
a=rtcp-fb:49 transport-cc
a=fmtp:49 repair-window=10000000

Answer SDP

o=- 1369810618 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS myKvsVideoStream
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 127.0.0.1
a=candidate:4 1 udp 2130706431 
a=candidate:3 1 udp 2130706431 
a=candidate:2 1 udp 2130706431 
a=candidate:1 1 udp 2130706431 
a=candidate:0 1 udp 2130706431 
a=msid:myKvsVideoStream myAudioTrack
a=ssrc:513183044 cname:LNWWPBfOOePX34lW
a=ssrc:513183044 msid:myKvsVideoStream myAudioTrack
a=ssrc:513183044 mslabel:myKvsVideoStream
a=ssrc:513183044 label:myAudioTrack
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:B2h9
a=ice-pwd:
a=ice-options:trickle
a=fingerprint:sha-256 DF:4F:E1:0C:56:3C:71:A5:A6:32:F1:62:FE:44:36:43:53:3F:B8:E4:81:9D:CB:E9:A5:C4:44:3C:3E:F7:C0:D0
a=setup:active
a=mid:0
a=sendonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=rtcp-fb:111 nack
a=rtcp-fb:111 goog-remb
a=rtcp-fb:111 transport-cc
m=video 9 UDP/TLS/RTP/SAVPF 106
c=IN IP4 127.0.0.1
a=candidate:4 1 udp 2130706431 
a=candidate:3 1 udp 2130706431 
a=candidate:2 1 udp 2130706431 
a=candidate:1 1 udp 2130706431 
a=candidate:0 1 udp 2130706431 
a=msid:myKvsVideoStream myVideoTrack
a=ssrc:1700416570 cname:LNWWPBfOOePX34lW
a=ssrc:1700416570 msid:myKvsVideoStream myVideoTrack
a=ssrc:1700416570 mslabel:myKvsVideoStream
a=ssrc:1700416570 label:myVideoTrack
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:B2h9
a=ice-pwd:+C/19QhETGBzqzgDN+53MKd9
a=ice-options:trickle
a=fingerprint:sha-256 DF:4F:E1:0C:56:3C:71:A5:A6:32:F1:62:FE:44:36:43:53:3F:B8:E4:81:9D:CB:E9:A5:C4:44:3C:3E:F7:C0:D0
a=setup:active
a=mid:1
a=sendonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:106 H264/90000
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtcp-fb:106 nack
a=rtcp-fb:106 goog-remb
a=rtcp-fb:106 transport-cc