[BUG] WebRTC Audio Forwarding: transport-cc Extension ID Mismatch

[bronze1man]

Which version are you using?

1.15.3

Which operating system are you using?

Linux amd64 standard

Describe the issue

android webrtc-> mediamtx -> android webrtc do not have any second.

mediamtx does not remap transport-cc extension IDs when forwarding audio, causing SDP/RTP mismatch and audio failures.

I have checked the mediamtx source code . I found that mediamtx does not remap any extension ID in rtp packet between webrtc publisher and webrtc reader.

Describe how to replicate the issue

detail: #5143

1. Android publish: a=extmap:4 ...transport-wide-cc...
2. mediamtx subscribe: a=extmap:3 ...transport-wide-cc...
3. RTP packets: extension ID=4
4. Receiver ignores ID=4 → BWE broken, audio may delay/fail

MediaMTX configuration

No response

MediaMTX logs

No response

Network dump

No response

[bronze1man]

• Modified the mediamtx code: When copying the audio stream logic during WebRTC playback, all extensions on the RTP packet were removed. Now, Android → mediamtx → Android has sound. Upon closer inspection, the playback's answerSdp contained a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01, but the RTP packet only had an extension with ID 4, which caused an error.

internal/protocols/webrtc/from_stream.go:367 (func setupAudioTrack)

r.OnData(
	media,
	opusFormat,
	func(u *unit.Unit) error {
		for _, orig := range u.RTPPackets {
			pkt := &rtp.Packet{
				Header:  orig.Header,
				Payload: orig.Payload,
			}

			// recompute timestamp from scratch.
			// Chrome requires a precise timestamp that FFmpeg doesn't provide.
			pkt.Timestamp = curTimestamp
			curTimestamp += uint32(opus.PacketDuration2(pkt.Payload))

			ntp := u.NTP.Add(timestampToDuration(int64(pkt.Timestamp-u.RTPPackets[0].Timestamp), 48000))
			for _, id := range pkt.GetExtensionIDs() { // delete all extensions in the rtp.
				pkt.DelExtension(id)
			}
			track.WriteRTPWithNTP(pkt, ntp) //nolint:errcheck
		}

		return nil
	})

• Without modifying the mediamtx code, but deleting all extensions related to http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 in the answerSdp. Now, Android → mediamtx → Android has sound.

• Without modifying the mediamtx code, changing the ID of the audio's a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 from 3 to 4 in the playback stream. This did not work, and the playback failed (no video and no sound).

• Modified the mediamtx code: When copying the audio stream logic during WebRTC playback, the ID 4 extension on the RTP packet was changed to 3. Now, Android → mediamtx → Android has sound.

internal/protocols/webrtc/from_stream.go:367 (func setupAudioTrack)

r.OnData(
	media,
	opusFormat,
	func(u *unit.Unit) error {
		for _, orig := range u.RTPPackets {
			pkt := &rtp.Packet{
				Header:  orig.Header,
				Payload: orig.Payload,
			}

			// recompute timestamp from scratch.
			// Chrome requires a precise timestamp that FFmpeg doesn't provide.
			pkt.Timestamp = curTimestamp
			curTimestamp += uint32(opus.PacketDuration2(pkt.Payload))

			ntp := u.NTP.Add(timestampToDuration(int64(pkt.Timestamp-u.RTPPackets[0].Timestamp), 48000))
			for _, id := range pkt.GetExtensionIDs() {
				if id == 4 {
					extContent := pkt.GetExtension(4)
					pkt.DelExtension(4)
					pkt.SetExtension(3, extContent)
				}
			}
			track.WriteRTPWithNTP(pkt, ntp) //nolint:errcheck
		}

		return nil
	})

This confirms the issue was related to a mismatch between the RTP header extension ID specified in the SDP (a=extmap:3) and the ID actually used in the RTP packet (ID 4) for the transport-wide congestion control extension (http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01).

[bronze1man]

publisher answerSdp

v=0
o=- 2573546199127750258 1761803600 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic:WMS *
a=fingerprint:sha-256 31:90:F7:E9:8D:FD:66:CF:C0:D5:4D:9F:53:0F:ED:30:0C:11:16:5C:F8:47:06:42:8E:E8:7C:DF:E5:30:B2:99
a=extmap-allow-mixed
a=group:BUNDLE 0 1
m=video 9 UDP/TLS/RTP/SAVPF 96 39 98 127 104
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:mlhVIRvLdsMOOQLH
a=ice-pwd:ejNgxCNupOTSxjkvEgruKRBeOsinTTSs
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 transport-cc
a=rtpmap:39 AV1/90000
a=fmtp:39 level-idx=5;profile=0;tier=0
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=rtcp-fb:39 transport-cc
a=rtpmap:98 VP9/90000
a=fmtp:98 profile-id=0
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtcp-fb:98 transport-cc
a=rtpmap:127 H264/90000
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=rtcp-fb:127 transport-cc
a=rtpmap:104 H265/90000
a=rtcp-fb:104 nack
a=rtcp-fb:104 nack pli
a=rtcp-fb:104 transport-cc
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=recvonly
a=candidate:1636828629 1 udp 2130706431 10.15.60.69 8189 typ host
a=candidate:1636828629 2 udp 2130706431 10.15.60.69 8189 typ host
a=end-of-candidates
m=audio 9 UDP/TLS/RTP/SAVPF 111 9 0 8
c=IN IP4 0.0.0.0
a=setup:active
a=mid:1
a=ice-ufrag:mlhVIRvLdsMOOQLH
a=ice-pwd:ejNgxCNupOTSxjkvEgruKRBeOsinTTSs
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=rtcp-fb:111 transport-cc
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=recvonly

important part

a=rtpmap:8 PCMA/8000
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=recvonly

reader answerSdp

v=0
o=- 2797484869260670988 1761803601 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic:WMS *
a=fingerprint:sha-256 F1:9E:8A:0B:66:69:02:EC:36:5F:D3:C2:46:73:60:D2:80:83:6A:B4:9D:AE:AD:B1:99:7B:A3:50:95:DA:01:72
a=extmap-allow-mixed
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:QZCCHtXqJMgGZunb
a=ice-pwd:eRLXdFRKarXkOmWDlpOBWxYAyKwPqXSU
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=rtcp-fb:111 transport-cc
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=ssrc:2989979931 cname:mediamtx
a=ssrc:2989979931 msid:mediamtx audio
a=ssrc:2989979931 mslabel:mediamtx
a=ssrc:2989979931 label:audio
a=msid:mediamtx audio
a=sendonly
a=candidate:684729552 1 udp 2130706431 10.15.60.69 8189 typ host
a=candidate:684729552 2 udp 2130706431 10.15.60.69 8189 typ host
a=end-of-candidates
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:active
a=mid:1
a=ice-ufrag:QZCCHtXqJMgGZunb
a=ice-pwd:eRLXdFRKarXkOmWDlpOBWxYAyKwPqXSU
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 transport-cc
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=ssrc:3449909277 cname:mediamtx
a=ssrc:3449909277 msid:mediamtx video
a=ssrc:3449909277 mslabel:mediamtx
a=ssrc:3449909277 label:video
a=msid:mediamtx video
a=sendonly

important part

a=rtcp-fb:111 transport-cc
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01

Looks like the audio and video ext id of http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 is both not matching. (publisher 4, reader 3) but only the audio has problem.

[bronze1man]

workaround of this bug right now (without change mediamtx code ):

• change the answerSdp , delete something like a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
  in audio part.

suggest fix in mediamtx

• Correct remap extension ID in rtp packet between webrtc publisher and webrtc reader in one path. (may be rtsp?)