Control over outgoing bitrate for Audio\Video and understaring bitrate strategy (HaishinKit.swift)

[hemule]

Hi!

We're using HaishinKit.swift for our RTMP broadcasting app on iOS and are curious about a few things regarding controlling outgoing stream parameters.

If somebody has already solved this and knows what might be the reason — please share:

1. Bitrate "spikes" at the start of the stream: We noticed that bitrate "spiked" for a moment at the start of the broadcast, sometimes much above the target value (we don't know what triggers this exactly): e.g, we're broadcasting at 2,000 kbit/s and it spiked to, like, 20,000. Some streaming platforms "don't like" this and drop the RTMP connection.

2. "Bitrate strategy" inconsistency for audio&video: When the internet connection quality drops — video & audio bitrate also drop, as expected. But as soon as the connection quality restores: video bitrate restores to target values as well, but audio bitrate stays the same regardless (using Constant Bitrate strategy). Any ideas why this might happen?

3. Actual outgoing bitrate monitoring: Regarding the first 2 "issues" — we hoped to implement some kind of internal monitoring to keep track of the actual bitrate for audio&video. But we didn't find any public properties to obtain such information during the broadcast. If you managed to find a way to monitor the actual outgoing bitrate for audio&video — please share.

[shogo4405]

2. 3 Bitrate strategy

I’m referring to the class names in the current main branch. In version 2.0.0, class names may differ due to typo fixes and other changes, so please interpret them accordingly.

I'm provided a protocol called StreamBitRateStrategy. By customizing this, you should be able to achieve what you’re aiming for.

let myStrategy = MyBitRateStrategy()
let stream = RTMPStream()
stream.setBitRateStrategy(myStrategy)

HaishinKit.swift/HaishinKit/Sources/Stream/StreamBitRateStrategy.swift

Lines 3 to 12 in ee08ab4

	/// A type with a network bitrate strategy representation.
	public protocol StreamBitRateStrategy: Sendable {
	/// The mamimum video bitRate.
	var mamimumVideoBitRate: Int { get }
	/// The mamimum audio bitRate.
	var mamimumAudioBitRate: Int { get }
	/// Adjust a bitRate.
	func adjustBitrate(_ event: NetworkMonitorEvent, stream: some StreamConvertible) async
	}

The current output bitrate can be obtained via the NetworkMonitorEvent used in the adjustBitrate method: Please refer to it for details.

HaishinKit.swift/HaishinKit/Sources/Network/NetworkMonitorReport.swift

Lines 3 to 15 in ee08ab4

	/// The struct represents a network statistics.
	public struct NetworkMonitorReport: Sendable {
	/// The statistics of total incoming bytes.
	public let totalBytesIn: Int
	/// The statistics of total outgoing bytes.
	public let totalBytesOut: Int
	/// The statistics of outgoing queue bytes per second.
	public let currentQueueBytesOut: Int
	/// The statistics of incoming bytes per second.
	public let currentBytesInPerSecond: Int
	/// The statistics of outgoing bytes per second.
	public let currentBytesOutPerSecond: Int
	}

[shogo4405]

1. Bitrate "spikes"

This is a characteristic behavior of VideoToolbox. Try specifying the dataRateLimits as shown below. You can finely control the bitrate to ensure it does not exceed 1.5 times the specified bitrate.

var videoCodecSettings = VideoCodecSettings()
videoCodecSettings.dataRateLimits = [0, 0]
stream.setVideoCodecSettings(videoCodecSettings)

HaishinKit.swift/HaishinKit/Sources/Codec/VideoCodecSettings.swift

Lines 185 to 192 in ee08ab4

	if bitRateMode == .average {
	if let dataRateLimits, dataRateLimits.count == 2 {
	var limits = [Double](repeating: 0.0, count: 2)
	limits[0] = dataRateLimits[0] == 0 ? Double(bitRate) / 8 * 1.5 : dataRateLimits[0]
	limits[1] = dataRateLimits[1] == 0 ? Double(1.0) : dataRateLimits[1]
	options.insert(.init(key: .dataRateLimits, value: limits as NSArray))
	}
	}

[hemule]

@shogo4405 Thank you for your prompt response! ❤️

Couple of followup questions:

1. Is there a way to monitor actual encoding bitrate separately for audio and video? (rather that final, "combined", network bitrate)?
2. Is there a way to manage bitrate strategy separately for Audio? (and if\how VideoBitrateStrategy affects strategy for Audio if affects at all)

P.S. Also we'd be happy to ask you for the personal consultation but I didn't manage to find a contact info to contact you directly.

[shogo4405]

The email address is now made public on GitHub. You should be able to see it once you're logged in: https://github.com/shogo4405

1. It is not possible to retrieve separate values. The total includes audio and video data, along with several bytes per message for RTMP control codes.
2. This is not provided by default. However, it is possible to change the audio bitrate during streaming as shown below, so please implement it yourself.

Compared to changing the video bitrate, changes in audio bitrate can sometimes result in a more unpleasant experience due to perceived drops in sound quality.

Also, while video bitrate is typically set around 2 Mbps, many services seem to stream audio at around 128 kbps.
Because the video bitrate is significantly higher, I have not implemented dynamic bitrate changes for audio.