Subscription channel can run out of capacity and drop events

[pborzenkov]

The broadcast channel used by subscriptions (

corrosion/crates/corro-agent/src/api/public/pubsub.rs

Line 621 in 93561d1

let (sub_tx, sub_rx) = broadcast::channel(10240);

) can run out of capacity and skip the events in case the data is read really fast from the disk.

Somehow, in this case, forward_sub_to_sender simply stops forwarding any event -

corrosion/crates/corro-agent/src/api/public/pubsub.rs

Line 728 in 93561d1

Ok((event_buf, meta)) = sub_rx.recv() => {

It looks like the select! never matches RecvError::Lagged for some reasons.

[pborzenkov]

The problem with select! is fixed by #184

The actual channel capacity problem is still present. Firing up a new subscription should probably behave the same way as subscribing to existing one: the initial rows are sent via mpsc channel and the broadcast channel is used after that to send change events only.