Optimize Pagination by Filtering Beacon Events

[torrybr]

I am encountering performance issues during pagination with the /messages API in rooms, due to processing a large number of org.matrix.msc3489.beacon events. While encrypted rooms still need to process each beacon event, I propose using the not_types RoomEventFilter to speed up pagination in non-encrypted rooms.

Additionally, I'm looking for guidance on optimizing live location sharing to improve its scalability within the SDK. Any insights or alternative strategies would be greatly appreciated.

Example timeline, the spec needs to only render the two m.text items in the timeline:

m.text
org.matrix.msc3489.beacon
org.matrix.msc3489.beacon
...2,000 more org.matrix.msc3489.beacon
m.text

Proposed solutions:

1. Modify the paginate function to accept an optional RoomEventFilter, allowing clients to specify a custom filter.

async fn paginate(
        &self,
        dir: Direction,
        num_events: UInt,
        room_event_filter: Option<RoomEventFilter>,
    ) 

2. Automatically filter by default on MessageOptions

impl MessagesOptions {
    /// Creates `MessagesOptions` with the given direction.
    ///
    /// All other parameters will be defaulted.
    pub fn new(dir: Direction) -> Self {
        Self { from: None, to: None, dir, limit: uint!(10), filter: assign!(RoomEventFilter::default(), {
                not_types: Some(vec![MessageLikeEvent::Beacon.to_string()])
            }) }
    }

[bnjbvr]

Hi! thanks for opening an issue. While I can imagine it's a bother to have to paginate through these beacon events, I don't think it's a good idea to filter them out by default, because it would be super hard to discover that they're filtered out. Especially in the event cache, which is supposed to fetch and store all the remote events by default; filtering should happen locally for the case of the event cache. Adding ways to filter out in the different pagination mechanisms would be good, though, so that's something we can improve in the different APIs we have.

[torrybr]

Thanks for the thoughtful response! That makes sense—I can see how filtering in the event cache by default could lead to surprising behavior, especially for clients that rely on the cache to reflect the full set of events from the server.

I'll look into implementing local filtering in the UI layer for now, and potentially wrapping pagination to handle this more gracefully on the client side. I also agree that allowing optional filtering in the pagination APIs would be a helpful addition—happy to explore contributing that if it's something the team would support.

Appreciate the insight!

[bnjbvr]

I also agree that allowing optional filtering in the pagination APIs would be a helpful addition—happy to explore contributing that if it's something the team would support.

That sounds great!