Should I be using tokio::LinearReader instead of MessageStream?

[lucasw]

I'd like to first scan through an mcap for all messages on a specific topic, and then read all message data out on all the topics for multi-gigabyte mcaps- this is currently working with MessageStream though I'd like to improve performance. My application is already using tokio, though possibly I'm doing something wrong with it to make it use up a lot of cpu.

examples/conformance_reader_async.rs shows opening a file using File instead of the memmap::Mmap, and then gets to parse_record, but I would have to check if the opcode is MESSAGE then get the channel let channel = summary.channels.get(&header.channel_id) to see what topic is was on?

[james-rms]

@lucasw there's not a lot of example code available yet for dealing with MCAPs in an async context. For what you're doing, I'd guess using the sans_io::IndexedReader would be your best bet. See the example code here: https://docs.rs/mcap/latest/mcap/sans_io/indexed_reader/struct.IndexedReader.html

Since the sans_io code does not depend on any specific I/O method, you can swap in a tokio::fs::File in that example and it should "just work". Also note that you can use the IndexedReader to filter for specific topics using IndexedReaderOptions::include_topics.

[bennetthardwick]

I would add that if you have a local file available then using mmap will likely get you the best performance. Just ensure that you're using tokio::spawn_blocking so it has a dedicated thread to run on.