How to handle large files without buffering in memory

[dmaiken]

I am creating a ktor app running on Netty to process images and other large files. I want to grab the first 1-4KB of the input to determine the content type and then use the ByteReadChannel exposed by the multipart provider(). The issue is that I am grabbing the first part of the ByteReadChannel but given the read-once property of ByteReadChannel, I cannot rewind. It looks like the DoubleReceive plugin won't work with streams either.

How do you think I can effectively merge the first part of the image I grabbed with the remaining, unread channel? Something like this:

val header = ByteArray(1024)
content.readAvailable(header, length = header.size) // content is the ByteReadChannel exposed from the multipart provider()
val mimeType = deriveValidMimeType(header.copyOf())

ByteReadChannel(header) + content

The goal here is to not have to buffer the entire file into heap.

Thank you!

[bjhham]

Hi @dmaiken, there is a ByteReadChannel.peek(Int): ByteString extension function for this kind of scenario.