Question : onSegment

[Alex-S-H-P]

Hi, I just wanted to know how one could run code for every segment (I'm trying to store the segments individually).

I think it would be possible to build something on top of onFrame, but I'm not sure whether the frame is actually the segment or really a frame of the video feed.

Thank you for your time

[aler9]

Hello, what do you mean by segment?

[Alex-S-H-P]

Hi !

By segment, i meant the result of the segment muxer, as defined by ffmpeg. I wish to cut my stream in many small video files, and store them on the server machine.

---

While waiting for your answer, i have started to experiment with the onFrame method of main.go, in such a manner :
A stack of bytes does, at every call of onFrame, append the current payload. When the stack gets too big (meaning that there are too many payload bytes in it, or alternatively that there are too many payloads), we "dump" it into memory.

My explanation isn't the easiest to understand, so there's a graph to potentially make it easier to understand.

I fear however, that once i start fetching the payloads from memory they won't be readable by ffmpeg and/or gstreamer (especially since they are limited in size to 1472 bytes as per my senders limitations). If you have insight, or ideas to upgrade this, then it would be strongly appreciated !

---

Graph

                _____________
               | rtsp server |__________________________________________________
               |_____________|                                                  |
incoming stream|    __________      _______                                     |
==============>|==>| on frame |===>| stack |                                    |
               |   |__________|    |_______|                                    |
               |                       ||if stack is too large                  |
               |_______________________||_______________________________________|
                                       \/
                                     memory

EDIT :

i'd also like to extract the stream name (as defined in the URL : a stream on rtsps://url.com:8555/hello should send data with hello in the name). From what i can see, tracks objects do handle URL parsing, but do not allow for the result to be accessible. I also do not know whether another access point is available...

[Alex-S-H-P]

For the question in the edit, a colleague and I found a solution. By shifting the import path to a local copy, you can then add a getter method in serversession.go. It's clunky, but it'll do.

func (self *ServerSession) StreamName() *string {
    return self.setuppedPath
}

[aler9]

Hello, in order to save the path name, it's enough to hook into OnSetup and store the path name:

func (sh *serverHandler) OnSetup(ctx *gortsplib.ServerHandlerOnSetupCtx) (*base.Response, *gortsplib.ServerStream, error) {
	pathName = ctx.Path

	return &base.Response{
		StatusCode: base.StatusOK,
	}, sh.stream, nil
}

[aler9]

Regarding the segments, first of all i'm assuming you're talking about a H264 video track.

• OnFrame allows you to access RTP/H264 packets.
• Segments are defined as sequences of NALUs (atomic units of H264) that start with a key frame. They can't be sequences of NALUs splitted arbitrarily

Therefore, you have to do some work:

1. you need to convert from RTP/H264 packets (which are limited to 1472 bytes) into NALUs (which don't have a size limit and can be splitted into multiple RTP/H264 packets)
2. you need to analyze the NALUs to find and wait for a IDR frame (which is a key frame which doesn't need any previous frame to be decoded)
3. after you have found an IDR, you can start filling the buffer
4. after a certain timeout has expired, and after you have found another IDR, you can flush the buffer and start again

start with the client-read-h264 example, convert it into a Server's OnFrame handler and edit it to analyze the NALUs:

var dec *rtph264.Decoder

func (sh *serverHandler) OnRecord(ctx *gortsplib.ServerHandlerOnRecordCtx) (*base.Response, error) {
	// instantiate a RTP/H264 decoder
	dec = rtph264.NewDecoder()

	return &base.Response{
		StatusCode: base.StatusOK,
	}, nil
}

func (sh *serverHandler) OnFrame(ctx *gortsplib.ServerHandlerOnFrameCtx) {
		if ctx.TrackID == h264Track {
			// convert RTP frames into H264 NALUs
			nalus, _, err := dec.Decode(ctx.Payload)
			if err != nil {
				return
			}

			for _, nalu := range nalus {
				typ := nalu[0] & 0x1F

				// we have an IDR frame and there isn't any existing buffer, or the current buffer is already of the desired duration
				if typ == 5 && (thereIsNoBuffer || time.Since(lastBufferCreation) => period) {
					// if there's an existing buffer, send it out and flush it
					// allocate a new buffer and start filling it
				} else {
					// if there's an existing buffer, insert the non-IDR frame into it
				}
			}
		}
}

A very similar approach is used to convert from RTSP to HLS into rtsp-simple-server:

https://github.com/aler9/rtsp-simple-server/blob/main/internal/hlsconverter/converter.go

[Alex-S-H-P]

I actually want to use h265, which is why i did not use the rtsp-simple-server, since they are more complex than h264. I will look into converting them. Do you have any good documentation for h265 i can read? I found none that were very indicative of how to handle this video format...

[aler9]

No sorry but work on H265 has to start yet. Only routing without decoding is supported.

[Alex-S-H-P]

Thanks. I feared it was the case, so I'll rely on ffmpeg + libx265 for now

EDIT :

For anyone in the future wondering, I'm planning on using this kind of code to pipe in/out of ffmpeg directly. In parrticular :

cmd := exec.Command("ffmpeg", "-y", // Yes to all
		//"-hide_banner", "-loglevel", "panic", // Hide all logs
		"-i", "pipe:0", // take stdin as input
		"-map_metadata", "-1", // strip out all (mostly) metadata
                // conversion happens here
		"-f", "mp3", // using mp3 muxer (IMPORTANT, output data to pipe require manual muxer selecting)
		"pipe:1", // output to stdout
	)

I just need to plug libx265 into the mix to handle conversion