Convert MJPEG RTP Packets to JPEG Image

[Tekunalogy]

Hello!
I was looking through the examples, and I wanted to develop something that would read incoming rtp packets and save them as JPEG images.

I did some preliminary work here: https://github.com/Tekunalogy/rtpmjpeg2jpeg

However, as you can see in the repository, the resulting jpeg image is nowhere close to the gstreamer "videotestsrc".

I did try using the server and "record-format-mjpeg" examples:

• server: https://github.com/bluenviron/gortsplib/blob/main/examples/server/main.go
• mjpeg record: https://github.com/bluenviron/gortsplib/blob/main/examples/client-record-format-mjpeg/main.go

and I got a strange issue:

...
2024/03/07 18:08:30 33 RTP packets lost
2024/03/07 18:08:30 11 RTP packets lost
2024/03/07 18:08:30 20 RTP packets lost
2024/03/07 18:08:30 27 RTP packets lost
2024/03/07 18:08:30 5 RTP packets lost
2024/03/07 18:08:30 24 RTP packets lost
2024/03/07 18:08:30 22 RTP packets lost
2024/03/07 18:08:30 20 RTP packets lost
2024/03/07 18:08:30 8 RTP packets lost
2024/03/07 18:08:31 8 RTP packets lost
2024/03/07 18:08:31 2 RTP packets lost
2024/03/07 18:08:31 15 RTP packets lost
2024/03/07 18:08:31 43 RTP packets lost
...

So I have a feeling that this is part of the issue. Adding queue/queue2 to the gstreamer command did not resolve the issue either.

Could someone point me in the right direction, or maybe some other examples to look at? I know in the Pion webrtc library, they have an h264 writer module, which takes the data and makes it into a writable format if I recall correctly, so am I going to need something similar to that for the mjpeg specification?

gstreamer command:

gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080,format=I420 ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=9000

[aler9]

Hello, you are using the wrong example (client-record-format-mjpeg is used to write a stream to the network, not to read one) and you are using raw RTP packets as the transmission mechanism - this is way harder to setup than RTSP and much less secure. Use RTSP instead.

Start the example RTSP server:

go run ./examples/server

Start streaming MJPEG frames to the server with rtspclientsink:

gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080,format=I420 ! jpegenc ! rtspclientsink location=rtsp://localhost:8554/mystream

Then use the client-play-format-mjpeg example to read the MJPEG stream and save JPEG images:

https://github.com/bluenviron/gortsplib/blob/main/examples/client-play-format-mjpeg/main.go