Support for Temporal Interlaced Stereo3D (Frame-sequential) Format

[mikekoetter]

I was wondering if UltraGrid currently supports—or if there has been any consideration for supporting—Temporal Interlaced Stereo3D streams, where the Left and Right eye views alternate temporally at double the target framerate (i.e., frame-sequential 3D).

This format seems to be represented in libav as AV_STEREO3D_FRAMESEQUENCE under the AVStereo3DType enum:

Views are alternated temporally.

 frame0   frame1   frame2   ...
LLLLLLLL RRRRRRRR LLLLLLLL
LLLLLLLL RRRRRRRR LLLLLLLL
LLLLLLLL RRRRRRRR LLLLLLLL
...      ...      ...

This format is becoming increasingly relevant, especially with the emergence of newer Stereo3D displays that rely on frame-sequential input. Having support for capturing dual-link stereo sources and encoding/decoding to this format would be incredibly useful for certain workflows.

Would love to hear your thoughts!

[txtpauld]

I second this as this is coming up more and more as the number of ways that 3D is transmitted is growing. One facility may rely on dual stream 3D to their projector and then want to stream to another facility that wants to show it on a direct view surface that requires the signal be time-sequential interleaved 3D.

[MartinPulec]

Hi, sorry for the delay - I've missed the notification. No, I didn't know about that mode. So if I understand it correctly, that tempral intelaced 3D stream could be transparent for UltraGrid - displayed alternating frames exactly in the same way as captured. Only required is to set this for the compression (which is quite obvious, because this mode would hurt inter-frame compression)?

If so, I believe that could add support for that mode. But I've grepped the FFmpeg source codes and it looks like for the encoder, it is supported just for libx264. For the decoders, H.264 and HEVC might be supported - it looks like that the decoder sets it just to the metadata but I think it's possible that the feature is transparent for decompression.

[MartinPulec]

I've looked into it a bit more and I have a question - how should be handled left/right eye frame information? Would UltraGrid be required to set it somehow to the display, to be passed in metadata? I believe so, because otherwise there would have to be some switch in the displaying device to set the order... I can imagine this, but then major problem I see is that if one or odd number of frames get lost, then the eyes would get silently reversed. (This is the current state - actually this could be mitigated by either adding dummy/copy other eye or dropping the single eye frame determined by sequence number).

If my thoughts in above paragraph are correct. which is the expected device to be used for capture/playback? Because eg. DeckLink doesn't seem to support that mode, see section 3.29 here (also useful is sec. 2.4.3) so I think that I won't be able to pass the left/right eye information even if I had it.

[mikekoetter]

Sorry for the delay in this - I'm not exactly sure what the proper solution wold be here. In my opinion, I think the left/right eye pair would be treated as a single frame - if either of the eyes didn't arrive then the frame would be considered dropped and both previous eyes would be duplicated

[MartinPulec]

Hi again,

I would rather take a step back to more higher level view - what is the required functionality from the UltraGrid perspectives?

UltraGrid already supports 3D (although, it may need some maintenance in UG), when received eg. from DeckLink. If I understand that correctly, the AVStereo3DType in FFmpeg actually works on container-level to co-host left and right eye inside one container, perhaps both compressed independently? At least in case of common inter-frame compressions like H.264, H.265 or AV1?

My remarks:

1. UG actually doesn't use media container like file - in case of 3D, 2 compression context are already utilized
2. the frame-alternate mode is not supported anyhow in UG
3. to utilize such a feature, UG would still need to be informed that the received stream is actually frame-altering (to initialize 2 encoding instances)

So actually from my perspective it will be perhaps better to use something like video capture filter (--capture-filter) to make UltraGrid aware that in the stream there is stereoscopic and then perhaps on the receiver side something like postprocessor (-p) parameter to "serialize" the stream?

[mikekoetter]

I had not thought of it that way. That’s a beautiful approach From a support matrix perspective the “post-process” feature may be the only side of this that is really useful. In the workflows I’ve observed it’s these high end direct view display requirements that are ridged, not the client source devices.

…

-- Mike Koetter

On Wed, Jul 16, 2025 at 2:06 AM Martin Pulec ***@***.***> wrote: Hi again, I would rather take a step back to more higher level view - what is the required functionality from the UltraGrid perspectives? UltraGrid already supports 3D <https://github.com/CESNET/UltraGrid/wiki/Stereoscopic-3D-Video-Settings> (although, it may need some maintenance in UG), when received eg. from DeckLink. If I understand that correctly, the AVStereo3DType in FFmpeg actually works on container-level to co-host left and right eye inside one container, perhaps both compressed independently? At least in case of common inter-frame compressions like H.264, H.265 or AV1? My remarks: 1. UG actually doesn't use media container like file - in case of 3D, 2 compression context are already utilized 2. the frame-alternate mode is not supported anyhow in UG 3. to utilize such a feature, UG would still need to be informed that the received stream is actually frame-altering (to initialize 2 encoding instances) So actually from my perspective it will be perhaps better to use something like video capture filter (--capture-filter) to make UltraGrid aware that in the stream there is stereoscopic and then perhaps on the receiver side something like *postprocessor* (-p) parameter to "serialize" the stream? — Reply to this email directly, view it on GitHub <#440 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABS6HRQAJEXIDJYUQLZOLR33IYIZ3AVCNFSM6AAAAAB3RXVY6KVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZXGQ4DCNI> . You are receiving this because you authored the thread.Message ID: ***@***.***>