Video Generation Endpoint

[omrishiv]

Hello,
Bedrock has support for video generation using Nova Reel and Luma Ray 2 models; however, OpenAI does not have a video generation endpoint on its API. Is there a way to support this feature using LiteLLM? Ideally this would not only be a pass through feature but one that's in the client.

Let's open it up for discussion. I would default to looking at the image_generation endpoint as inspiration. I don't think we need n, but the rest of the fields seem reasonable to start with.

[krrishdholakia]

Do other providers have a video_Generation endpoint?

• vertex ai
• vllm

etc.?

[krrishdholakia]

I can see google has one - https://cloud.google.com/vertex-ai/generative-ai/docs/video/generate-videos

But they also began doing image generation on gemini via the generate content endpoint

If Nova Reel / Luma Ray 2 - support text input -> b64 output, should it just be via /chat/completion?

[omrishiv]

https://docs.aws.amazon.com/nova/latest/userguide/video-gen-access.html#video-gen-start-a-job is the API. Typewise, this looks very similar to the image generation Nova Canvas tasks. Does it make sense to have its own endpoint for it rather than chat completions?

[krrishdholakia]

It would be weird to call an image_generation() endpoint, and expect video output.

We see similar feedback with /chat/completion, and supporting video_url's via a separate video_url object, instead of image_url in the content list.

It would be good to collect a couple examples of how providers are supporting this - and that should help guide implementation.

Curious if any openai-compatible providers have similar functionality.

[omrishiv]

to clarify: I don't think we should add this to the image_generation() endpoint, I think we need a video_generation() endpoint. I also see in the future wanting an audio_generation() endpoint, so I think it would be good to have a base media_generation() endpoint that image, video, and audio tasks could extend, but maybe that's premature?