Prompt flow - can you use serverless endpoints?

[Kubex212]

I deployed mistral-ocr-2503 model as a serverless endpoint since:

"Dedicated serverless endpoint: This model will be deployed as a dedicated serverless endpoint—it's not yet available for unified endpoint deployment."

I wonder if it is possible to use this endpoint in prompt flow or is there any other way to use this model's output as another model's input other than just calling the models manually on my side?

Thanks, Jakub.

[leestott]

Hi @Kubex212 models like mistral-ocr-2503 deployed via dedicated serverless endpoints aren't yet part of the unified endpoint ecosystem, you can still integrate them into Prompt Flow with a bit of orchestration.

Here are two practical ways to connect the output of this serverless endpoint to downstream models:

Custom Tool in Prompt Flow
You can wrap the OCR endpoint call into a custom Python tool and include it in your Prompt Flow pipeline. Here's how:

• Create a Python module (tool) that sends a request to the mistral-ocr-2503 endpoint using the requests library.
• Extract the text response from the model.
• Pass that text as an input to a downstream node (e.g., a prompt for a GPT model).

Prompt Flow's ability to use custom components means you can easily chain models—even across endpoint types.

Chained Orchestration with Azure Functions
Since you're well-versed in Azure Functions, another elegant way is:

• Use an Azure Function to invoke the OCR model.
• Have the function return the result in a format consumable by Prompt Flow (e.g., return JSON with content key).
• In Prompt Flow, call that function from within a component node or via an HTTP request tool.

This keeps everything modular and enables better reuse and deployment flexibility.

For official guidance and deployment details on the mistral-ocr-2503 model and its integration with Azure AI Foundry, you can check out this Microsoft Tech Community blog post. It outlines how the model works, deployment steps, and its capabilities as a serverless endpoint.