Suggestion: Support for Chat with File API

[davidames]

Technical Feedback

Describe the precise feedback.
One of the core use-cases for LLM user's is "Chat with File" - I know Foundry offers several solutions for Chat with File Search, but Chat with File is different.

If I upload a 75 page PDF and ask to "summarize this document"; File Search/Vector search solutions can't answer that question.

OAI have supported this feature in their API's for several months; Claude supports this too, even IaaS offerings such as AWS Bedrock.

Would love for AOAI to support this too and I guarantee it would drive up token usage dramatically! :)

Here is the link to the OAI announcement feature: https://community.openai.com/t/direct-pdf-file-input-now-supported-in-the-api/1146647

Desired Outcome

Upload a PDF and ask "Summerize this document" to return a valid answer.

Current Workaround

PDF=> Image and then use vision images. Slow process!

[leestott]

"Chat with File" (as in full-document understanding and summarization) is a distinct use case from "Chat with File Search", which is optimized for retrieval-augmented generation (RAG) and chunk-level grounding. Azure AI Foundry currently emphasizes the latter via vector stores and semantic search, but there are ways to approximate your desired outcome using Foundry’s orchestration capabilities.

Here’s a breakdown of how you can implement true document summarization in Azure AI Foundry today, along with some solution patterns and workarounds:

Understanding the Gap

Feature	File Search (Foundry)	Chat with File (OAI-style)
Chunk-level retrieval	✅	❌
Full-document summarization	❌	✅
Token-aware orchestration	❌ (currently)	✅
Native PDF ingestion	Partial (via Azure AI Document Intelligence or manual parsing)	✅

Workaround: Summarization Pipeline in Foundry

You can build a custom summarization agent using Azure AI Foundry by combining:

• Azure Blob Storage (for PDF upload)
• Azure AI Document Intelligence or pdfplumber (to extract text)
• LangChain or Foundry SDK (to chunk and orchestrate)
• Azure OpenAI GPT-4o (for summarization)

Sample Architecture

1. Upload PDF to Blob Storage
2. Extract text using:
   • pdfplumber (for raw text)
   • or Azure AI Document Intelligence (for layout-aware extraction)
3. Chunk text using RecursiveCharacterTextSplitter (LangChain or Foundry SDK)
4. Summarize using:
   • map_reduce or refine summarization chains
   • GPT-4o via Azure AI Foundry’s chat.completions API
5. Return final summary to user

Example Code Snippet (Python)

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.summarize import load_summarize_chain
from langchain_openai import AzureChatOpenAI

llm = AzureChatOpenAI(deployment_name="gpt-4o", temperature=0.3)

text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=200)
docs = text_splitter.create_documents([full_pdf_text])

chain = load_summarize_chain(llm, chain_type="map_reduce")
summary = chain.run(docs)

You can wrap this in a Foundry Agent using the custom tool interface and expose it via a chat UI.

Alternative: Azure AI Language Native Document Summarization

Azure AI Language now supports native PDF summarization via REST API (in preview). This allows you to:

• Upload a PDF to Blob Storage
• Call the summarization API with a SAS token
• Get back a ranked extractive summary
  Learn how to use it

Suggestion for Foundry Team
A native summarize_file tool in Foundry Agents would be a game-changer, especially if it supports:

• Full-document ingestion
• Token-aware chunking
• Streaming summarization