Deep Research model reliability issues

[davidames]

I have setup a Deep Research agent in Foundry following these instructions:

https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/deep-research-samples?pivots=csharp

It works really well on most requests such as

Give me the latest research into quantum computing over the last year.

That is, we get the complete report and the results look really good.

But if I ask questions that are probably not all of that appropriate for a Deep Research tool, such as

What is the best way to attract top software developers to a small startup in Washington DC?

It will ask the clarifying questions (as in the good case) and say "Starting Research" / "Producing Report" and then the run status changes to Complete immediately without actually doing the research or outputting anything else.

Here is the final response with the status set to complete:

Here is the entire conversation

My guess is that the deep research tool inside of AI Services is failing/refusing the request, is there any way to know for sure? I would like to be able to handle this case gracefully.

[amynic]

Hi @davidames thanks for haring in detail your experience with deep research model in Azure AI Foundry. I was having a look around and a couple of next steps come to mind

• are you using streaming mode? I noted this sentence in the doc: Limitation: The Deep Research tool is currently recommended only in nonstreaming scenarios. Using it with streaming can work, but it might occasionally time out and is therefore not recommended. which sounds a little bit like what you might be facing

• you also asked about how you can find out more - I would recommend enabling tracing in Azure AI Foundry to be able to break down further calls and input/outputs that are going to your agent: https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/observability/telemetry-with-azure-ai-foundry-tracing

[davidames]

Non streaming and thanks for the tip! Didn't know about tracing!

[davidames]

Hi @amynic -

1. As per the screenshot below, streaming is false.
2. I enabled telemetry logging as per above, and I can see them in foundry but this only logs inside my application - I'm actually after logs that show what is going on inside the Agent AI Service, and why the Deep Research request is failing to start & why.

This problem is intermittent but regular. On poorly worded questions, (eg, non-research type) it happens in 8/10 requests. In better worked questions, it happens 2/10 requests.

I had an interesting failure today where it failed to start, I stared a new run with the message "Are you done yet" and that resulted in the agent service starting the deep research task, but that technique only worked once.

I'm running out of ideas for "next steps" - what would you suggest?