Not possible to have more than a few hundred open threads with file search when BYO resources

[jackirvine97]

Technical Feedback

Describe the precise feedback.
Hi everyone, my team found a potential limitation of the file search capability in the new foundry agent service, when you BYO resources.

In the context of BYO resources, you hook your own ai search resource into foundry.

Then, say when you upload a file to a thread, what happens is an ai search index is created in your resource for that thread. Each thread gets its own index.

The issue here is that AI search has a limit on the number of indexes per resource - for an S2 tier, it’s 200 (ie only 200 chats could be opened before the system fails).

I suspect official advice would be to not use thread vector stores in this way, and I agree. However, I wonder could this be better documented as it would be easy to fall into this trap and only release it when you get to load testing.

Has anyone else noticed this? I wonder would it be better if a single agent thread index was created for all threads with a thread Id being used as a filterable field under the hood? Would he interested to hear Microsoft’s thoughts?

Desired Outcome

Describe the desired outcome.

• improved documentation
• discussion on what is best way to use this capability
• raise awareness

Current Workaround

Don’t use file search in this way!

[arenk]

We’re having similar discussions here and would really value Microsoft’s perspective on how to approach this.

[justuswolff-audit]

We just ran into the same issue. We decided on the standard setup which uses it's own Azure AI Search. Given that we want to use file search on our threads, we are currently limited to a few chats (50 on S1 or 200 for S2 AI Search). We looked into providing the threadId as metadata to the file and then use the AzureAISearchTool for which we could provide a filter option. However, this does not seem to work, since we can't attach metadata to a file we upload.

@amynic are there any news on this issue? What is the recommended workaround?

[nicholasdbrady]

Adding @lindazqli for agent tools detailed documentation feedback and @farzad528 for AI Search (feel free to add someone else to support this question if you feel they can provide better support.

[justuswolff-audit]

We now pivoted to use the AI Search Tool. We upload the files to a default vector store and within a thread filter for the document ids that are available to the user. This approach might help someone else who also ran into this limitation...

However, we would appreciate suggestions from Microsoft. And that the tools at thread level don't work is another issue :)