feat: fdp changes

[Roopan-Microsoft]

Purpose

This pull request introduces significant changes to the AI infrastructure and backend codebase, transitioning from the "AI Hub" model to the new "AI Foundry" model. Key changes include updates to resource definitions, configuration management, and dependencies to align with the new architecture. Below is a summary of the most important changes:

Infrastructure Updates

• Added new abbreviations for aiFoundry and aiFoundryProject in infra/abbreviations.json to support the updated naming conventions.
• Replaced AI Hub resources with AI Foundry resources in infra/deploy_ai_foundry.bicep and infra/main.bicep, including definitions for aiFoundryName, aiFoundryProject, and related properties. The previous AI Hub resources are now commented out. [1] [2]
• Updated outputs in infra/main.bicep to reflect the new AI Foundry model, including aiFoundryName, aiProjectName, and projectEndpointString.

Backend Configuration

• Added a new environment variable AI_PROJECT_ENDPOINT to the configuration class in src/backend/common/config/config.py to support the AI Foundry endpoint.
• Debug logging added to print the AI_PROJECT_ENDPOINT value for easier troubleshooting during development.

Dependency Updates

• Updated semantic-kernel[azure] to version 1.32.2 and pinned openai to version 1.84.0 in src/backend/requirements.txt to ensure compatibility with the new AI Foundry model.
• Upgraded azure-ai-projects to version 1.0.0b11 for access to the latest features and APIs.

Code Refactoring

• Updated imports in src/backend/sql_agents/agents/agent_base.py to reflect changes in the Azure AI SDK, switching from azure.ai.projects.models to azure.ai.agents.models.
• Modified the process_batch_async function in src/backend/sql_agents/process_batch.py to use the AI_PROJECT_ENDPOINT for creating an AzureAIAgent client.

Resource Management

• Adjusted role assignments in infra/main.bicep for AI Foundry resources, including the addition of aiUserAccessProj and aiUserAccessFoundry for finer-grained role-based access control.

These changes collectively enable the migration to the AI Foundry model, providing enhanced flexibility and improved resource management for AI projects.

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information