Automatic rollover/fallback for same name deployments (Azure)

[a-premo]

Reached out about this in discord and seems we have to manually create fallbacks for multi region azure open ai deployments. Seems counter intuitive given proxy should handle by default. Have not dug into the code for this but would love to either have an eng help or point me in direction to where I can make a PR.

Basically we should be able to set model name to gpt-4o multiple times in config map, with each having a different base endpoint (ex: useast1, useast2, uswest1 etc). Then if useast1 fails we can have it fallback/rollover to next endpoint.

Discord discussion: https://discord.com/channels/1123360753068540065/1180582199393787945/threads/1359560544566382652
If this is already supported please let me know! Happy to help or provide more info!