Support identical Routes from multiple configuration sources

[rwkarg]

Following on from #1534

It looks like having two duplicate Routes would be rejected which is a reasonable approach.

Duplicates are an error and will not be merged.

Ask - extend the multiple config support to (perhaps optionally):

• Collapse duplicate (same key) and identical Routes to a single Route
• Combine duplicate (same key) Clusters to one aggregate Cluster that contains all Destinations

My use case is acting as a load balancer in front of multiple (usually identical) k8s clusters. Services in each k8s cluster would have annotations to configure the YARP Route and the Pods behind that Service would make up the YARP Cluster Destinations. The happy path would be that all k8s cluster define the same Route the same way, so here duplicates that are identical would be collapsed to one Route. Then duplicate Clusters would be merged together. This would give one Route in YARP that directs to all of the Pods across all k8s clusters.

In cases where the Routes disagree, then the existing behavior of rejecting the new config and keeping the previous config (or crashing on startup) is reasonable.

[rwkarg]

And now that I've started this discussion, I don't think that multiple IProxyConfigProviders would be the way I'd use it since the underlying k8s clusters is not a static set (I can't just create one KubernetesConfigProvider for each cluster at startup since clusters can be created/destroyed at any time).

In actuality, each k8s cluster will have a controller that watches Services and pushes it's cluster's config to object storage. Then YARP will be watching that bucket for updates and pulling down and merging the configs from each cluster. So it's more of a Config Merging solution I'm looking for. I'll take a look at what trying to reuse the multiple configuration sources merging logic might look like in a single IProxyConfigProvider.

[rwkarg]

It's looking more like this will work better as special case merging logic for my use case as opposed to trying to support everything in Routes/Clusters (ex. we won't be going crazy with HttpClientConfig but a general purpose solution would need to handle that in a sane way).