Context parameters in UDPA URIs embedded in server returned resources

[htuch]

The treatments of context parameters in xDS transport next steps indicates that client context parameters, e.g. node metadata, are unconditionally included in URIs placed in discovery requests.

This makes sense for URIs that originate at the client, e.g. those found in a bootstrap file, but not so much for URIs provided in server returned resources, e.g. a RDS URI in an LDS response. In the server returned URI case, we should not force the client to have to include potentially irrelevant metadata. There are multiple rationales for this:

• This reduces the amount of noise in and size of URIs when the node metadata is irrelevant.
• Caching can be improved by not having to consider node metadata for resources where it's immaterial.
• This matches a RESTful style of resource management, which provides a well-understood division of responsibility.
• The server can always add the context parameters to URIs in returned resources based on the resources' URI context parameters if needed.

The only potential downside is that the server may need to do some additional work in generating embedded URIs when it is necessary to propagate node metadata context parameters to resource dependencies.

I'd like to propose an addendum to xDS transport next steps where we limit per-node client capabilities (and possibly client feature capabilities) to URIs that originate on the client.

@markdroth @louiscryan @howardjohn @costinm @mattklein123 @envoyproxy/udpa-wg

[markdroth]

I totally agree with doing this for the context params that originate in the node.

The ones for the client capabilities might be important, though. Is the server in the position to make the choice about whether those parameters are important? In particular, in the case where the server is telling the client to go to a different server for the resource, will the server sending the URI know whether the server it is pointing to cares about those capabilities?

[htuch]

Yeah, I think client capabilities might need to be set by the client; in particular when they are resource specific, e.g. the client sets specific capabilities based on the type URL being requested.

[mattklein123]

The only potential downside is that the server may need to do some additional work in generating embedded URIs when it is necessary to propagate node metadata context parameters to resource dependencies.

Admittedly I have not thought about it that much, but this seems like potentially a big downside that would make management servers more stateful and have to be aware of more business logic in some cases (particularly the relay case). Why would we not allow this to be client configurable? I can see different deployments wanting different options here and I don't really see any downside to allowing a client to resend metadata if it would make the deployment simpler?

[htuch]

I think in the case of the relay server, this would be propagated all the way across the relay server. The origin server would be responsible for deciding whether or not to embed the node context parameters in the returned resource.

I think there's two things worth discussion here:

1. Does this make things more stateful? I'd argue no, since you should be able to make the decision on whether to reflect the context parameters in a returned resource based purely on inputs from a given resource request, so purely functional.
2. Does this make the origin server more complicated? I'd argue yes, since there is this additional context parameter reflection logic. OTOH, origin servers for resources need to have smarts and awareness around context parameters anyway, so it's probably not that much more complicated.

The only downside of making this client configurable is that there is potential for mismatch between client and server behaviors, e.g. if the server opts not to reflect context parameters, or picks different context parameter values to reflect back (e.g. to alias different node IDs to a single project ID), then there is possibility for confusion. But, I think the context parameter precedence rules protect us here.

[htuch]

Something else just occurred to me (while implementing a module to compute context params). Since under this proposal, the only context params that ever include node information from the client effectively come from the roots specified in the bootstrap (LDS, CDS), does it even make sense to have Envoy (or gRPC) be in the business of transferring node info to the generated URL? Why not have the author of the LDS/CDS URL include things like node ID or metadata items directly?

I can see the value of https://docs.google.com/document/d/1zZav-IYxMO0mP2A7y5XHBa9v0eXyirw1CxrLLSUqWR8/edit?ts=5ec43456#heading=h.wdyjyh724jol from the perspective of pure convenience or DRY, but that's it, there's no functional need.

[markdroth]

I would personally be fine with just including these params in the LDS URIs.

As we've discussed elsewhere, in the long run, I'd still like to see us move most use-cases toward a mode where we figure out what CDS resources to fetch dynamically based on the RouteConfiguration, rather than proactively fetching all clusters at startup. If we do that, then DRY is less of an issue, because the context params would exist only in the LDS URIs.

[mattklein123]

I don't have a super strong opinion on this one way or the other. I think the nice thing about not embedding in URI is it makes config inspection/generation slightly easier in certain cases, though, as long as the URI format is well specified (which it is) that is not a very strong argument.

I still think that there is no real need to have the protocol limit what the client sends in terms of context params, since different deployments have different needs and it doesn't really change the protocol itself, but I don't feel that strongly about this either and I would be fine seeing how it goes. I think this could always be made configurable later.