[Feature] Add support for async OAuth token refreshes

[renaudhartert-db]

What changes are proposed in this pull request?

This PR aims at eliminating long-tail latency due to OAuth token refreshes in scenarios where a single client is responsible for a relatively high (e.g. > 1 QPS) continuous outbound traffic. The feature is disabled by default — which arguably makes this PR a functional no-op.

Precisely, the PR introduces a new token cache which attempts to always keep its token fresh by asynchronously refreshing the token before it expires. We differentiate three token states:

• fresh: The token is valid and is not close to its expiration.
• stale: The token is valid but will expire soon.
• expired: The token has expired and cannot be used.

Each time a request tries to access the token, we do the following:

• If the token is fresh, return the current token;
• If the token is stale, trigger an asynchronous refresh and return the current token;
• If the token is expired, make a blocking refresh call to update the token and return it.

In particular, asynchronous refreshes use a lock to guarantee that there can only be one pending refresh at a given time.

The performance of the algorithm depends on the length of the stale and fresh periods. On the first hand, the stale period must be long enough to prevent tokens from entering the expired state. On the other hand, a long stale period reduces the length of the fresh period, thus increasing the refresh frequency.

Right now, the stale period is configured to 3 minutes by default (i.e. 5% of the expected token lifespan of 1 hour). This value might be changed in the future to guarantee that the default behavior achieves the best performance for the majority of users.

For reviewers:

• This PR only uses the new cache in control-plane auth flows; I plan to send a follow-up PR to enable asynchronous refresh in data-plane flows once this one has been merged.
• Interface oauth2.TokenSource is likely not sufficient for us and we would need one with a Token method that takes a context.Context as parameter: Token(context.Context) (Token, error).

How is this tested?

Complete test coverage with a focus on various concurrency scenarios.

[pietern]

The logic is clear now (and simpler with one less function and less lock acquisition). @denik Could you take an async (LOL) look at the PR as well?

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

• PR number: 1135
• Commit SHA: f1159eaab87010aa4ab2d534f7ac47f034970691

Checks will be approved automatically on success.