Authorization Code Flow with Flink

[jhart0]

When using Flink in a managed environment, it's not possible to initiate the Authorization Code flow as there is no way to follow the redirect URL.

Ideally, it would be possible to provide an access token and refresh token to authmgr and it would then handle the token refresh using those credentials. Presently, only an access token can be directly provided and it does not honour a refresh token, rather it waits until the access token expires.

This is to support flows where the Flink job is created programatically by an already authenticated user and the downstream catalog needs to make authorization decisions based on a user, so client_credentials is not suitable.

Is there a way to support this workflow today, for example providing the refresh token as a header perhaps?

[adutra]

Hi @jhart0 thanks for reporting this.

I have some concerns about allowing a refresh token to be provided through configuration. I have explained this in detail here:

apache/iceberg#12362 (comment)

In short, my fear is that even if that was implemented, it wouldn't fly, because refresh tokens are generally "one-time" tokens. So, if you share a refresh token, chances are that only one party would be able to use it, and all other parties would get a 401.

Instead, I think we need to investigate different ways. Token exchange might be a solution: if the user is already authenticated when the catalog is created, then we could configure the catalog to exchange that token against a token for itself. The resulting token would "retain" information about the user. Would that work for you?

[jhart0]

Hey @adutra , thanks so much for the prompt reply.

I read your response on apache/iceberg#12362 (comment) - this seems mostly in response to the client_credentials flow. I'd be interested in using a refresh_token with the authorization_code flow per the spec. So would expect auth manager to handle storing the refresh token each time a new one is retrieved (as you say, they're one time, but we get a new one each time).

we could configure the catalog to exchange that token against a token for itself.

Could you expand on that a little please, what would that look like from an OIDC perspective?

[adutra]

Hi @jhart0 thanks for replying.

It seems that we are considering two situations: 1) the auth manager is responsible for fetching tokens, but there is a problem with the redirect URL – and 2) the auth manager wouldn't be responsible for fetching tokens, but would be given tokens from an outside component (Flink).

1) the auth manager is responsible for fetching tokens

If the auth manager fetches its own tokens, everything should be fine. This auth manager is already capable of doing a full Authorization Code flow. In this case, it manages the access token and the refresh token.

For the redirect URL issue, indeed you may need to configure DNS and firewalls. That's the Achilles heel of Authorization Code.
Have a look at the Authorization Code options, and especially this one:

https://github.com/dremio/iceberg-auth-manager/blob/main/docs/configuration.md#restauthoauth2auth-codecallback-bind-host

If none of this works, you can maybe consider the Device Code grant. It was designed specifically to overcome the redirect URL issue, and is much easier to use:

https://github.com/dremio/iceberg-auth-manager/blob/main/docs/configuration.md#device-code-settings

2) the auth manager would be given tokens, including a refresh token

That's where I think we'd have a problem. let's assume that this option exists today:

• Flink would perform the Authorization Code for user X and would acquire access token AT1 and refresh token RT1.
• Flink would pass AT1 and RT1 to the catalog then to the auth manager.
• Flink would attempt to refresh AT1 using RT1, acquires AT2 and RT2 => RT1 is now invalid
• Auth manager would attempt to refresh AT1 using RT1 => since RT1 is invalid => 401 💥

IOW this option could only work if either of the below conditions are met:

1. Flink "gives up" on the tokens it fetched and transfer ownership of the refresh token to the auth manager
2. The IDP does not invalidate refresh tokens

Unfortunately 1) is probably not feasible as Flink needs those tokens as well? And 2) goes against RFC 9700.

Wdyt? Am I missing something? Do you have any ideas how to overcome this refresh token problem?

(sorry I wrote Trino initially – fixed 😅 )

[jhart0]

Hi @adutra - thanks for the detailed consideration.

Perhaps if I describe my scenario a little more.

I have a multi-tenant SaaS application. A user would login in to this SaaS application via a web UI (using an auth code flow), they can then perform some actions in the app that might run a flink job in the background. This isn't interactive so they can't interact with an auth/device code flow. I had hoped to pass the tokens in to this job and let auth manager handle their lifecycle (they're just passing through Flink). Which works fine with an AT, but then the job can only run as long as the AT is valid, hence I'd hoped to just stick the RT in there and from auth manager's perspective it's the same as if it did an auth flow to vend the original credentials from that point. But I do also agree, this kind of falls under "vending credentials" which is not a best practice...

After some more consideration, I've decided to proceed using OPA to provide authz at the application layer, then just use client_credentials for machine to machine auth between Flink and the Catalog for now.

[adutra]

Also, when you put things like S3 request signing in the mix, the situation can become even more complex: see #132 for some insights.

Basically, worker nodes cannot get access to a user's token that easily. Again, token exchange might be one of the solutions to explore: if a user token is provided in the catalog configuration as a subject token, that subject token will get propagated to worker nodes. Then each worker node could exchange that token for a new one.