Skip to content

Commit 6529f52

Browse files
Malewaresiegfriedweberadwk67
authored
OpenID Connect Support (#524)
* First parts of airflow OIDC * Add integration test for OIDC * Document OIDC support * Update wip * Adapt module tests in operator-binary/src/config.rs * Adding unit tests * WIP before merge * WIP compiling operator * Fixng auth tests * Update the OIDC integration test * Env vars oidc * Cleaning up * Adding changelog * Remove reference * Regenerate charts and Nix files * making pre-commit happy * fixing typo * again pre-commit * pre-commit nr3 * pre-commit nr4 * pre-commit nr5 * Env var set by operator rather then envoverrides * Removing TODO, sort toml alphabetically * Andrew comments Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com> * Specify every arm rather then just matching all --------- Co-authored-by: Siegfried Weber <mail@siegfriedweber.net> Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com>
1 parent 1dd21f6 commit 6529f52

32 files changed

+2100
-231
lines changed

‎CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
Use the env var `KUBERNETES_CLUSTER_DOMAIN` or the operator Helm chart property `kubernetesClusterDomain` to set a non-default cluster domain ([#518]).
1010
- Support for `2.9.3` ([#494]).
1111
- Experimental Support for `2.10.2` ([#512]).
12+
- Add support for OpenID Connect ([#524])
1213

1314
### Changed
1415

@@ -30,6 +31,7 @@
3031
[#494]: https://github.com/stackabletech/airflow-operator/pull/494
3132
[#518]: https://github.com/stackabletech/airflow-operator/pull/518
3233
[#520]: https://github.com/stackabletech/airflow-operator/pull/520
34+
[#524]: https://github.com/stackabletech/airflow-operator/pull/524
3335

3436
## [24.7.0] - 2024-07-24
3537

‎Cargo.lock

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.nix

Lines changed: 24 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ built = { version = "0.7", features = ["chrono", "git2"] }
1515
clap = "4.5"
1616
fnv = "1.0"
1717
futures = { version = "0.3", features = ["compat"] }
18+
indoc = "2.0"
1819
product-config = { git = "https://github.com/stackabletech/product-config.git", tag = "0.7.0" }
1920
rstest = "0.23"
2021
semver = "1.0"

‎deploy/helm/airflow-operator/crds/crds.yaml

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -452,13 +452,27 @@ spec:
452452
properties:
453453
authentication:
454454
default: []
455-
description: The Airflow [authentication](https://docs.stackable.tech/home/nightly/airflow/usage-guide/security.html) settings. Currently the underlying Flask App Builder only supports one authentication mechanism at a time. This means the operator will error out if multiple references to an AuthenticationClass are provided.
456455
items:
457456
properties:
458457
authenticationClass:
459-
description: Name of the [AuthenticationClass](https://docs.stackable.tech/home/nightly/concepts/authentication.html#authenticationclass) used to authenticate the users. At the moment only LDAP is supported. If not specified the default authentication (AUTH_DB) will be used.
460-
nullable: true
458+
description: Name of the [AuthenticationClass](https://docs.stackable.tech/home/nightly/concepts/authentication) used to authenticate users.
461459
type: string
460+
oidc:
461+
description: This field contains OIDC-specific configuration. It is only required in case OIDC is used.
462+
nullable: true
463+
properties:
464+
clientCredentialsSecret:
465+
description: A reference to the OIDC client credentials secret. The secret contains the client id and secret.
466+
type: string
467+
extraScopes:
468+
default: []
469+
description: An optional list of extra scopes which get merged with the scopes defined in the [`AuthenticationClass`].
470+
items:
471+
type: string
472+
type: array
473+
required:
474+
- clientCredentialsSecret
475+
type: object
462476
syncRolesAt:
463477
default: Registration
464478
description: If we should replace ALL the user's roles each login, or only on registration. Gets mapped to `AUTH_ROLES_SYNC_AT_LOGIN`
@@ -474,6 +488,8 @@ spec:
474488
default: Public
475489
description: This role will be given in addition to any AUTH_ROLES_MAPPING. Gets mapped to `AUTH_USER_REGISTRATION_ROLE`
476490
type: string
491+
required:
492+
- authenticationClass
477493
type: object
478494
type: array
479495
credentialsSecret:

‎docs/modules/airflow/pages/usage-guide/security.adoc

Lines changed: 109 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,21 @@
11
= Security
2-
:description: Secure Apache Airflow by configuring user authentication and authorization, either with built-in methods or LDAP.
2+
:description: Secure Apache Airflow by configuring user authentication and authorization.
33
:airflow-access-control-docs: https://airflow.apache.org/docs/apache-airflow/stable/security/access-control.html
4+
:keycloak: https://www.keycloak.org/
45

56
Secure Apache Airflow by configuring user authentication and authorization.
6-
Airflow provides built-in user and role management, but can also connect to a LDAP server to manage users centrally instead.
7+
Airflow provides built-in user and role management, but can also connect to an LDAP server or an OIDC provider to manage users centrally instead.
78

89
== Authentication
910

10-
Users need to authenticate themselves before using Airflow, and there are two ways to configure users:
11-
The built-in user management or LDAP.
11+
Users need to authenticate themselves before using Airflow, and there are several ways in which this can be set up.
12+

13+
[IMPORTANT]
14+
.Multiple authentication methods
15+
====
16+
Only one authentication method is supported at a time, and in case of LDAP, only one authentication class is allowed.
17+
This means, it is not possible to configure both LDAP and OIDC authentication methods at the same time, but *it is* possible to configure multiple OIDC classes *or* one LDAP authentication class.
18+
====
1219

1320
=== Built-in user management
1421

@@ -19,7 +26,7 @@ image::airflow_security.png[Airflow Security menu]
1926

2027
=== LDAP
2128

22-
Airflow supports xref:concepts:authentication.adoc[user authentication] via LDAP.
29+
Airflow supports xref:concepts:authentication.adoc[user authentication] against a single LDAP server.
2330
Set up an AuthenticationClass for the LDAP server and reference it in the Airflow Stacklet resource as shown:
2431

2532
[source,yaml]
@@ -30,7 +37,7 @@ metadata:
3037
name: airflow-with-ldap
3138
spec:
3239
image:
33-
productVersion: 2.9.3
40+
productVersion: 2.10.2
3441
clusterConfig:
3542
authentication:
3643
- authenticationClass: ldap # <1>
@@ -48,6 +55,79 @@ The users and roles can be viewed as before in the Webserver UI, but the blue "+
4855

4956
image::airflow_security_ldap.png[Airflow Security menu]
5057

58+
=== [[oidc]]OpenID Connect
59+
60+
An OpenID Connect provider can be used for authentication.
61+
Unfortunately, there is no generic support for OpenID Connect built into Airflow.
62+
This means that only specific OpenID Connect providers can be configured.
63+
64+
IMPORTANT: Airflow deployments on the Stackable Data Platform only support {keycloak}[Keycloak].
65+
66+
[source,yaml]
67+
----
68+
apiVersion: airflow.stackable.tech/v1alpha1
69+
kind: AirflowCluster
70+
metadata:
71+
name: airflow-with-oidc
72+
spec:
73+
image:
74+
productVersion: 2.10.2
75+
clusterConfig:
76+
authentication:
77+
- authenticationClass: keycloak # <1>
78+
oidc:
79+
clientCredentialsSecret: airflow-keycloak-client # <2>
80+
userRegistrationRole: User # <3>
81+
----
82+
83+
<1> The reference to an AuthenticationClass called `keycloak`
84+
<2> The reference to the Secret containing the Airflow client credentials
85+
<3> The default role to which all users are assigned
86+
87+
Users that log in with OpenID Connect are assigned to a default {airflow-access-control-docs}[role] which is specified with the `userRegistrationRole` property.
88+
89+
The Secret containing the Airflow client credentials:
90+
91+
[source,yaml]
92+
----
93+
apiVersion: v1
94+
kind: Secret
95+
metadata:
96+
name: airflow-keycloak-client
97+
stringData:
98+
clientId: airflow # <1>
99+
clientSecret: airflow_client_secret # <2>
100+
----
101+
102+
<1> The client ID of Airflow as defined in Keycloak
103+
<2> The client secret as defined in Keycloak
104+
105+
A minimum client configuration in Keycloak for this example looks like this:
106+
107+
[source,json]
108+
----
109+
{
110+
"clientId": "airflow",
111+
"enabled": true,
112+
"clientAuthenticatorType": "client-secret", # <1>
113+
"secret": "airflow_client_secret",
114+
"redirectUris": [
115+
"*"
116+
],
117+
"webOrigins": [
118+
"*"
119+
],
120+
"standardFlowEnabled": true, # <2>
121+
"protocol": "openid-connect" # <3>
122+
}
123+
----
124+
125+
<1> Sets the OIDC type to confidential access type.
126+
<2> Enables the OAuth2 "Authorization Code Flow".
127+
<3> Enables OpenID Connect and OAuth2 support.
128+
129+
Further information for specifying an AuthenticationClass for an OIDC provider can be found at the xref:concepts:authentication.adoc#_oidc[concepts page].
130+
51131
== Authorization
52132
The Airflow Webserver delegates the {airflow-access-control-docs}[handling of user access control] to https://flask-appbuilder.readthedocs.io/en/latest/security.html[Flask AppBuilder].
53133

@@ -74,3 +154,26 @@ spec:
74154

75155
<1> The reference to an AuthenticationClass called `ldap`
76156
<2> All users are assigned to the `Admin` role
157+
158+
=== OpenID Connect
159+
160+
The mechanism for assigning roles to users described in the LDAP section also applies to OpenID Connect.
161+
Airflow supports assigning {airflow-access-control-docs}[Roles] to users based on their OpenID Connect scopes, though this is not yet supported by the Stackable operator.
162+
All the users logging in via OpenID Connect get assigned to the same role which you can configure via the attribute `authentication[*].userRegistrationRole` on the `AirflowCluster` object:
163+
164+
[source,yaml]
165+
----
166+
apiVersion: airflow.stackable.tech/v1alpha1
167+
kind: AirflowCluster
168+
metadata:
169+
name: airflow-with-oidc
170+
spec:
171+
clusterConfig:
172+
authentication:
173+
- authenticationClass: keycloak
174+
oidc:
175+
clientCredentialsSecret: airflow-keycloak-client
176+
userRegistrationRole: Admin # <1>
177+
----
178+
179+
<1> All users are assigned to the `Admin` role

‎rust/crd/Cargo.toml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@ repository.workspace = true
99
publish = false
1010

1111
[dependencies]
12+
indoc.workspace = true
13+
product-config.workspace = true
1214
serde.workspace = true
1315
serde_json.workspace = true
1416
snafu.workspace = true
1517
stackable-operator.workspace = true
16-
product-config.workspace = true
1718
strum.workspace = true
19+
tokio.workspace = true
1820
tracing.workspace = true
1921

2022
[dev-dependencies]

0 commit comments

Comments
 (0)