Skip to content

Commit df6134a

Browse files
robo-caphyder
robo-cap
authored and
hyder
committed
Add OIDC configuration support
1 parent 3d2ccd1 commit df6134a

15 files changed

+315
-14
lines changed

‎docs/src/SUMMARY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@
1212
- [Subnets](./guide/network_subnets.md)
1313
- [Network Security Groups](./guide/network_nsgs.md)
1414
- [Cluster](./guide/cluster.md)
15+
- [Cluster OIDC Authentication](./guide/cluster_oidc_authentication.md)
16+
- [Cluster OIDC Discovery](./guide/cluster_oidc_discovery.md)
1517
- [Cluster Add-ons](./guide/cluster_addons.md)
1618
- [Workers](./guide/workers.md)
1719
- [Mode](./guide/workers_mode.md)

‎docs/src/guide/cluster_addons.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Cluster Add-ons
22

3-
With this module to manage both essential and optional add-ons on **enhanced** OKE clusters.
3+
With this module you can manage both essential and optional add-ons on **enhanced** OKE clusters.
44

55
This module provides the option to remove [Essential addons](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengintroducingclusteraddons.htm#contengintroducingclusteraddons__section-essential-addons) and to manage, both essential & [optional addons](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengintroducingclusteraddons.htm#contengintroducingclusteraddons__section-optional-addons).
66

‎docs/src/guide/cluster_oidc_authentication.md

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# OpenID Connect Authentication
2+
3+
By default, OKE clusters are set up to authenticate individuals (human users, groups or service principals) accessing the API endpoint using OCI Identity and Access Management (IAM).
4+
5+
Using OKE OIDC Authentication, we can authenticate OKE API Endpoint requests (from human users or service principals) using tokens issued by third-party Identity Providers without the need for federation with OCI IAM.
6+
7+
## Prerequisites
8+
9+
Note the following prerequisites for enabling a cluster for OIDC authentication:
10+
11+
- The cluster must be an enhanced cluster. OIDC authentication is not supported for basic clusters.
12+
- The cluster must be running Kubernetes version 1.21 (or later) -- for single external OIDC IdP setup.
13+
- The cluster must be running Kubernetes version 1.30 (or later) -- for multiple external OIDC IdPs setup.
14+
15+
## Configuration
16+
17+
In addition to the implicit OCI IAM, you can configure the OKE cluster to authenticate the cluster API endpoint requests using a **single** external OIDC (OpenID Connect) Identity Provider (IdP).
18+
19+
For this is necessary to set the following variables:
20+
21+
```
22+
oidc_discovery_enabled = true
23+
oidc_token_authentication_config = {
24+
client_id = ...,
25+
issuer_url = ...,
26+
username_claim = ...,
27+
username_prefix = ...,
28+
groups_claim = ...,
29+
groups_prefix = ...,
30+
required_claims = [
31+
{
32+
key = ...,
33+
value = ...
34+
},
35+
{
36+
key = ...,
37+
value = ...
38+
}
39+
],
40+
ca_certificate = ...,
41+
signing_algorithms = []
42+
}
43+
```
44+
45+
In case you're looking to authenticate the cluster API endpoint requests with **multiple** OIDC IdPs, you can take advantage of the [authentication configuration via file Kubernetes feature](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#using-authentication-configuration).
46+
47+
```
48+
oidc_discovery_enabled = true
49+
oidc_token_authentication_config = {
50+
configuration_file = base64encode(yamlencode(
51+
{
52+
"apiVersion" = "apiserver.config.k8s.io/v1beta1"
53+
"kind" = "AuthenticationConfiguration"
54+
"jwt" = [
55+
{
56+
"issuer"= {
57+
"url" = "...",
58+
"audiences" = [
59+
"..."
60+
],
61+
"audienceMatchPolicy" = "MatchAny"
62+
}
63+
"claimMappings" = {
64+
"username" = {
65+
"claim" = "..."
66+
"prefix" = ""
67+
}
68+
}
69+
"claimValidationRules" = [
70+
{
71+
"claim" = "..."
72+
"requiredValue" = "..."
73+
}
74+
]
75+
}
76+
]
77+
}
78+
))
79+
}
80+
```
81+
82+
The authenticated users are mapped to a `User` resource in Kubernetes and you have to setup the desired RBAC polices to provide access.
83+
84+
E.g. for Github Action workflow:
85+
86+
```
87+
---
88+
apiVersion: rbac.authorization.k8s.io/v1
89+
kind: Role
90+
metadata:
91+
namespace: default
92+
name: actions-oidc-role
93+
rules:
94+
- apiGroups: [""]
95+
resources: ["pods"]
96+
verbs: ["get", "watch", "list"]
97+
- apiGroups: ["apps"]
98+
resources: ["deployments"]
99+
verbs: ["get", "watch", "list", "create", "update", "delete"]
100+
---
101+
apiVersion: rbac.authorization.k8s.io/v1
102+
kind: RoleBinding
103+
metadata:
104+
name: actions-oidc-binding
105+
namespace: default
106+
roleRef:
107+
apiGroup: rbac.authorization.k8s.io
108+
kind: Role
109+
name: actions-oidc-role
110+
subjects:
111+
- apiGroup: rbac.authorization.k8s.io
112+
kind: User
113+
name: actions-oidc:repo:GH-ACCOUNT/GH-REPO:ref:refs/heads/main
114+
```
115+
116+
**Note:**
117+
1. You need to make sure the OKE Control Plane endpoint is allowed to connect to the IdP.
118+
119+
```
120+
allow_rules_cp = {
121+
"Allow egress to anywhere HTTPS from OKE CP" : {
122+
protocol = "6", port=443, destination = "0.0.0.0/0", destination_type = "CIDR_BLOCK",
123+
}
124+
}
125+
```
126+
2. You cannot configure cluster OIDC Authentication using the arguments of the `oidc_token_authentication_config` (`client_id`, `issuer_url`, etc..) **and** the `configuration_file` at the same time.
127+
128+
## OpenID Connect Discovery
129+
130+
### Prerequisites
131+
132+
Note the following points when using OIDC Discovery:
133+
134+
- The cluster must be an enhanced cluster. OIDC Discovery is not supported for basic clusters.
135+
- The cluster must be running Kubernetes version 1.21 (or later).
136+
137+
### Configuration
138+
139+
OKE already supports Workload Identity to enable Kubernetes pods to access OCI resources, such as a secret or cloud storage bucket without storing access credentials in your Kubernetes cluster.
140+
141+
If you are looking to authorize Kubernetes pods to access non-OCI resources you can enable OKE OIDC Discovery.
142+
143+
When you enable OIDC discovery for an OKE cluster, OKE provides an OpenID Connect issuer endpoint. This endpoint serves the OIDC discovery document and the JSON web key set (JWKS), which contain the public key necessary for token validation. These resources enable third-party IdP to validate tokens issued for pods in the OKE cluster, allowing those pods to access non-OCI resources.
144+
145+
[ ![](../images/oidc-discovery.png) ](../images/oidc-discovery.png)
146+
*Figure 1: OIDC Discovery*
147+
148+
To enable the OKE OIDC Discovery, you have to set the following variable:
149+
150+
```
151+
open_id_connect_discovery_enabled = true
152+
```
153+
154+
The OpenID Connect issuer endpoint is available in the output:
155+
156+
```
157+
cluster_oidc_discovery_endpoint
158+
```
159+
160+
## Example usage
161+
162+
OIDC Authentication setup using Kubernetes API server flags
163+
164+
```javascript
165+
{{#include ../../../examples/cluster-addons/vars-cluster-oidc-auth-single.auto.tfvars:4:}}
166+
```
167+
168+
OIDC Authentication setup using Kubernetes API server configuration file
169+
170+
```javascript
171+
{{#include ../../../examples/cluster-addons/vars-cluster-oidc-auth-multiple.auto.tfvars:4:}}
172+
```
173+
174+
## Reference
175+
* [OKE OpenID Authetication](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengOpenIDConnect-Authentication.htm)
176+
* [OKE Cluster Terraform resource](https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/containerengine_cluster)
177+
* [Github workflow OKE OIDC authentication](https://docs.oracle.com/en/learn/gaw-oke-odic/index.html#introduction)
178+
* [Kubernetes OIDC Authentication setup using Kubernetes API server configuration file](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#using-authentication-configuration)
179+
* [Kubernetes OIDC Authentication setup using Kubernetes API server flags](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#using-flags)

‎docs/src/guide/cluster_oidc_discovery.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# OpenID Connect Discovery
2+
3+
With OKE OIDC Discovery, it is possible to validate Kubernetes pods running on OKE clusters with third-party STS (Security Token Service) issuers, whether on-premises or in cloud service providers (CSPs) such as Amazon Web Services (AWS) and Google Cloud Platform (GCP), and authorize them to access non-OCI resources. OKE OIDC Discovery enables this integration.
4+
5+
## Prerequisites
6+
7+
Note the following points when using OIDC Discovery:
8+
9+
- The cluster must be an enhanced cluster. OIDC Discovery is not supported for basic clusters.
10+
- The cluster must be running Kubernetes version 1.21 (or later).
11+
12+
## Configuration
13+
14+
When you enable OIDC discovery for an OKE cluster, OKE provides an OpenID Connect issuer endpoint. This endpoint serves the OIDC discovery document and the JSON web key set (JWKS), which contain the public key necessary for token validation. These resources enable third-party IdP to validate tokens issued for pods in the OKE cluster, allowing those pods to access non-OCI resources.
15+
16+
[ ![](../images/oidc-discovery.png) ](../images/oidc-discovery.png)
17+
*Figure 1: OIDC Discovery*
18+
19+
To enable the OKE OIDC Discovery, you have to set the following variable:
20+
21+
```
22+
open_id_connect_discovery_enabled = true
23+
```
24+
25+
The OpenID Connect issuer endpoint is available in the output:
26+
27+
```
28+
cluster_oidc_discovery_endpoint
29+
```
30+
31+
## Example usage
32+
33+
OIDC Discovery setup using Kubernetes API server flags
34+
35+
```javascript
36+
{{#include ../../../examples/cluster-addons/vars-cluster-oidc-discovery.auto.tfvars:4:}}
37+
```
38+
39+
## Reference
40+
* [OKE OpenID Discovery](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengOpenIDConnect-Discovery.htm)
41+
* [OKE Cluster Terraform resource](https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/containerengine_cluster)
42+
* [OKE Pods access AWS resources](https://umashankar-s.medium.com/multicloud-use-case-oke-apps-pods-accessing-aws-resources-using-openid-disovery-8e147500656f)

‎docs/src/images/oidc-discovery.png

191 KB
Loading

‎docs/src/outputs.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,10 @@
5252
<!-- BEGIN_TF_CLUSTER -->
5353

5454
* **`cluster_id`**&nbsp;&nbsp;
55-
* **`endpoints`**&nbsp;&nbsp;
55+
* **`cluster_endpoints`**&nbsp;&nbsp;
56+
* **`cluster_oidc_discovery_endpoint`**&nbsp;&nbsp;
57+
* **`cluster_kubeconfig`**&nbsp;&nbsp;
58+
* **`cluster_ca_cert`**&nbsp;&nbsp;
5659

5760
<!-- END_TF_CLUSTER -->
5861

‎examples/cluster/vars-cluster-basic.auto.tfvars

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl
33

44
cluster_name = "oke-example"
5-
kubernetes_version = "v1.26.2"
5+
kubernetes_version = "v1.31.1"

‎examples/cluster/vars-cluster-enhanced.auto.tfvars

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ cluster_type = "enhanced" // *basic/enhanced
99
cni_type = "flannel" // *flannel/npn
1010
assign_public_ip_to_control_plane = true // true/*false
1111
image_signing_keys = []
12-
kubernetes_version = "v1.26.2"
12+
kubernetes_version = "v1.31.1"
1313
pods_cidr = "10.244.0.0/16"
1414
services_cidr = "10.96.0.0/16"
1515
use_signed_images = false // true/*false

‎module-cluster.tf

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,22 @@ data "oci_containerengine_clusters" "existing_cluster" {
77
count = var.cluster_id != null ? 1 : 0
88
compartment_id = local.compartment_id
99

10-
state = ["ACTIVE","UPDATING"]
10+
state = ["ACTIVE", "UPDATING"]
1111
filter {
12-
name = "id"
12+
name = "id"
1313
values = [var.cluster_id]
1414
}
1515
}
1616

1717
data "oci_containerengine_cluster_kube_config" "public" {
18-
count = local.cluster_enabled && local.public_endpoint_available ? 1 : 0
18+
count = local.cluster_enabled && local.public_endpoint_available ? 1 : 0
1919

2020
cluster_id = local.cluster_id
2121
endpoint = "PUBLIC_ENDPOINT"
2222
}
2323

2424
data "oci_containerengine_cluster_kube_config" "private" {
25-
count = local.cluster_enabled && local.private_endpoint_available ? 1 : 0
25+
count = local.cluster_enabled && local.private_endpoint_available ? 1 : 0
2626

2727
cluster_id = local.cluster_id
2828
endpoint = "PRIVATE_ENDPOINT"
@@ -35,11 +35,11 @@ locals {
3535

3636
cluster-context = try(format("context-%s", substr(local.cluster_id, -11, -1)), "")
3737

38-
existing_cluster_endpoints = coalesce(one(flatten(data.oci_containerengine_clusters.existing_cluster[*].clusters[*].endpoints)), tomap({}))
39-
public_endpoint_available = var.cluster_id != null ? length(lookup(local.existing_cluster_endpoints, "public_endpoint", "")) > 0 : var.control_plane_is_public && var.assign_public_ip_to_control_plane
40-
private_endpoint_available = var.cluster_id != null ? length(lookup(local.existing_cluster_endpoints, "private_endpoint", "")) > 0 : true
41-
kubeconfig_public = var.control_plane_is_public ? try(yamldecode(replace(lookup(one(data.oci_containerengine_cluster_kube_config.public), "content", ""), local.cluster-context, var.cluster_name)), tomap({})) : null
42-
kubeconfig_private = try(yamldecode(replace(lookup(one(data.oci_containerengine_cluster_kube_config.private), "content", ""), local.cluster-context, var.cluster_name)), tomap({}))
38+
existing_cluster_endpoints = coalesce(one(flatten(data.oci_containerengine_clusters.existing_cluster[*].clusters[*].endpoints)), tomap({}))
39+
public_endpoint_available = var.cluster_id != null ? length(lookup(local.existing_cluster_endpoints, "public_endpoint", "")) > 0 : var.control_plane_is_public && var.assign_public_ip_to_control_plane
40+
private_endpoint_available = var.cluster_id != null ? length(lookup(local.existing_cluster_endpoints, "private_endpoint", "")) > 0 : true
41+
kubeconfig_public = var.control_plane_is_public ? try(yamldecode(replace(lookup(one(data.oci_containerengine_cluster_kube_config.public), "content", ""), local.cluster-context, var.cluster_name)), tomap({})) : null
42+
kubeconfig_private = try(yamldecode(replace(lookup(one(data.oci_containerengine_cluster_kube_config.private), "content", ""), local.cluster-context, var.cluster_name)), tomap({}))
4343

4444
kubeconfig_clusters = try(lookup(local.kubeconfig_private, "clusters", []), [])
4545
apiserver_private_host = (var.create_cluster
@@ -131,6 +131,11 @@ module "cluster" {
131131
},
132132
local.service_lb_freeform_tags,
133133
)
134+
135+
oidc_discovery_enabled = var.oidc_discovery_enabled
136+
oidc_token_auth_enabled = var.oidc_token_auth_enabled
137+
oidc_token_authentication_config = var.oidc_token_authentication_config
138+
134139
depends_on = [
135140
module.iam_cluster_prerequisites,
136141
]
@@ -146,6 +151,11 @@ output "cluster_endpoints" {
146151
value = var.create_cluster ? one(module.cluster[*].endpoints) : local.existing_cluster_endpoints
147152
}
148153

154+
output "cluster_oidc_discovery_endpoint" {
155+
description = "OIDC discovery endpoint for the OKE cluster"
156+
value = var.create_cluster && var.oidc_discovery_enabled ? one(module.cluster[*].oidc_discovery_endpoint) : null
157+
}
158+
149159
output "cluster_kubeconfig" {
150160
description = "OKE kubeconfig"
151161
value = var.output_detail ? (

‎modules/cluster/cluster.tf

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,40 @@ resource "oci_containerengine_cluster" "k8s_cluster" {
3939
}
4040

4141
options {
42+
43+
dynamic "open_id_connect_token_authentication_config" {
44+
for_each = var.oidc_token_auth_enabled ? [1] : []
45+
46+
content {
47+
is_open_id_connect_auth_enabled = var.oidc_token_auth_enabled
48+
49+
50+
issuer_url = lookup(var.oidc_token_authentication_config, "issuer_url", null)
51+
ca_certificate = lookup(var.oidc_token_authentication_config, "ca_certificate", null)
52+
client_id = lookup(var.oidc_token_authentication_config, "client_id", null)
53+
signing_algorithms = lookup(var.oidc_token_authentication_config, "signing_algorithms", null)
54+
55+
groups_claim = lookup(var.oidc_token_authentication_config, "groups_claim", null)
56+
groups_prefix = lookup(var.oidc_token_authentication_config, "groups_prefix", null)
57+
58+
username_claim = lookup(var.oidc_token_authentication_config, "username_claim", null)
59+
username_prefix = lookup(var.oidc_token_authentication_config, "username_prefix", null)
60+
61+
dynamic "required_claims" {
62+
for_each = lookup(var.oidc_token_authentication_config, "required_claims", [])
63+
content {
64+
key = lookup(required_claims.value, "key")
65+
value = lookup(required_claims.value, "value")
66+
}
67+
}
68+
configuration_file = lookup(var.oidc_token_authentication_config, "configuration_file", null)
69+
}
70+
}
71+
72+
open_id_connect_discovery {
73+
is_open_id_connect_discovery_enabled = var.oidc_discovery_enabled
74+
}
75+
4276
kubernetes_network_config {
4377
pods_cidr = var.pods_cidr
4478
services_cidr = var.services_cidr

0 commit comments

Comments
 (0)