Skip to content

feat: OPA Authorizer #777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
May 21, 2025
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
88944a6
add support for authorization with OPA
labrenbe Mar 28, 2025
142f867
add docs on authorization with OPA
labrenbe Mar 28, 2025
e603d7d
update helm chart
labrenbe Mar 31, 2025
39caff2
Merge remote-tracking branch 'origin/main' into feat/opa-authorizer
labrenbe Apr 10, 2025
032ab37
rename input.user to input.identity
labrenbe Apr 15, 2025
4470e83
Merge remote-tracking branch 'origin/main' into feat/opa-authorizer
labrenbe Apr 15, 2025
6bd4831
add opa cm watch
labrenbe Apr 15, 2025
b87430b
remove unused test
labrenbe Apr 16, 2025
3c9c7c1
update rego with new opa response schema
labrenbe May 6, 2025
7dd6e38
update docs
labrenbe May 8, 2025
47f1ab5
improve docs
labrenbe May 9, 2025
150dfdd
update docs
labrenbe May 9, 2025
9b3e0d3
Merge remote-tracking branch 'origin/main' into feat/opa-authorizer
labrenbe May 9, 2025
031820b
fix docs
labrenbe May 9, 2025
13e60bc
refactor references_config_map function
labrenbe May 9, 2025
cfa6783
fix docs
labrenbe May 9, 2025
f9bcd44
wip: add integration test
labrenbe May 14, 2025
46fc511
add retries to nifi calls during test
labrenbe May 15, 2025
8f0a02f
simplify rego rules
labrenbe May 15, 2025
ca61f92
move nifi flow copying into init container
labrenbe May 15, 2025
6e66627
remove oidc test and fix oidc-opa test with tls disabled
labrenbe May 16, 2025
96d66db
Apply suggestions from code review
labrenbe May 19, 2025
3107704
improve docs
labrenbe May 19, 2025
2a9a992
require opa operator during integration tests
labrenbe May 19, 2025
cd7e5bb
Merge remote-tracking branch 'origin/main' into feat/opa-authorizer
labrenbe May 20, 2025
ca80daa
reduce flow election time
maltesander May 20, 2025
71519af
reduce verbosity, improve response handling
maltesander May 20, 2025
c26759f
linter attempt 1
maltesander May 20, 2025
860e346
remove securityContext from keycloak
labrenbe May 20, 2025
6d77992
remove securityContext from test container
labrenbe May 20, 2025
993d4d9
Apply suggestions from code review
labrenbe May 20, 2025
fff5c79
remove 'json' annotation from code snippet
labrenbe May 20, 2025
74a8243
fix docs formatting
labrenbe May 20, 2025
0e7f10b
Update docs/modules/nifi/pages/usage_guide/security.adoc
labrenbe May 21, 2025
df3d756
Merge remote-tracking branch 'origin/main' into feat/opa-authorizer
labrenbe May 21, 2025
19225a9
add caching default values to docs
labrenbe May 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion deploy/helm/nifi-operator/crds/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ spec:
description: Settings that affect all roles and role groups. The settings in the `clusterConfig` are cluster wide settings that do not need to be configurable at role or role group level.
properties:
authentication:
description: Authentication options for NiFi (required). Read more about authentication in the [security documentation](https://docs.stackable.tech/home/nightly/nifi/usage_guide/security).
description: Authentication options for NiFi (required). Read more about authentication in the [security documentation](https://docs.stackable.tech/home/nightly/nifi/usage_guide/security#_authentication).
items:
properties:
authenticationClass:
Expand All @@ -55,6 +55,42 @@ spec:
- authenticationClass
type: object
type: array
authorization:
description: Authorization options. Learn more in the [NiFi authorization usage guide](https://docs.stackable.tech/home/nightly/nifi/usage-guide/security#authorization).
nullable: true
properties:
opa:
description: Configure the OPA stacklet [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) and the name of the Rego package containing your authorization rules. Consult the [OPA authorization documentation](https://docs.stackable.tech/home/nightly/concepts/opa) to learn how to deploy Rego authorization rules with OPA.
nullable: true
properties:
cache:
default:
entryTimeToLive: 30s
maxEntries: 10000
description: Least Recently Used (LRU) cache with per-entry time-to-live (TTL) value.
properties:
entryTimeToLive:
default: 30s
description: Time to live per entry
type: string
maxEntries:
default: 10000
description: Maximum number of entries in the cache; If this threshold is reached then the least recently used item is removed.
format: uint32
minimum: 0.0
type: integer
type: object
configMapName:
description: The [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) for the OPA stacklet that should be used for authorization requests.
type: string
package:
description: The name of the Rego package containing the Rego rules for the product.
nullable: true
type: string
required:
- configMapName
type: object
type: object
createReportingTaskJob:
default:
enabled: true
Expand Down
86 changes: 85 additions & 1 deletion docs/modules/nifi/pages/usage_guide/security.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
= Security
:description: Secure Apache NiFi on Kubernetes with TLS, authentication, and authorization using the Stackable operator. Configure LDAP, OIDC, and sensitive data encryption.
:nifi-docs-authorization: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization
:nifi-docs-access-policies: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#access-policies
:nifi-docs-fileusergroupprovider: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileusergroupprovider
:nifi-docs-fileaccesspolicyprovider: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileaccesspolicyprovider
:nifi-docs-sensitive-properties-key: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#nifi_sensitive_props_key
Expand Down Expand Up @@ -168,7 +169,7 @@ stringData:

NiFi supports {nifi-docs-authorization}[multiple authorization methods], the available authorization methods depend on the chosen authentication method.

Authorization is not fully implemented by the Stackable Operator for Apache NiFi.
The Stackable Operator for Apache NiFi supports a default authorization methods for each authentication method and authorization with Open Policy Agent.

[#authorization-single-user]
=== Single user
Expand All @@ -190,6 +191,89 @@ With this authorization method, all authenticated users have administrator capab
An admin user with an auto-generated password is created that can access the NiFi API.
The password for this user is stored in a Kubernetes Secret called `<nifi-name>-oidc-admin-password`.

[#authorization-opa]
=== Open Policy Agent (OPA)

NiFi can be configured to delegate authorization decisions to an Open Policy Agent (OPA) instance. More information on the setup and configuration of OPA can be found in the xref:opa:index.adoc[OPA Operator documentation].

A NiFi cluster can be configured with OPA authorization by adding this section to the configuration:

[source,yaml]
----
spec:
clusterConfig:
authorization:
opa:
configMapName: simple-opa <1>
package: my-druid-rules <2>
cache:
entryTimeToLive: 5s <3>
maxEntries: 10 <4>
----
<1> The name of your OPA Stacklet (`simple-opa` in this case)
<2> The RegoRule package to use for policy decisions.
The package needs to contain an `allow` rule.
This is optional and defaults to the name of the Druid Stacklet.
<3> TTL for items in the cache in NiFi.
<4> Maximum number of concurrent entries in the cache in NiFi

=== Defining rego rules

For a general explanation of how rules are written, please refer to the {opa-rego-docs}[OPA documentation]. For more information on NiFi's access policies please refer to the {nifi-docs-access-policies}[NiFi documentation].

Inside of your rego rules you have access to the content of the authorization request from NiFi as input for OPA:

[source,json]
----
{
"action": {
"name": <String> <1>
},
"properties": {
"isAccessAttempt": <String>, <2>
"isAnonymous": <String> <3>
},
"resource": { <4>
"id": <String>,
"name": <String>,
"safeDescription": <String>
},
"requestedResource": { <5>
"id": <String>,
"name": <String>,
"safeDescription": <String>
},
"identity": {
"name": <String>, <6>
"groups": String, <7>
},
"resourceContext": <Object>, <8>
"userContext": <Object> <9>
}
----
<1> The action taken against the resource. Possible values: `"read"` or `"write"`.
<2> Whether this is a direct access attempt of the resource or if it's being checked as part of another response. Possible values: `"true"` or `"false"`
<3> Whether the entity accessing the resource is anonymous. Possible values: `"true"` or `"false"`.
<4> The current resource being authorized on.
<5> The original resource being requested. In cases with inherited policies, this will be an ancestor resource of the current resource. For the initial request, and cases without inheritance, the requested resource will be the same as the current resource.
<6> The identity/user accessing the resource.
<7> Comma-separated list of groups that the identity/user accessing the resource belongs to.
<8> The event attributes to make additional access decisions for provenance events. Will be `{"": ""}` if empty.
<9> Additional context for the user to make additional access decisions. Will be `{"": ""}` if empty.

The result of an 'allow' rego rule has to be

[source,json]
----
{
"allowed": <String>, <1>
"dumpCache": <Boolean> <2>
}
----
<1> Whether the action against the resource is allowed. Possible values: `"true"` or `"false"`.
<2> Whether the complete local cache in NiFi should be invalidated.


[#encrypting-sensitive-properties]
== Encrypting sensitive properties on disk

Expand Down
49 changes: 36 additions & 13 deletions rust/operator-binary/src/controller.rs
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ use crate::{
AUTHORIZERS_XML_FILE_NAME, LOGIN_IDENTITY_PROVIDERS_XML_FILE_NAME,
NifiAuthenticationConfig, STACKABLE_SERVER_TLS_DIR, STACKABLE_TLS_STORE_PASSWORD,
},
authorization::NifiAuthorizationConfig,
build_tls_volume, check_or_generate_oidc_admin_password, check_or_generate_sensitive_key,
tls::{KEYSTORE_NIFI_CONTAINER_MOUNT, KEYSTORE_VOLUME_NAME, TRUSTSTORE_VOLUME_NAME},
},
Expand Down Expand Up @@ -296,6 +297,11 @@ pub enum Error {
source: crate::security::authentication::Error,
},

#[snafu(display("Invalid NiFi Authorization Configuration"))]
InvalidNifiAuthorizationConfig {
source: crate::security::authorization::Error,
},

#[snafu(display("Failed to resolve NiFi Authentication Configuration"))]
FailedResolveNifiAuthenticationConfig {
source: crate::crd::authentication::Error,
Expand Down Expand Up @@ -455,19 +461,22 @@ pub async fn reconcile_nifi(
obj_ref: ObjectRef::new(&nifi.name_any()).within(namespace),
})?;

let nifi_authentication_config = NifiAuthenticationConfig::try_from(
let authentication_config = NifiAuthenticationConfig::try_from(
AuthenticationClassResolved::from(nifi, client)
.await
.context(FailedResolveNifiAuthenticationConfigSnafu)?,
)
.context(InvalidNifiAuthenticationConfigSnafu)?;

if let NifiAuthenticationConfig::Oidc { .. } = nifi_authentication_config {
if let NifiAuthenticationConfig::Oidc { .. } = authentication_config {
check_or_generate_oidc_admin_password(client, nifi)
.await
.context(SecuritySnafu)?;
}

let authorization_config =
NifiAuthorizationConfig::from(&nifi.spec.cluster_config.authorization);

let (rbac_sa, rbac_rolebinding) = build_rbac_resources(
nifi,
APP_NAME,
Expand Down Expand Up @@ -516,7 +525,8 @@ pub async fn reconcile_nifi(
let rg_configmap = build_node_rolegroup_config_map(
nifi,
&resolved_product_image,
&nifi_authentication_config,
&authentication_config,
&authorization_config,
role,
&rolegroup,
rolegroup_config,
Expand All @@ -541,7 +551,8 @@ pub async fn reconcile_nifi(
role,
rolegroup_config,
&merged_config,
&nifi_authentication_config,
&authentication_config,
&authorization_config,
rolling_upgrade_supported,
replicas,
&rbac_sa.name_any(),
Expand Down Expand Up @@ -591,7 +602,7 @@ pub async fn reconcile_nifi(
nifi,
&resolved_product_image,
&client.kubernetes_cluster_info,
&nifi_authentication_config,
&authentication_config,
&rbac_sa.name_any(),
)
.context(ReportingTaskSnafu)?
Expand Down Expand Up @@ -699,7 +710,8 @@ pub fn build_node_role_service(
async fn build_node_rolegroup_config_map(
nifi: &v1alpha1::NifiCluster,
resolved_product_image: &ResolvedProductImage,
nifi_auth_config: &NifiAuthenticationConfig,
authentication_config: &NifiAuthenticationConfig,
authorization_config: &NifiAuthorizationConfig,
role: &Role<NifiConfigFragment, GenericRoleConfig, JavaCommonConfig>,
rolegroup: &RoleGroupRef<v1alpha1::NifiCluster>,
rolegroup_config: &HashMap<PropertyNameKind, BTreeMap<String, String>>,
Expand All @@ -708,10 +720,14 @@ async fn build_node_rolegroup_config_map(
) -> Result<ConfigMap> {
tracing::debug!("building rolegroup configmaps");

let (login_identity_provider_xml, authorizers_xml) = nifi_auth_config
.get_auth_config()
let login_identity_provider_xml = authentication_config
.get_authentication_config()
.context(InvalidNifiAuthenticationConfigSnafu)?;

let authorizers_xml = authorization_config
.get_authorizers_config(authentication_config)
.context(InvalidNifiAuthorizationConfigSnafu)?;

let jvm_sec_props: BTreeMap<String, Option<String>> = rolegroup_config
.get(&PropertyNameKind::File(
JVM_SECURITY_PROPERTIES_FILE.to_string(),
Expand Down Expand Up @@ -761,7 +777,7 @@ async fn build_node_rolegroup_config_map(
&nifi.spec,
&merged_config.resources,
proxy_hosts,
nifi_auth_config,
authentication_config,
rolegroup_config
.get(&PropertyNameKind::File(NIFI_PROPERTIES.to_string()))
.with_context(|| ProductConfigKindNotSpecifiedSnafu {
Expand Down Expand Up @@ -870,7 +886,8 @@ async fn build_node_rolegroup_statefulset(
role: &Role<NifiConfigFragment, GenericRoleConfig, JavaCommonConfig>,
rolegroup_config: &HashMap<PropertyNameKind, BTreeMap<String, String>>,
merged_config: &NifiConfig,
nifi_auth_config: &NifiAuthenticationConfig,
authentication_config: &NifiAuthenticationConfig,
authorization_config: &NifiAuthorizationConfig,
rolling_update_supported: bool,
replicas: Option<i32>,
sa_name: &str,
Expand Down Expand Up @@ -922,12 +939,14 @@ async fn build_node_rolegroup_statefulset(
&nifi.spec.cluster_config.zookeeper_config_map_name,
));

if let NifiAuthenticationConfig::Oidc { oidc, .. } = nifi_auth_config {
if let NifiAuthenticationConfig::Oidc { oidc, .. } = authentication_config {
env_vars.extend(AuthenticationProvider::client_credentials_env_var_mounts(
oidc.client_credentials_secret_ref.clone(),
));
}

env_vars.extend(authorization_config.get_env_vars());

let node_address = format!(
"$POD_NAME.{}-node-{}.{}.svc.{}",
rolegroup_ref.cluster.name,
Expand Down Expand Up @@ -973,7 +992,11 @@ async fn build_node_rolegroup_statefulset(
]);

// This commands needs to go first, as they might set env variables needed by the templating
prepare_args.extend_from_slice(nifi_auth_config.get_additional_container_args().as_slice());
prepare_args.extend_from_slice(
authentication_config
.get_additional_container_args()
.as_slice(),
);

prepare_args.extend(vec![
"echo Templating config files".to_string(),
Expand Down Expand Up @@ -1222,7 +1245,7 @@ async fn build_node_rolegroup_statefulset(
}
}

nifi_auth_config
authentication_config
.add_volumes_and_mounts(&mut pod_builder, vec![
&mut container_prepare,
container_nifi,
Expand Down
25 changes: 24 additions & 1 deletion rust/operator-binary/src/crd/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ use stackable_operator::{
commons::{
affinity::StackableAffinity,
authentication::ClientAuthenticationDetails,
cache::UserInformationCache,
cluster_operation::ClusterOperation,
opa::OpaConfig,
product_image_selection::ProductImage,
resources::{
CpuLimitsFragment, MemoryLimitsFragment, NoRuntimeLimits, NoRuntimeLimitsFragment,
Expand Down Expand Up @@ -115,10 +117,15 @@ pub mod versioned {
#[serde(rename_all = "camelCase")]
pub struct NifiClusterConfig {
/// Authentication options for NiFi (required).
/// Read more about authentication in the [security documentation](DOCS_BASE_URL_PLACEHOLDER/nifi/usage_guide/security).
/// Read more about authentication in the [security documentation](DOCS_BASE_URL_PLACEHOLDER/nifi/usage_guide/security#_authentication).
// We don't add `#[serde(default)]` here, as we require authentication
pub authentication: Vec<ClientAuthenticationDetails>,

/// Authorization options.
/// Learn more in the [NiFi authorization usage guide](DOCS_BASE_URL_PLACEHOLDER/nifi/usage-guide/security#authorization).
#[serde(skip_serializing_if = "Option::is_none")]
pub authorization: Option<NifiAuthorization>,

/// Configuration of allowed proxies e.g. load balancers or Kubernetes Ingress. Using a proxy that is not allowed by NiFi results
/// in a failed host header check.
#[serde(default)]
Expand Down Expand Up @@ -272,6 +279,22 @@ impl v1alpha1::NifiCluster {
}
}

#[derive(Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct NifiAuthorization {
#[serde(skip_serializing_if = "Option::is_none")]
pub opa: Option<NifiOpaConfig>,
}

#[derive(Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct NifiOpaConfig {
#[serde(flatten)]
pub opa: OpaConfig,
#[serde(default)]
pub cache: UserInformationCache,
}

#[derive(Clone, Debug, Deserialize, JsonSchema, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct HostHeaderCheckConfig {
Expand Down
7 changes: 7 additions & 0 deletions rust/operator-binary/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -180,4 +180,11 @@ fn references_config_map(
};

nifi.spec.cluster_config.zookeeper_config_map_name == config_map.name_any()
|| match nifi.spec.cluster_config.authorization.to_owned() {
Some(trino_authorization) => match trino_authorization.opa {
Some(opa_config) => opa_config.opa.config_map_name == config_map.name_any(),
None => false,
},
None => false,
}
}
4 changes: 2 additions & 2 deletions rust/operator-binary/src/reporting_task/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ pub fn build_maybe_reporting_task(
nifi: &v1alpha1::NifiCluster,
resolved_product_image: &ResolvedProductImage,
cluster_info: &KubernetesClusterInfo,
nifi_auth_config: &NifiAuthenticationConfig,
authentication_config: &NifiAuthenticationConfig,
sa_name: &str,
) -> Result<Option<(Job, Service)>> {
if resolved_product_image.product_version.starts_with("1.") {
Expand All @@ -139,7 +139,7 @@ pub fn build_maybe_reporting_task(
nifi,
resolved_product_image,
cluster_info,
nifi_auth_config,
authentication_config,
sa_name,
)?,
build_reporting_task_service(nifi, resolved_product_image)?,
Expand Down
Loading
Loading