Skip to content

Commit 3d2b687

Browse files
razvanadwk67sbernauerlfranckesiegfriedweber
authored
feat: add support for 2.6.0 and OPA authorizer (#506)
* feat(test): make it easy to test custom images * revert some tests to hbase-latest * fix indentation * fix access-hbase container image * wip: temp test setup * fix yaml lint * fix opa pull policy * use AllowAccessController * Added custom logging to the kerberos test * Set log4j2 properties in the kerberos test * Use log4j2 properties starting with HBase 2.6 * Fix clippy errors * fix kerberos test * separated opa from kerberos tests * restore kerberos tests from main * readd custom image support to the kerberos tests * reduce opa test dimensions * example rego rules and test * use 2.6.0 sandbox image for tests * working integration test * update profiler integration test * fix logging test * update krb5 image for opa tests * fix snapshot tests * fix yaml lint * working rego rules test * Operator adds OPA authorizer props * regenerate charts * adapted tests to addition of scanner checks * Update OPA test with authorization props * replace realm dimension with CLUSTER.LOCAL * add docs * Update changelog and test-definition.yaml * Minor doc tweaks * docs: separate Rego examples from authorizer configuration * fix yaml lint * temporarily use 2.6 for most tests * Use Hbase 2.6 built in Prometheus exporter. * Update CHANGELOG.md Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de> * Cleanup * fix markdown lint * temp: ignore metrics port * Revert "temp: ignore metrics port" This reverts commit 4f89ebf. * Remove dedicated metrics port for 2.6 * Fix typo * Update monitoring docs * Move the function closer to where it's actually used. * Validate custom resource * regenerate charts * Add resource type to OPA rules. * Update op-rs branch * Add support for 2.4.18 * Update changelog * Update changelog * test 2.4.18 and 2.6.0 at the same time * Fix the log4j2 filename * cleanups * Update Cargo.lock * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/partials/supported-versions.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * review feedback * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> * Update docs/modules/hbase/pages/usage-guide/security.adoc Co-authored-by: Lars Francke <git@lars-francke.de> --------- Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.de> Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de> Co-authored-by: Lars Francke <git@lars-francke.de> Co-authored-by: Siegfried Weber <mail@siegfriedweber.net>
1 parent ff4e276 commit 3d2b687

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+2140
-113
lines changed

‎CHANGELOG.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,23 @@
22

33
## [Unreleased]
44

5+
### Added
6+
7+
- Added support for HBase 2.6.0 with the following changes ([#506]):
8+
- Added `clusterConfig.authorization` property to support the OPA authorizer
9+
- Configure log4j2 properties
10+
- Use built-in prometheus metric exporter
11+
- Added support for HBase 2.4.18 ([#523])
12+
513
### Changed
614

715
- Bump `stackable-operator` from `0.64.0` to `0.70.0` ([#524]).
816
- Bump `product-config` from `0.6.0` to `0.7.0` ([#524]).
917

18+
[#506]: https://github.com/stackabletech/hbase-operator/pull/506
19+
[#523]: https://github.com/stackabletech/hbase-operator/pull/523
20+
[#524]: https://github.com/stackabletech/hbase-operator/pull/524
21+
1022
## [24.3.0] - 2024-03-20
1123

1224
### Added
@@ -37,7 +49,6 @@
3749
[#441]: https://github.com/stackabletech/hbase-operator/pull/441
3850
[#454]: https://github.com/stackabletech/hbase-operator/pull/454
3951
[#511]: https://github.com/stackabletech/hbase-operator/pull/511
40-
[#524]: https://github.com/stackabletech/hbase-operator/pull/524
4152

4253
## [23.11.0] - 2023-11-24
4354

‎Cargo.lock

Lines changed: 8 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,5 @@ strum = { version = "0.26", features = ["derive"] }
2727
tokio = { version = "1.37", features = ["full"] }
2828
tracing = "0.1"
2929

30-
# [patch."https://github.com/stackabletech/operator-rs.git"]
31-
# stackable-operator = { git = "https://github.com/stackabletech//operator-rs.git", branch = "main" }
30+
#[patch."https://github.com/stackabletech/operator-rs.git"]
31+
#stackable-operator = { git = "https://github.com/stackabletech//operator-rs.git", branch = "main" }

‎deploy/helm/hbase-operator/crds/crds.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,25 @@ spec:
5151
required:
5252
- kerberos
5353
type: object
54+
authorization:
55+
nullable: true
56+
properties:
57+
opa:
58+
description: Configure the OPA stacklet [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) and the name of the Rego package containing your authorization rules. Consult the [OPA authorization documentation](https://docs.stackable.tech/home/nightly/concepts/opa) to learn how to deploy Rego authorization rules with OPA.
59+
properties:
60+
configMapName:
61+
description: The [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) for the OPA stacklet that should be used for authorization requests.
62+
type: string
63+
package:
64+
description: The name of the Rego package containing the Rego rules for the product.
65+
nullable: true
66+
type: string
67+
required:
68+
- configMapName
69+
type: object
70+
required:
71+
- opa
72+
type: object
5473
hdfsConfigMapName:
5574
description: Name of the [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) for an HDFS cluster.
5675
type: string

‎docs/modules/hbase/examples/getting_started/getting_started.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ version() {
131131
echo "Check cluster version..."
132132
cluster_version=$(version | jq -r '.Version')
133133

134-
if [ "$cluster_version" == "2.4.17" ]; then
134+
if [ "$cluster_version" == "2.4.18" ]; then
135135
echo "Cluster version: $cluster_version"
136136
else
137137
echo "Unexpected version: $cluster_version"

‎docs/modules/hbase/examples/getting_started/getting_started.sh.j2

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ version() {
131131
echo "Check cluster version..."
132132
cluster_version=$(version | jq -r '.Version')
133133

134-
if [ "$cluster_version" == "2.4.17" ]; then
134+
if [ "$cluster_version" == "2.4.18" ]; then
135135
echo "Cluster version: $cluster_version"
136136
else
137137
echo "Unexpected version: $cluster_version"

‎docs/modules/hbase/examples/getting_started/hbase.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
name: simple-hbase
66
spec:
77
image:
8-
productVersion: 2.4.17
8+
productVersion: 2.4.18
99
clusterConfig:
1010
hdfsConfigMapName: simple-hdfs
1111
zookeeperConfigMapName: simple-hbase-znode

‎docs/modules/hbase/examples/getting_started/hbase.yaml.j2

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
name: simple-hbase
66
spec:
77
image:
8-
productVersion: 2.4.17
8+
productVersion: 2.4.18
99
clusterConfig:
1010
hdfsConfigMapName: simple-hdfs
1111
zookeeperConfigMapName: simple-hbase-znode

‎docs/modules/hbase/examples/rego/hbase.rego

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
package hbase
2+
3+
import rego.v1
4+
5+
default allow := false
6+
default matches_identity(identity) := false
7+
8+
# table is null if the request is for namespace permissions, but as parameters cannot be
9+
# undefined, we have to set it to something specific:
10+
checked_table_name := input.table.qualifierAsString if {input.table.qualifierAsString}
11+
checked_table_name := "__undefined__" if {not input.table.qualifierAsString}
12+
13+
allow if {
14+
some acl in acls
15+
matches_identity(acl.identity)
16+
matches_resource(input.namespace, checked_table_name, acl.resource)
17+
action_sufficient_for_operation(acl.action, input.action)
18+
}
19+
20+
# Identity mentions the (long) userName explicitly
21+
matches_identity(identity) if {
22+
identity in {
23+
concat("", ["user:", input.callerUgi.userName])
24+
}
25+
}
26+
27+
# Identity regex matches the (long) userName
28+
matches_identity(identity) if {
29+
match_entire(identity, concat("", ["userRegex:", input.callerUgi.userName]))
30+
}
31+
32+
# Identity mentions group the user is part of (by looking up using the (long) userName)
33+
matches_identity(identity) if {
34+
some group in groups_for_user[input.callerUgi.userName]
35+
identity == concat("", ["group:", group])
36+
}
37+
38+
# Allow all resources
39+
matches_resource(namespace, table, resource) if {
40+
resource == "hbase:"
41+
}
42+
43+
# Allow all namespaces
44+
matches_resource(namespace, table, resource) if {
45+
resource == "hbase:namespace:"
46+
}
47+
48+
# Resource mentions the namespace explicitly
49+
matches_resource(namespace, table, resource) if {
50+
resource == concat(":", ["hbase:namespace", namespace])
51+
}
52+
53+
# Resource mentions the namespaced table explicitly
54+
matches_resource(namespace, table, resource) if {
55+
resource == concat("", ["hbase:table:", namespace, "/", table])
56+
}
57+
58+
match_entire(pattern, value) if {
59+
# Add the anchors ^ and $
60+
pattern_with_anchors := concat("", ["^", pattern, "$"])
61+
62+
regex.match(pattern_with_anchors, value)
63+
}
64+
65+
action_sufficient_for_operation(action, operation) if {
66+
action_hierarchy[action][_] == action_for_operation[operation]
67+
}
68+
69+
action_hierarchy := {
70+
"full": ["full", "rw", "ro"],
71+
"rw": ["rw", "ro"],
72+
"ro": ["ro"],
73+
}
74+
75+
action_for_operation := {
76+
"ADMIN": "full",
77+
"CREATE": "full",
78+
"WRITE": "rw",
79+
"READ": "ro",
80+
}
81+
82+
groups_for_user := {
83+
"hbase/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": ["admins"],
84+
"testuser/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": ["admins"],
85+
"admin/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": ["admins"],
86+
"alice/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": ["developers"],
87+
"readonlyuser1/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": [],
88+
"readonlyuser2/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": [],
89+
"bob/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL": []
90+
}
91+
92+
acls := [
93+
{
94+
"identity": "group:admins",
95+
"action": "full",
96+
"resource": "hbase:",
97+
},
98+
{
99+
"identity": "group:developers",
100+
"action": "full",
101+
"resource": "hbase:namespace:developers",
102+
},
103+
{
104+
"identity": "user:alice/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL",
105+
"action": "rw",
106+
"resource": "hbase:table:developers/table2",
107+
},
108+
{
109+
"identity": "user:bob/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL",
110+
"action": "rw",
111+
"resource": "hbase:table:developers/table1",
112+
},
113+
{
114+
"identity": "user:bob/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL",
115+
"action": "rw",
116+
"resource": "hbase:table:public/table3",
117+
},
118+
{
119+
"identity": "user:readonlyuser1/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL",
120+
"action": "ro",
121+
"resource": "hbase:table:public/test",
122+
},
123+
{
124+
"identity": "user:readonlyuser2/test-hbase-permissions.default.svc.cluster.local@CLUSTER.LOCAL",
125+
"action": "ro",
126+
"resource": "hbase:namespace:",
127+
},
128+
]

0 commit comments

Comments
 (0)