Main changes since 0.45.0
This release contains the following new features and improvements:
- Add support for Kafka 4.0.0.
Remove support for Kafka 3.8.0 and 3.8.1. - Support for ZooKeeper-based Apache Kafka clusters and for KRaft migration has been removed
- Support for MirrorMaker 1 has been removed
- Support for storage class overrides has been removed
- Added support to configure
dnsPolicy
anddnsConfig
using thetemplate
sections. - Store Kafka node certificates in separate Secrets, one Secret per pod.
- Allow configuring
ssl.principal.mapping.rules
and custom trusted CAs in Kafka brokers withtype: custom
authentication - Moved HTTP bridge configuration to the ConfigMap setup by the operator.
- Dependency updates (Vert.x 4.5.14, Netty 4.1.118.Final)
- Moved Kafka Connect configuration to the ConfigMap created by the operator.
- Update Kafka Exporter to 1.9.0
- Adopted new Kafka Connect health check endpoint (see proposal 89).
- Update standalone User Operator to handle Cluster CA cert Secret being missing when TLS is not needed.
- Strimzi Drain Cleaner updated to 1.3.0 (included in the Strimzi installation files)
- Implicit IPv4 preference when enabling JMX has been removed, and will now use JVM defaults.
This will make the cluster boot up correctly in IPv6 only environments, where IPv4 preference will break it due to lack of IPv4 addresses. - Improved the MirrorMaker2 example Grafana dashboard to set metric units and include chart descriptions.
- The
ContinueReconciliationOnManualRollingUpdateFailure
feature gate moves to GA stage and is permanently enabled without the possibility to disable it. - Update OAuth library to 0.16.2.
- Update HTTP bridge to 0.32.0.
- Kubernetes events emitted during a Pod restart updated to have the Kafka resource as the
regardingObject
and the Pod in therelated
field.
All changes can be found under the 0.46.0 milestone.
Watch out also for some notable changes, deprecations, and removals:
- Support for ZooKeeper-based clusters and for migration from ZooKeeper-based clusters to KRaft has been removed.
Please make sure all your clusters are using KRaft before upgrading to Strimzi 0.46.0 or newer! - Support for MirrorMaker 1 has been removed.
Please make sure to migrate to MirrorMaker 2 before upgrading to Strimzi 0.46 or newer. - Strimzi EnvVar Configuration Provider (deprecated in Strimzi 0.38.0) and Strimzi MirrorMaker 2 Extensions (deprecated in Strimzi 0.28.0) plugins were removed from Strimzi container images.
Please use the Apache Kafka EnvVarConfigProvider and Identity Replication Policy instead. - When using Kafka Connect or Kafka MirrorMaker 2 operands and upgrading from Strimzi 0.38 or older, make sure the
StableConnectIdentities
feature gate is enabled andStrimziPodSets
are used before upgrading. - When using the Kafka operand and upgrading from Strimzi 0.34 or older, make sure the
UseStrimziPodSets
feature gate is enabled andStrimziPodSet
resources are used before upgrading. - The storage overrides for configuring per-broker storage class are not supported anymore.
If you are using the storage overrides, you should instead use multiple KafkaNodePool resources with a different storage class each.
For more details about migrating from storage overrides, please follow the documentation. - The Open Policy Agent authorization (
type: opa
) has been deprecated and will be removed in the future.
To use the Open Policy Agent authorizer, you can use thetype: custom
authorization. - Removed the
statefulset.kubernetes.io/pod-name
label from pods and external listeners Kubernetes Services.- If you have any custom setup leveraging such label, please use the
strimzi.io/pod-name
one instead.
- If you have any custom setup leveraging such label, please use the
- The
secrets
list for mounting additional Kubernetes Secrets intype: custom
authentication was deprecated and will be removed in the future.
Please use the template section to configure additional volumes instead. - Kafka 4.0 and newer is using Log4j2 for logging instead of Reload4j/Log4j1.
If you have any custom logging configuration, you might need to update it during the upgrade to Kafka 4.0. - Kubernetes events for Pod restarts no longer have the Pod as the
regardingObject
.
If you are usingregardingObject
as afield-selector
for listing events you must update the selector to specify the Kafka resource instead. - From Kafka 4.0.0, to enable the JMXReporter you must either enable metrics in
.spec.kafka.metrics
, or explicitly add JMXReporter inmetric.reporters
.
Upgrading from Strimzi 0.45.0
See the documentation for upgrade instructions.
Upgrading from Strimzi 0.22 or earlier
Direct upgrade from Strimzi 0.22 or earlier is not supported anymore! You have to upgrade first to one of the previous versions of Strimzi. You will also need to convert the CRD resources. For more details, see the documentation.
Container images
The following container images are part of this release:
Name | Image |
---|---|
Operators | quay.io/strimzi/operator@sha256:ac434a48ac2b27b2b8f36001814897649c0193a78eb76d7cb4719c7f8cb4b275 |
Apache Kafka 3.9.0 | quay.io/strimzi/kafka@sha256:3c976468f590b464bd224fdf0adb45556fccfba6025bc6b770e3fa69f7398b04 |
Apache Kafka 4.0.0 | quay.io/strimzi/kafka@sha256:e0a017c707e101d373d992061e4531db22da7a80bac1a5af9ad8c2ffd604c41d |
Strimzi Bridge | quay.io/strimzi/kafka-bridge@sha256:3e1815aed3f74b8573cebfced66b2c78ca0cf7c746c26db2002380cd39d09ae6 |
Kaniko executor | quay.io/strimzi/kaniko-executor@sha256:ef1a39c363e145041d80103c3c12da9429ce06cf21dff6fb1fb75d0c0ed9c35b |
Maven Builder | quay.io/strimzi/maven-builder@sha256:b532234c234fc30fecec992ad1a6154c5ef08ebf9a20ec8bc62e0c88d791ff36 |
New Contributors
- @gertschouten made their first contribution in #10949
- @ItaiFonio made their first contribution in #11043
- @HirokiSakonju made their first contribution in #11164
- @pnagy-cldr made their first contribution in #11204
- @darwin67 made their first contribution in #11247
- @shk3 made their first contribution in #11263
- @Wilya3 made their first contribution in #11314
- @pboers1988 made their first contribution in #11337
- @dongjinleekr made their first contribution in #11356
What's Changed
- Bump the main branch to Strimzi 0.46.0-SNAPSHOT by @ppatierno in #10920
- Update Prometheus JMX Exporter to 1.1.0 by @scholzj in #10917
- Update Fabric8 Kubernetes Client to 7.0.0 by @scholzj in #10907
- Stop including MirrorMaker2 Extensions and EnvVarConfigProvider by @scholzj in #10923
- Remove Secret from ClusterCa generate cert methods by @katheris in #10915
- Remove unused certs method in Ca.java by @katheris in #10927
- Remove Vertx from KafkaConnect API by @tinaselenge in #10911
- [ST] Fix Javadoc warnings in STs about wrong links by @im-konge in #10934
- [system test] [doc] LoggingChangeST by @see-quick in #10842
- Remove MirrorMaker 1 support by @scholzj in #10924
- Fix network policies to allow CO talk to the controller nodes by @scholzj in #10940
- docs(edit): removes some mm1 content by @PaulRMellor in #10945
- [ST] Update mechanism for changing the Docker registry, repository, and tag - main branch by @im-konge in #10947
- Avoid failing the CI pipelines on "special" commit messages by @scholzj in #10956
- Rename
04A-Crd-kafkanodepool.yaml
to045-Crd-kafkanodepool.yaml
by @scholzj in #10953 - Added Kate Stanley as new maintainer by @ppatierno in #10960
- Remove Connect and MM2 upgrade from Deployment support by @scholzj in #10930
- Add the 0.45.0 release to the main branch by @ppatierno in #10962
- Remove Kafka and ZooKeeper upgrades from StatefulSets by @scholzj in #10935
- Update default logging configurations for Kafka brokers and MM2 by @scholzj in #10961
- [ST] Add Strimzi 0.45.0 to the upgrade/downgrade tests by @im-konge in #10963
- Update Fabric8 Kubernetes client to 7.0.1 by @scholzj in #10965
- Remove unused options from UO and TO logging test configurations by @scholzj in #10966
- [ST] Specify correct registry, org, and tag for Helm and remove unused code for OCP by @im-konge in #10955
- docs(KafkaNodePool): clean up feature gate description by @PaulRMellor in #10974
- Remove ZooKeeper support - Part I by @scholzj in #10964
- [system test] Remove un-necessary vertx Description annotation from t… by @see-quick in #10976
- Update Strimzi Test Container 0.109.1 by @see-quick in #10914
- Fix OPA Integration systemtests by @scholzj in #10980
- Deprecate ZooKeeper-based cluster fields in the API by @scholzj in #10979
- Exposing dnsPolicy and dnsConfig on the Strimzi api PodTemplate to allow custom name resolution for Pod resources by @gertschouten in #10949
- Remove ZooKeeper from packaging files by @scholzj in #10982
- bump tiered storage plugin version by @showuon in #10981
- Clean some ZooKeeper leftovers from
operator-common
module by @scholzj in #10984 - Clean-up the
KRaftUtils
class by @scholzj in #10987 - Remove deprecated EO Secret methods by @scholzj in #10993
- Consolidate methods updating
Kafka
CR status inKafkaReconciler
by @scholzj in #10995 - Clean-up the
KafkaClusterCreator
class after ZooKeeper removal by @scholzj in #10997 - Clean
KafkaBrokerConfigurationBuilder
class after ZooKeeper removal by @scholzj in #10998 - Updated release documentation by @ppatierno in #10999
- Update Helm Chart README with Zoo and MM1 removal and upgrade info by @scholzj in #11001
- docs(bug): code example formatting fix by @PaulRMellor in #11003
- Remove ZooKeeper from the docs by @scholzj in #10988
- Remove outdated upgrade docs by @scholzj in #11006
- docs(oauth): updates example client config for truststore by @PaulRMellor in #10969
- docs(mm2): considerations for active/passive disaster recovery by @PaulRMellor in #10942
- Fixed check Optional not null by @ppatierno in #11015
- Fix naming of bootstrap service for oddly named listener by @scholzj in #11016
- Make TopicOperatorConfig consistent with the other operators by @fvaleri in #11017
- Fix additional volumes in MirrorMaker 2 by @scholzj in #11022
- [ST] ZK removal from STs - regression tests and migration by @im-konge in #11018
- Remove the external service name method from
KafkaResources
by @scholzj in #11023 - Update Maven Builder image to 1.21 by @scholzj in #11024
- [system test] [doc] Add LogSettingST by @see-quick in #10938
- [ST] Remove ZooKeeper from upgrade/downgrade tests by @im-konge in #11026
- Fix minor typo by @ItaiFonio in #11043
- Remove last ZooKeeper methods from KafkaResources by @scholzj in #11040
- Store Kafka node certificates in separate Secrets by @katheris in #10967
- Remove support for storage class overrides by @scholzj in #11034
- Remove unused logger from Docs and test configs - Closes #11047 by @scholzj in #11048
- [ST] Remove rest of the Zookeeper occurences from STs, together with other unused code (UTO, NodePools/KRaft checks) by @im-konge in #11041
- Handle response with status 409 when talking to KafkaConnect API by @tinaselenge in #11042
- [system test] - Make it possible to use POSTGRES_IMAGE env by @see-quick in #11044
- Add support for CEL validation to our CRDs by @scholzj in #11068
- Loosen the restrictions for TLS options in
type: custom
listener authentication to enable new features by @scholzj in #11052 - Moved bridge configuration setup within the operator by @ppatierno in #11032
- docs(kraft): cleans up the docs for out-of-date descriptions relating to KRaft by @PaulRMellor in #11071
- Refactoring around bridge ConfigMap naming and volume mounts by @ppatierno in #11075
- Bumped Vert.x 4.5.12 and Netty 4.1.117.Final by @ppatierno in #11076
- Renamed constants for config providers env vars by @ppatierno in #11088
- Removed usage of
statefulset.kubernetes.io/pod-name
label on services and pods by @ppatierno in #11087 - [DOC] Clarify the role of the PKCS12 file in the CA secrets and remove the OpenSSL command by @scholzj in #11089
- docs(cleanup): removes or replaces content related to zookeeper or mirrormaker1 by @PaulRMellor in #11093
- Try fix race condition inside TopicControllerIT and KafkaHandlerIT by @see-quick in #11070
- Improve Cruise Control client creation and testing by @fvaleri in #10918
- [system test] [doc] Metrics package by @see-quick in #11036
- Improve memory consumption on large Kubernetes clusters when NodePort listeners are used by @scholzj in #11098
- Update Fabric8 Kubernetes Client to 7.1.0 by @scholzj in #11102
- Refactoring to use mapEmpty() to map to a Future by @ppatierno in #11105
- [ST] Update test-clients to 0.10.0 after the release by @im-konge in #11090
- Deprecate Secrets field in custom server authentication by @scholzj in #11108
- Add Kafka Access Operator 0.1.1 to the packaging directory by @scholzj in #11112
- Prepare for Vert.x 5 by @scholzj in #11107
- Minor CRD Generator cleanup by @scholzj in #11110
- [ST] Configure number of messages per transaction into one more test by @im-konge in #11111
- Avoid adding futures to immutable list by @scholzj in #11099
- docs(feature gates): updates the feature gates content by @PaulRMellor in #11097
- Fix ACL Deny rules documentation by @scholzj in #11119
- docs(acls): describes handling of special users by @PaulRMellor in #11118
- Update Vert.x to 4.5.13 by @scholzj in #11124
- Added dev documentation about running CVEs respin by @ppatierno in #11128
- Update KindContainer used by MockKube3 by @scholzj in #11129
- Trivial typo on the Helm Charts release doc by @ppatierno in #11140
- Remove past workaround of json-path CVE by @mstruk in #11135
- Revert PR 10016 Allow KafkaRoller talk to controller directly by @tinaselenge in #11132
- Fix release doc typo about helm-acceptance pipeline by @ppatierno in #11142
- docs(tuning): updates description of managed broker configurations by @PaulRMellor in #11138
- Remove the Desired Kafka replicas column from the Kafka CRD by @scholzj in #11137
- docs(api): updates description of config exclusions by @PaulRMellor in #11139
- Move connect configuration set up to the operator by @tinaselenge in #11062
- docs(downgrade): highlights the standard order of downgrade by @PaulRMellor in #11144
- Update Kafka Exporter to a new version 1.9.0 by @scholzj in #11152
- Fix deprecated GitHub actions/cache plugin by @scholzj in #11158
- docs: fix missing escape characters by @HirokiSakonju in #11164
- feat(examples,metrics,kube-state-metrics): add configmap and promethe… by @sebastiangaiser in #10919
- [ST] Use correct variable types for metrics, fix wrong logging by @im-konge in #11160
- [system test] [perf] Add documentation to our performance tests by @see-quick in #11165
- Remove Zookeeper relevant logic from KafkaRoller by @tinaselenge in #11149
- [system test] [perf] Add to TO scalability tests also printing the table by @see-quick in #11157
- Added development doc about the Operators Catalog release by @ppatierno in #11155
- [system test] Kafka rolling update with dedicated controllers by @see-quick in #11031
- [ST] Fix issue with specified Bridge image when running tests with Helm installation type and add info about configuring images into TESTING.md by @im-konge in #11161
- Update KindContainer to 1.4.9 and to Kube 1.32 by @scholzj in #11173
- Refactoring to use enhanced switch by @ppatierno in #11174
- Enable multi version single step downgrade for KRaft based clusters by @MichaelMorrisEst in #10929
- Remove unused fields from
kafka-versions.yaml
file by @scholzj in #11175 - Remove the Log Message and Inter Broker Protocol annotations by @scholzj in #11177
- [ST] Change the way how we assert values in logs from Pods by @im-konge in #11187
- Add Podman Compatibility and Enhance Kind cluster setup by @see-quick in #11086
- feat(examples,metrics,kube-state-metrics): extend configmap and prome… by @sebastiangaiser in #11189
- docs(cruise control): updates limitations note for cruise control by @PaulRMellor in #11188
- Remove JenkinsFile for x64 arch by @see-quick in #11196
- Load also ip_tables kernel module into podman VMs, Also fix pasta issue. by @see-quick in #11193
- [ST] Fix creation of CollectorElement based on installation type by @im-konge in #11197
- Fix Mirror Maker 2 connector reconciliation race condition by @scholzj in #11194
- Add support for Kafka 4.0.0 by @scholzj in #11176
- [ST] Annotate the Pod resources instead of the StrimziPodSet resource for starting a rolling restart in KafkaRollerST by @pnagy-cldr in #11204
- Refactoring with enhanced switch by @ppatierno in #11211
- [ST] Add Kafka Access Operator ST by @im-konge in #11192
- docs(operators): updates to the operator overviews by @PaulRMellor in #11163
- Add Cloudera to ADOPTERS file by @urbandan in #11213
- Deprecate the
type: opa
authorizer by @scholzj in #11202 - [system test] Fix downgrade procedure by @see-quick in #11154
- Refactor MirrorMaker2Connectors configuration setup by @tinaselenge in #11150
- docs(kraft): kRaft diagram updates by @PaulRMellor in #11225
- Adopt new Kafka Connect health check endpoint by @fvaleri in #11218
- docs(cruise control): update to cruise control flow diagram by @PaulRMellor in #11226
- User Operator handles the cluster CA secret being missing by @katheris in #11229
- [ST] Add Service to LogCollector's list of resources to collect by @im-konge in #11235
- Add Drain Cleaner 1.3.0 to the Operators repository by @im-konge in #11234
- [ST] Correct names in loggers for KAO ST by @im-konge in #11238
- [ST] Use
AutoOffsetResetStrategy
in clients instead ofOffsetResetStrategy
by @im-konge in #11239 - docs(downgrade): updates from review of downgrade content by @PaulRMellor in #11231
- Unify configuration hash annotations by @fvaleri in #11237
- Topic Operator test refactoring by @fvaleri in #11220
- Minor refactoring of starting SPS controller method by @ppatierno in #11255
- setup.kind fix IPv6/dual provisioning by @see-quick in #11257
- Allow run LoadBalancer tests on TestingFarm by @see-quick in #11222
- Remove preferIPv4Stack option and let it use JVM defaults by @darwin67 in #11247
- [system test] [perf] Fix executor handling by @see-quick in #11270
- [DOC] clarifies configuration of jmxOptions and Prometheus Exporter by @PaulRMellor in #11264
- Updated to Kafka 4.0.0 by @ppatierno in #11233
- Remove unnecessary executeBlocking call by @katheris in #11269
- Set units and chart descriptions for MirrorMaker2 Grafana example dashboard by @shk3 in #11263
- Replaced openssl with Java security to add key/cert into keystore by @ppatierno in #11224
- docs(logging): log4j2 support updates by @PaulRMellor in #11254
- docs(upgrade): clarifies how upgrade/downgrade paths work by @PaulRMellor in #11290
- Strimzi test container 0.110.0 by @see-quick in #11287
- bump the tiered storage plugin version by @showuon in #11300
- [system test] [perf] fix 409 error code by @see-quick in #11308
- Fix default value typo in README.md by @Wilya3 in #11314
- Bump skodjob-test-docs generator to version 0.4.0 by @Frawless in #11322
- Store CA generation in Ca class by @katheris in #11277
- Bumped Vert.x 4.5.14 by @ppatierno in #11324
- docs(connect): updates the deployment instructions for kafka connect by @PaulRMellor in #11301
- Bump the Cruise Control version to 2.5.142 by @ShubhamRwt in #11288
- Mount OAuth related client configuration via volume within Kafka Bridge by @tinaselenge in #11253
- Mount OAuth related client configuration via volume within Kafka Connect by @tinaselenge in #11306
- Normalize volume size for PVC to skip unnecessary patches by @im-konge in #11315
- Update ADOPTERS.md add SURF by @pboers1988 in #11337
- Promote
ContinueReconciliationOnManualRollingUpdateFailure
FG to GA by @im-konge in #11323 - Clean Azure agent in AZPs before running tests by @im-konge in #11330
- docs(opa): update of metrics and security content by @PaulRMellor in #11307
- Increase timeout for signing manifest by @see-quick in #11342
- [system test] [perf] capacity test by @see-quick in #11319
- Refactor CaReconcilerTest to improve readability by @katheris in #11343
- Minor improvement for tiered storage system test by @showuon in #11320
- Add a NFS system test for tiered storage by @showuon in #11341
- Bumped OAuth and Jackson dependencies by @ppatierno in #11351
- [docs] Fix typos and broken forms, etc. by @dongjinleekr in #11356
- docs(authz): removes AclAuthorizer by @PaulRMellor in #11345
- feat(kafkaconnect): allow ignore path kaniko opt by @coltmcnealy-lh in #11367
- docs(mm2): updates the deployment instructions for MirrorMaker 2 by @PaulRMellor in #11358
- Update bridge to 0.32.0 by @ppatierno in #11361
- fix QuotasOperator javadoc by @see-quick in #11372
- Update restart events to put resource in regarding by @katheris in #11352
- ST: Make several systemtests checks more resilient by @Frawless in #11373
Full Changelog: 0.45.0...0.46.0