Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

OwenCorrigan76 · 2025-01-16T17:14:53Z

Type of change

Enhancement / new feature

Description

This patch adds support for the Strimzi Metrics Reporter to Kafka brokers/controllers components as described by the following proposal:

https://github.com/strimzi/proposals/blob/main/064-prometheus-metrics-reporter.md

Related to #10753

Support for Kafka Connect and MirrorMaker2 and Strimzi Kafka Bridge will be added in subsequent PRs.

CEL Validation has been added to the MetricsConfig and CruiseControl API in this PR.
CEL Validation was introduced to Strimzi in this PR:
#11068

We won’t initially support the CruiseControl component. To make it work, CC should be changed to expose metrics through its HTTP endpoint.

Checklist

Write tests
Make sure all tests pass
Update documentation
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md

scholzj

I left some comments after an initial pass. You should also update the names / description / CHANGELOG to make it clear what this really does. Kafka compoenents are brokers/controllers, Connect and MM2. You add this only for brokers/controllers. It should be clear from the CHANGELOG and PR name / desc.

CHANGELOG.md

packaging/examples/metrics/grafana-dashboards/strimzi-kafka.json

packaging/examples/metrics/grafana-dashboards/strimzi-kraft.json

api/src/main/java/io/strimzi/api/kafka/model/kafka/KafkaClusterSpec.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/ConfigMapUtils.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/CruiseControl.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaCluster.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaCluster.java

OwenCorrigan76 · 2025-01-17T12:01:44Z

@scholzj I am currently working on the changes I did not comment on yet.

CHANGELOG.md

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/StrimziMetricsReporterValues.java

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/JmxPrometheusExporterMetrics.java

scholzj

I left some more comments. There also seem to be many unaddressed older comments. Not sure if that was intentional or not at this point.

CHANGELOG.md

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/CruiseControl.java

...tor/src/main/java/io/strimzi/operator/cluster/model/metrics/StrimziMetricsReporterModel.java

...ator/src/main/java/io/strimzi/operator/cluster/model/metrics/JmxPrometheusExporterModel.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/metrics/MetricsModel.java

packaging/examples/metrics/reporter/kafka-metrics.yaml

fvaleri · 2025-02-17T07:56:46Z

There also seem to be many unaddressed older comments. Not sure if that was intentional or not at this point.

@scholzj I'm supporting Owen with this feature. He is working on and will eventually address all comments.

OwenCorrigan76 · 2025-02-17T11:15:26Z

@scholzj I know progress is going quite slowly. I will address the previous comments shortly. I'm currently working on the CEL validation. When that is committed, I'll go back and tackle the older comments and work through them.

scholzj · 2025-02-17T13:24:54Z

@scholzj I know progress is going quite slowly. I will address the previous comments shortly. I'm currently working on the CEL validation. When that is committed, I'll go back and tackle the older comments and work through them.

No worries Owen, I was just unsure what the expected state is. I did not intend to put any pressure on your. Sorry.

OwenCorrigan76 · 2025-02-25T11:02:31Z

The build failed with the following tests:
io.strimzi.api.kafka.model.kafka.KafkaCrdIT.testKafkaWithNullMaintenance
Currently investigating why.

fvaleri

Hi @OwenCorrigan76, we still need to do some changes and address the remaining comments, but it works fine.

For the KafkaCrdIT.testKafkaWithNullMaintenance failure, I think we simply need to update the expected error messages removing the invalid: prefix. Now, you get invalid: [, rather than invalid: . With CEL validation rules, we get the additional error message "validation rules were not checked".

api/src/main/java/io/strimzi/api/kafka/model/kafka/KafkaClusterSpec.java

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/JmxPrometheusExporterMetrics.java

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/StrimziMetricsReporterValues.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/CruiseControl.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaConnectCluster.java

docker-images/kafka-based/kafka/scripts/kafka_connect_run.sh

docker-images/kafka-based/kafka/scripts/kafka_run.sh

packaging/examples/metrics/strimzi-metrics-reporter/kafka-metrics.yaml

fvaleri

@OwenCorrigan76 I was wrong on the "Unsupported metrics type", please correct.

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/JmxPrometheusExporterMetrics.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaCluster.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaConnectCluster.java

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

fvaleri

Changes looks good to me for now, but we still have some comments to address.

scholzj · 2025-04-23T16:13:21Z

I guess this will need a rebase after #11379.

ppatierno · 2025-04-30T08:51:14Z

@OwenCorrigan76 before having another review pass, this PR should be rebased, let us know when it's ready again. Thanks!

OwenCorrigan76 · 2025-05-02T08:24:36Z

Squashed commits for rebasing. I will deal with conflicts now.

fvaleri · 2025-05-06T16:25:24Z

@strimzi/maintainers can one of you trigger the ST pipeline? Thanks.

ppatierno · 2025-05-06T16:26:37Z

/azp run regression

azure-pipelines · 2025-05-06T16:26:48Z

Azure Pipelines successfully started running 1 pipeline(s).

fvaleri · 2025-05-07T06:05:24Z

@ppatierno looks like failing tests hit timeout and seems to be unrelated.

Frawless · 2025-05-07T07:04:06Z

/packit test --labels regression,upgrade

Frawless · 2025-05-07T07:28:25Z

TF failures are caused by TF outage. I will restart the jobs once the outage will be resolved.

Frawless · 2025-05-07T15:56:45Z

/packit test --labels regression,upgrade

OwenCorrigan76 · 2025-05-07T16:24:00Z

TF failed again.

Frawless · 2025-05-07T19:08:38Z

TF failed again.

I am afraid they are still experiencing the outage. In case it is blocking you, feel free to ignore it. I will try to restart it tomorrow and hope everything will be fine that time.

OwenCorrigan76 · 2025-05-08T12:46:46Z

This is ready for another review, when you get a chance. Thanks a mil.
@ppatierno @scholzj

scholzj · 2025-05-12T20:24:03Z

...tor/src/test/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilderTest.java

@@ -471,9 +502,9 @@ public void testNullUserConfiguration()  {
    }

    @ParallelTest
-    public void testNullUserConfigurationAndCCReporter()  {
+    public void testNullUserConfigurationWithCruiseControlMetricsReporter() {


What if you add the new tests at the end rather than to the middle and mess up the diff and GitHub history?

...tor/src/test/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilderTest.java

scholzj · 2025-05-12T20:26:06Z

packaging/examples/metrics/strimzi-metrics-reporter/kafka-metrics.yaml

+spec:
+  kafka:
+    version: 4.0.0
+    metadataVersion: 4.0-IV0


Should this be IV3 as in the regular example (https://github.com/strimzi/strimzi-kafka-operator/blob/main/packaging/examples/metrics/kafka-metrics.yaml)?

scholzj · 2025-05-12T20:29:33Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

+     * @param values List configuration values.
+     */
+    public static void createOrAddListConfig(AbstractConfiguration kafkaConfig, String key, String values) {
+        if (kafkaConfig != null && key != null && !key.isBlank() && values != null && !values.isBlank()) {


Not 100% sure, but is this missing some else branch?

The Javadc says Append list configuration values or create a new list configuration if missing. -> So I would expect that you need to handle the situation when the things are not set?

Or maybe you know they are always set and do not need this check?

@OwenCorrigan76 this method should have private visibility.

@scholzj null/blank checks are there because this will become a shared utility and move to ModelUtils a soon as the KC/MM2 PR will be ready. Currently, we simply ignore invalid input, but we could raise and exception or log a warning. Wdyt?

I'm not sure what PR you're referring to. But I think the comment stands. If you really need to do the check, there really needs to be error handling for it.

Yep, I agree on adding error handling.

scholzj · 2025-05-12T20:31:00Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

+                    Arrays.stream(existingConfig.split(","))
+                            .map(String::trim)
+                            .filter(s -> !s.isEmpty())
+                            .collect(Collectors.toCollection(LinkedHashSet::new));
+            Set<String> newValues = Arrays.stream(values.split(","))
+                    .map(String::trim)
+                    .filter(s -> !s.isEmpty())
+                    .collect(Collectors.toCollection(LinkedHashSet::new));


Should we use something like asList(namespacesList.trim().split("\\s*,+\\s*"))? We already use that in ConfigParameterParser to parse namespace list. But to be honest, I have no idea which way would be faster.

Also, why do you use LinkedHashSet? Does the order really matter here? If it does, a comment explaining that might be useful.

@scholzj list would be probably faster, but it hardly makes a difference with few configuration values. Once this method becomes a shared utility, values could potentially be user-provided values, so we thought it's nice to preserve ordering.

With the speed, I was more referring to the splitting than list versus set.

As for the ordering, if it matters, fine. But it should be explained in the comment as I suggest.

scholzj

Few more nits. But hopefully that will be it.

scholzj · 2025-05-13T20:51:32Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

+        if (kafkaConfig == null) throw new IllegalArgumentException("Configuration is required");
+        if (key == null || key.isBlank()) throw new IllegalArgumentException("Configuration key is required");
+        if (values == null || values.isBlank()) throw new IllegalArgumentException("Configuration values are required");


I think if you properly format it with { / }, it would be more readable and clearer what exactly are you doing there. I do not think we use this form in many parts of the code base, if somewhere at all.

scholzj · 2025-05-13T20:54:55Z

...tor/src/test/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilderTest.java

+                .map(key -> assertThrows(IllegalArgumentException.class, () -> KafkaBrokerConfigurationBuilder.createOrAddListConfig(config, key, "test-value-3")))
+                .forEach(e -> assertThat(e.getMessage(), is("Configuration key is required")
+                ));
+


Empty line not needed.

Frawless · 2025-05-14T07:13:20Z

/packit test --labels regression,upgrade

This patch adds support for the Strimzi Metrics Reporter to brokers and controllers as described by the following proposal: https://github.com/strimzi/proposals/blob/main/064-prometheus-metrics-reporter.md Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Squash commits Address initial feedback Signed-off-by: ocorriga <ocorriga@redhat.com> Reverse dashboards and fix build issue Signed-off-by: ocorriga <ocorriga@redhat.com> Update derived resources Signed-off-by: ocorriga <ocorriga@redhat.com> Trigger build Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Add MetricsModel interface and refactor affected classes Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Add CEL validation and address comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Addressing latest comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> add TODO in withStrimziMetricsReporter Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> address latest comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Add StrimziMetricsReporterConfig class Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Add defaults for allowlist Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Refactor withUserConfiguration and add tests Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Revert Connect script ENV VAR Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Comments on common-configuration doc and fix typos Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Fix ConfigMap naming for tests Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address additional Comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Jakub's and Tina's Comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Add comments to distinguish between kafka and yammer metrics Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Paolo's comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Remove added comment writer.println Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Refactor behaviour of user config in KafkaBrokerConfigurationBuilder Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Fede's comments Signed-off-by: ocorriga <ocorriga@redhat.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Mickael's comments Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Fix Kafka dashboard Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Update API doc Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Add system test Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Minor ST improvements Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address Kate's and Paolo's comments Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Remove jayway-json-path dependency from kafka-thirdparty-libs and add listener.enabed=true Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address nit and move prometheus-jmx-exporter examples into it's own directory Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address ST comments Signed-off-by: Federico Valeri <fedevaleri@gmail.com> Address Jakub's latest comments Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address comments on KafkaBrokerConfigurationBuilder Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com> Address comments from Jakub Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

OwenCorrigan76 · 2025-05-14T10:08:02Z

Thanks @scholzj Nits addressed

ppatierno

Just a couple of nits but overall LGTM! Thanks for the effort @OwenCorrigan76 (and @fvaleri) ;-)

ppatierno · 2025-05-14T18:47:15Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

+            createOrAddListConfig(userConfig, "metric.reporters", CruiseControlMetricsReporter.CRUISE_CONTROL_METRIC_REPORTER);
+        }
+        if (injectKafkaJmxReporter) {
+            createOrAddListConfig(userConfig, "metric.reporters", "org.apache.kafka.common.metrics.JmxReporter");


What about having a constant for "org.apache.kafka.common.metrics.JmxReporter" within the JmxPrometheusExporterModel class?

ppatierno · 2025-05-14T18:47:46Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

        }
+        if (injectStrimziMetricsReporter) {
+            createOrAddListConfig(userConfig, "metric.reporters", "io.strimzi.kafka.metrics.KafkaPrometheusMetricsReporter");


Ditto as above, maybe a constant for "io.strimzi.kafka.metrics.KafkaPrometheusMetricsReporter" within the StrimziMetricsReporterModel class?

ppatierno · 2025-05-14T18:49:36Z

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java

+     */
+    private void maybeAddYammerMetricsReporters(KafkaConfiguration userConfig, boolean injectStrimziMetricsReporter) {
+        if (injectStrimziMetricsReporter) {
+            createOrAddListConfig(userConfig, "kafka.metrics.reporters", "io.strimzi.kafka.metrics.YammerPrometheusMetricsReporter");


Maybe we can have a constant for "io.strimzi.kafka.metrics.YammerPrometheusMetricsReporter" within the StrimziMetricsReporterModel class because it's anyway set in related to the Strimzi Metrics Reporter.

OwenCorrigan76 requested review from a team and mimaison January 16, 2025 17:15

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch 2 times, most recently from e569be2 to 1180ee2 Compare January 16, 2025 17:23

scholzj reviewed Jan 16, 2025

View reviewed changes

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from 1180ee2 to f943007 Compare January 17, 2025 12:00

fvaleri added this to the 0.46.0 milestone Jan 17, 2025

OwenCorrigan76 changed the title ~~Add support for the Strimzi Metrics Reporter to Kafka component~~ Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components Jan 17, 2025

scholzj reviewed Jan 19, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/StrimziMetricsReporterValues.java Outdated Show resolved Hide resolved

scholzj reviewed Jan 19, 2025

View reviewed changes

api/src/main/java/io/strimzi/api/kafka/model/common/metrics/JmxPrometheusExporterMetrics.java Show resolved Hide resolved

scholzj mentioned this pull request Jan 22, 2025

Add support for CEL validation to our CRDs #11068

Merged

4 tasks

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch 2 times, most recently from 84fb016 to 8f5b28f Compare February 13, 2025 11:16

scholzj reviewed Feb 16, 2025

View reviewed changes

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch 3 times, most recently from ece3b1f to 75a829f Compare February 24, 2025 13:23

fvaleri reviewed Feb 25, 2025

View reviewed changes

...perator/src/main/java/io/strimzi/operator/cluster/model/KafkaBrokerConfigurationBuilder.java Outdated Show resolved Hide resolved

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from 4304607 to f88ec2c Compare February 26, 2025 10:51

OwenCorrigan76 requested a review from fvaleri February 26, 2025 12:36

fvaleri reviewed Feb 26, 2025

View reviewed changes

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch 2 times, most recently from 3c94f43 to 3969049 Compare March 4, 2025 12:21

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from 6af487c to d4a8609 Compare May 2, 2025 08:23

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from d4a8609 to b89fa8d Compare May 6, 2025 14:07

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from b89fa8d to b28fde7 Compare May 9, 2025 15:28

fvaleri requested a review from scholzj May 12, 2025 18:02

scholzj reviewed May 12, 2025

View reviewed changes

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from b28fde7 to 22dd7a2 Compare May 13, 2025 14:32

scholzj approved these changes May 13, 2025

View reviewed changes

OwenCorrigan76 added 6 commits May 14, 2025 11:05

Refactor to address Jmx Reporter change

0a026ff

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

Improve tests in KafkaBrokerConfigurationBuilderTest

6334ded

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

Address comments from Jakub

5ac7beb

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

Fix testCreateOrAddListConfigWithNullConfig

ef34b6c

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

Address nits

2215a9b

Signed-off-by: OwenCorrigan76 <owencorrigan76@gmail.com>

OwenCorrigan76 force-pushed the integrate_metrics_reporter branch from ec69460 to 2215a9b Compare May 14, 2025 10:07

ppatierno approved these changes May 14, 2025

View reviewed changes

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

Are you sure you want to change the base?

Add support for the Strimzi Metrics Reporter to Kafka brokers/controllers components #11051

Conversation

OwenCorrigan76 commented Jan 16, 2025 • edited Loading

Type of change

Description

Checklist

scholzj left a comment

Choose a reason for hiding this comment

OwenCorrigan76 commented Jan 17, 2025

scholzj left a comment

Choose a reason for hiding this comment

fvaleri commented Feb 17, 2025 • edited Loading

OwenCorrigan76 commented Feb 17, 2025

scholzj commented Feb 17, 2025

OwenCorrigan76 commented Feb 25, 2025

fvaleri left a comment

Choose a reason for hiding this comment

fvaleri left a comment • edited Loading

Choose a reason for hiding this comment

fvaleri left a comment

Choose a reason for hiding this comment

scholzj commented Apr 23, 2025

ppatierno commented Apr 30, 2025

OwenCorrigan76 commented May 2, 2025

fvaleri commented May 6, 2025 • edited Loading

ppatierno commented May 6, 2025

azure-pipelines bot commented May 6, 2025

fvaleri commented May 7, 2025 • edited Loading

Frawless commented May 7, 2025

Frawless commented May 7, 2025

Frawless commented May 7, 2025

OwenCorrigan76 commented May 7, 2025

Frawless commented May 7, 2025

OwenCorrigan76 commented May 8, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scholzj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frawless commented May 14, 2025

OwenCorrigan76 commented May 14, 2025

ppatierno left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OwenCorrigan76 commented Jan 16, 2025 •

edited

Loading

fvaleri commented Feb 17, 2025 •

edited

Loading

fvaleri left a comment •

edited

Loading

fvaleri commented May 6, 2025 •

edited

Loading

fvaleri commented May 7, 2025 •

edited

Loading