[controller] integrate OpenTelemetry into Controller & add log compaction metrics#1665

Draft

WhitneyDeng wants to merge 33 commits intolinkedin:mainfrom

WhitneyDeng:log_compaction/observability/integrate-otel-to-controller

Commits on May 23, 2025

ControllerMetricEntity enum class
Whitney Deng
committed
[WIP] setup VeniceMetricsRepositoru for controller
Whitney Deng
committed
MetricsRepository -> VeniceMetricRepository in VeniceController and VeniceControllerContext
Whitney Deng
committed
import otel in controller build.gradle
Whitney Deng
committed
[wip] LogCompactionServiceStats.java
Whitney Deng
committed
[WIP] Attributes builder
Whitney Deng
committed
`LogCompactionService` basic constructor
Whitney Deng
committed
add JOB_RUN_STATUS dimension
Whitney Deng
committed
add LOG_COMPACTION_SELECTION_REASON dimension
Whitney Deng
committed
add VENICE_CONTROLLER_ENDPOINT dimension
Whitney Deng
committed
REPUSH_STORE_ENDPOINT_CALL_COUNT in `ControllerMetricEntity`
Whitney Deng
committed
rebase: LogCompactionService scoped to cluster level
Whitney Deng
committed
rename LogCompactionServiceStats -> LogCompactionStats
Whitney Deng
committed
rebase
Whitney Deng
committed
LogCompactionStats extends AbstractVeniceStats
Whitney Deng
committed
delete ControllerEndpoints.java
Whitney Deng
committed
add MetricsRepository to VeniceParentHelixAdmin constructor
Whitney Deng
committed
adding `REPUSH_STORE_ENDPOINT_CALL_COUNT` metric
Whitney Deng
committed
move repushStore endpoint to parent controller
Whitney Deng
committed
add Optional<MetricsRepository> in AbstractTestVeniceParentHelixAdmin::initializeParentAdmin to pass in metricsRepository for testing metric emission in VeniceParentHelixAdmin
Whitney Deng
committed
add metricName in ControllerMetricEntity to streamline access to metric name
Whitney Deng
committed
LogCompactionStatsTest
Whitney Deng
committed
nit delete testing print statement
Whitney Deng
committed
pass in `LogCompactionStats` to `CompactionManager`
Whitney Deng
committed
add STORE_NOMINATED_FOR_SCHEDULED_COMPACTION metric entity
Whitney Deng
committed
add `STORE_NOMINATED_FOR_SCHEDULED_COMPACTION` to `LogCompactionStats`
Whitney Deng
committed
cleanup after rebase
Whitney Deng
committed
add createGuage()
Whitney Deng
committed
nit format
Whitney Deng
committed
revert mistake
Whitney Deng
committed
merge conflict
Whitney Deng
committed

Commits on May 24, 2025

add storeCurrentVersionNumber as metric dimension
Whitney Deng
committed
emit storeNominatedForScheduledCompactionMetric for store nomination & compaction
Whitney Deng
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[controller] integrate OpenTelemetry into Controller & add log compaction metrics#1665

Commits on May 23, 2025

ControllerMetricEntity enum class

[WIP] setup VeniceMetricsRepositoru for controller

MetricsRepository -> VeniceMetricRepository in VeniceController and VeniceControllerContext

import otel in controller build.gradle

[wip] LogCompactionServiceStats.java

[WIP] Attributes builder

`LogCompactionService` basic constructor

add JOB_RUN_STATUS dimension

add LOG_COMPACTION_SELECTION_REASON dimension

add VENICE_CONTROLLER_ENDPOINT dimension

REPUSH_STORE_ENDPOINT_CALL_COUNT in `ControllerMetricEntity`

rebase: LogCompactionService scoped to cluster level

rename LogCompactionServiceStats -> LogCompactionStats

rebase

LogCompactionStats extends AbstractVeniceStats

delete ControllerEndpoints.java

add MetricsRepository to VeniceParentHelixAdmin constructor

adding `REPUSH_STORE_ENDPOINT_CALL_COUNT` metric

move repushStore endpoint to parent controller

add Optional<MetricsRepository> in AbstractTestVeniceParentHelixAdmin::initializeParentAdmin to pass in metricsRepository for testing metric emission in VeniceParentHelixAdmin

add metricName in ControllerMetricEntity to streamline access to metric name

LogCompactionStatsTest

nit delete testing print statement

pass in `LogCompactionStats` to `CompactionManager`

add STORE_NOMINATED_FOR_SCHEDULED_COMPACTION metric entity

add `STORE_NOMINATED_FOR_SCHEDULED_COMPACTION` to `LogCompactionStats`

cleanup after rebase

add createGuage()

nit format

revert mistake

merge conflict

Commits on May 24, 2025

add storeCurrentVersionNumber as metric dimension

emit storeNominatedForScheduledCompactionMetric for store nomination & compaction