-
Notifications
You must be signed in to change notification settings - Fork 3.1k
[connector/metricsaslogsconnector] first version of metricsaslogs connector #42052
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kyo-ke
wants to merge
32
commits into
open-telemetry:main
Choose a base branch
from
kyo-ke:metricsaslogsconnector
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 24 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
0908994
feat: first version of metricsaslogs connector
kyo-ke 398c37c
feat: update go.mod, go.sum
kyo-ke 1976421
make goporto
kyo-ke ddaaf73
make addlicense
kyo-ke f1b4754
add changelog
kyo-ke 88bccd1
update metadata + versions
kyo-ke 6ccdfaa
update codeowner
kyo-ke 1a95e3f
fix lint
kyo-ke b0b3efe
Merge branch 'main' into metricsaslogsconnector
kyo-ke 4051098
Merge branch 'main' of github.com:kyo-ke/opentelemetry-collector-cont…
kyo-ke 376837f
update module
kyo-ke d846ebf
Merge branch 'metricsaslogsconnector' of github.com:kyo-ke/openteleme…
kyo-ke 879490b
update tidylist
kyo-ke e22dd70
Merge branch 'main' into metricsaslogsconnector
kyo-ke 8c27372
Merge branch 'main' into metricsaslogsconnector
kyo-ke 828933e
Merge branch 'main' into metricsaslogsconnector
kyo-ke 3b49e67
Merge branch 'main' into metricsaslogsconnector
kyo-ke d29f303
Merge branch 'main' into metricsaslogsconnector
kyo-ke 8913ef3
Merge branch 'main' into metricsaslogsconnector
kyo-ke 98f964f
Merge branch 'main' into metricsaslogsconnector
kyo-ke a0e1f5f
update go mod
kyo-ke 8010c5d
change alpha to development & make generate & make genlabels
kyo-ke 56efd9d
Merge branch 'main' into metricsaslogsconnector
kyo-ke 53ce598
Merge branch 'main' into metricsaslogsconnector
kyo-ke c583dc5
make gencodecov
kyo-ke e51e5dc
update module version
kyo-ke ac90551
Merge branch 'main' into metricsaslogsconnector
kyo-ke 2459573
Merge branch 'main' into metricsaslogsconnector
kyo-ke afe0189
Merge branch 'main' into metricsaslogsconnector
kyo-ke 2a4e632
update mod versions
kyo-ke 8bb8c64
update versions.yaml
kyo-ke 9837c50
Update .chloggen/metricsaslogsconnector.yaml
atoulme File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Use this changelog template to create an entry for release notes. | ||
|
|
||
| # One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
| change_type: new_component | ||
|
|
||
| # The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver) | ||
| component: metricsaslogsconnector | ||
|
|
||
| # A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
| note: Add connector to convert metrics to logs | ||
|
|
||
| # Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. | ||
| issues: [40938] | ||
|
|
||
| # (Optional) One or more lines of additional information to render under the primary note. | ||
| # These lines will be padded with 2 spaces and then inserted directly into the document. | ||
| # Use pipe (|) for multiline entries. | ||
| subtext: | ||
|
|
||
| # If your change doesn't affect end users or the exported elements of any package, | ||
| # you should instead start your pull request title with [chore] or use the "Skip Changelog" label. | ||
| # Optional: The change log or logs in which this entry should be included. | ||
| # e.g. '[user]' or '[user, api]' | ||
| # Include 'user' if the change is relevant to end users. | ||
| # Include 'api' if there is a change to a library API. | ||
| # Default: '[user]' | ||
| change_logs: [user] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| include ../../Makefile.Common |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| # metricsaslogsconnector | ||
|
|
||
| <!-- status autogenerated section --> | ||
| | Status | | | ||
| | ------------- |-----------| | ||
| | Distributions | [] | | ||
| | Issues | [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aconnector%2Fmetricsaslogs) [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aconnector%2Fmetricsaslogs) | | ||
| | Code coverage | [](https://app.codecov.io/gh/open-telemetry/opentelemetry-collector-contrib/tree/main/?components%5B0%5D=connector_metricsaslogs&displayType=list) | | ||
| | [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@atoulme](https://www.github.com/atoulme) | | ||
|
|
||
| [development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development | ||
|
|
||
| ## Supported Pipeline Types | ||
|
|
||
| | [Exporter Pipeline Type] | [Receiver Pipeline Type] | [Stability Level] | | ||
| | ------------------------ | ------------------------ | ----------------- | | ||
| | metrics | logs | [development] | | ||
|
|
||
| [Exporter Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#exporter-pipeline-type | ||
| [Receiver Pipeline Type]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/connector/README.md#receiver-pipeline-type | ||
| [Stability Level]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#stability-levels | ||
| <!-- end autogenerated section --> | ||
|
|
||
| This connector converts OpenTelemetry metrics into logs, creating one log entry per metric data point. Each metric data point is transformed into a structured log record with configurable JSON body format. | ||
|
|
||
| ## Current Limitations | ||
|
|
||
| ⚠️ **Current implementation discards the following metric features:** | ||
| - Metric exemplars | ||
| - Advanced metadata | ||
|
|
||
| These features may be added in future iterations. | ||
|
|
||
| ## Configuration | ||
|
|
||
| The following settings can be optionally configured: | ||
|
|
||
| - `include_resource_attributes` (default = `true`): Whether to include resource attributes in the generated logs | ||
| - `include_scope_info` (default = `true`): Whether to include instrumentation scope information in the generated logs | ||
|
|
||
| ## Log Body Format | ||
|
|
||
| The connector always generates log bodies in the following JSON format: | ||
| ```json | ||
| {"metric_name": "$NAME", "value": "$VALUE"} | ||
| ``` | ||
|
|
||
| Where: | ||
| - `$NAME` is the actual metric name | ||
| - `$VALUE` is the metric value (simple values for gauge/sum, complex JSON for histogram/summary) | ||
|
|
||
| ## Example Usage | ||
|
|
||
| ### Basic Configuration | ||
|
|
||
| ```yaml | ||
| connectors: | ||
| metricsaslogs: | ||
|
|
||
| service: | ||
| pipelines: | ||
| logs: | ||
| receivers: [metricsaslogs] | ||
| processors: [] | ||
| exporters: [logging] | ||
| metrics: | ||
| receivers: [otlp] | ||
| processors: [] | ||
| exporters: [metricsaslogs] | ||
| ``` | ||
|
|
||
| ### Advanced Configuration | ||
|
|
||
| ```yaml | ||
| connectors: | ||
| metricsaslogs: | ||
| include_resource_attributes: false | ||
| include_scope_info: false | ||
| ``` | ||
|
|
||
|
|
||
| ### Example Metric Conversions | ||
|
|
||
| For a gauge metric `cpu_usage` with value `85.2`: | ||
| ```json | ||
| { | ||
| "body": {"metric_name": "cpu_usage", "value": "85.2"}, | ||
| "attributes": { | ||
| "metric.name": "cpu_usage", | ||
| "metric.type": "Gauge", | ||
| "metric.description": "CPU usage percentage", | ||
| "metric.unit": "%" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| For a histogram metric `request_duration`: | ||
| ```json | ||
| { | ||
| "body": { | ||
| "metric_name": "request_duration", | ||
| "value": "{\"count\":100,\"sum\":1.5,\"bucket_counts\":[10,50,40],\"explicit_bounds\":[0.1,0.5,1.0]}" | ||
| }, | ||
| "attributes": { | ||
| "metric.name": "request_duration", | ||
| "metric.type": "Histogram", | ||
| "metric.description": "Request duration in seconds", | ||
| "metric.unit": "s", | ||
| "metric.aggregation_temporality": "Delta" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Output Structure | ||
|
|
||
| Each metric data point is converted to a log record with: | ||
|
|
||
| - **Body**: Fixed JSON format: `{"metric_name": "$NAME", "value": "$VALUE"}` | ||
| - **Timestamp**: Metric data point timestamp | ||
| - **Observed Timestamp**: Metric data point start timestamp (if available) | ||
| - **Attributes**: | ||
| - Original metric data point attributes (labels) | ||
| - `metric.name`: The metric name | ||
| - `metric.type`: The metric type (Gauge, Sum, Histogram, etc.) | ||
| - `metric.description`: Metric description (if available) | ||
| - `metric.unit`: Metric unit (if available) | ||
| - Additional type-specific attributes: | ||
| - For Sum metrics: `metric.is_monotonic`, `metric.aggregation_temporality` | ||
| - For Histogram/ExponentialHistogram: `metric.aggregation_temporality` | ||
| - Resource attributes (if `include_resource_attributes` is true) | ||
| - Instrumentation scope information (if `include_scope_info` is true) | ||
|
|
||
| ## Supported Metric Types | ||
|
|
||
| All OpenTelemetry metric types are supported: | ||
|
|
||
| - **Gauge**: Point-in-time measurements | ||
| - **Sum**: Cumulative or delta measurements | ||
| - **Histogram**: Distribution of measurements with buckets | ||
| - **Exponential Histogram**: Distribution with exponentially sized buckets | ||
| - **Summary**: Distribution with quantile values | ||
|
|
||
| ## Value Encoding | ||
|
|
||
| The `value` field in the JSON body contains: | ||
|
|
||
| - For simple metrics (Gauge, Sum): numeric value as string | ||
| - For complex metrics (Histogram, etc.): JSON-encoded object as string | ||
|
|
||
| All JSON special characters in metric names and values are properly escaped. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package metricsaslogsconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/metricsaslogsconnector" | ||
|
|
||
| import ( | ||
| "go.opentelemetry.io/collector/component" | ||
| ) | ||
|
|
||
| type Config struct { | ||
| IncludeResourceAttributes bool `mapstructure:"include_resource_attributes"` | ||
|
|
||
| IncludeScopeInfo bool `mapstructure:"include_scope_info"` | ||
|
|
||
| _ struct{} | ||
| } | ||
|
|
||
| func (*Config) Validate() error { | ||
| return nil | ||
| } | ||
|
|
||
| func createDefaultConfig() component.Config { | ||
| return &Config{ | ||
| IncludeResourceAttributes: true, | ||
| IncludeScopeInfo: true, | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.