You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* This enhancement adds the source `iostream` to the attributes sent from collected container logs. The value is set to either `stdout` or `stderr` based on how the collector received it. (link:https://issues.redhat.com/browse/LOG-5292[LOG-5292])
15
+
16
+
* With this update, the default memory limit for the collector increases from 1024 Mi to 2048 Mi. Users should adjust resource limits based on their cluster’s specific needs and specifications. (link:https://issues.redhat.com/browse/LOG-6072[LOG-6072])
17
+
18
+
* With this update, users can now set the syslog output delivery mode of the `ClusterLogForwarder` CR to either `AtLeastOnce` or `AtMostOnce.` (link:https://issues.redhat.com/browse/LOG-6355[LOG-6355])
19
+
20
+
=== Log Storage
21
+
22
+
* With this update, the new `1x.pico` LokiStack size supports clusters with fewer workloads and lower log volumes (up to 50GB/day). (link:https://issues.redhat.com/browse/LOG-5939[LOG-5939])
:FeatureName: The OpenTelemetry Protocol (OTLP) output log forwarder
28
+
include::snippets/technology-preview.adoc[]
29
+
30
+
* With this update, OpenTelemetry logs can now be forwarded using the `OTel` (OpenTelemetry) data model to a Red Hat Managed LokiStack instance. To enable this feature, add the `observability.openshift.io/tech-preview-otlp-output: "enabled"` annotation to your `ClusterLogForwarder` configuration. For additional configuration information, see link:https://github.com/openshift/cluster-logging-operator/blob/master/docs/features/logforwarding/outputs/opentelemetry-lokistack-forwarding.adoc[OTLP Forwarding].
31
+
32
+
* With this update, a `dataModel` field has been added to the `lokiStack` output specification. Set the `dataModel` to `Otel` to configure log forwarding using the OpenTelemetry data format. The default is set to `Viaq`. For information about data mapping see link:https://opentelemetry.io/docs/specs/otlp/[OTLP Specification].
Sizing for Loki follows the format of `1x.<size>` where the value `1x` is number of instances and `<size>` specifies performance capabilities.
9
9
10
-
For logging 6.1+,`1x.pico` configuration defines a single Loki deployment with minimal resource and limit requirements, offering high availability (HA) support for all Loki components. This configuration is suited for deployments that do not require a single replication factor or auto-compaction.
10
+
The`1x.pico` configuration defines a single Loki deployment with minimal resource and limit requirements, offering high availability (HA) support for all Loki components. This configuration is suited for deployments that do not require a single replication factor or auto-compaction.
11
11
12
12
Disk requests are similar across size configurations, allowing customers to test different sizes to determine the best fit for their deployment needs.
The `ClusterLogForwarder` custom resource (CR) is the central configuration point for log collection and forwarding.
10
10
11
-
== Inputs and Outputs
11
+
[id="inputs-and-outputs_6-1_{context}"]
12
+
== Inputs and outputs
12
13
13
14
Inputs specify the sources of logs to be forwarded. Logging provides built-in input types: `application`, `receiver`, `infrastructure`, and `audit`, which select logs from different parts of your cluster. You can also define custom inputs based on namespaces or pod labels to fine-tune log selection.
14
15
15
16
Outputs define the destinations where logs are sent. Each output type has its own set of configuration options, allowing you to customize the behavior and authentication settings.
16
17
17
-
18
-
== Receiver Input Type
18
+
[id="receiver-input-type_6-1_{context}"]
19
+
== Receiver input type
19
20
The receiver input type enables the Logging system to accept logs from external sources. It supports two formats for receiving logs: `http` and `syslog`.
20
21
21
22
The `ReceiverSpec` defines the configuration for a receiver input.
22
23
23
-
== Pipelines and Filters
24
+
[id="pipelines-and-filters_6-1_{context}"]
25
+
== Pipelines and filters
24
26
25
27
Pipelines determine the flow of logs from inputs to outputs. A pipeline consists of one or more input refs, output refs, and optional filter refs. Filters can be used to transform or drop log messages within a pipeline. The order of filters matters, as they are applied sequentially, and earlier filters can prevent log messages from reaching later stages.
26
28
27
-
== Operator Behavior
29
+
[id="operator-behavior_6-1_{context}"]
30
+
== Operator behavior
28
31
29
32
The Cluster Logging Operator manages the deployment and configuration of the collector based on the `managementState` field of the `ClusterLogForwarder` resource:
30
33
31
34
- When set to `Managed` (default), the operator actively manages the logging resources to match the configuration defined in the spec.
32
35
- When set to `Unmanaged`, the operator does not take any action, allowing you to manually manage the logging components.
33
36
37
+
[id="validation_6-1_{context}"]
34
38
== Validation
35
39
Logging includes extensive validation rules and default values to ensure a smooth and error-free configuration experience. The `ClusterLogForwarder` resource enforces validation checks on required fields, dependencies between fields, and the format of input values. Default values are provided for certain fields, reducing the need for explicit configuration in common scenarios.
36
40
37
-
[id="quick-start_{context}"]
41
+
[id="quick-start_6-1_{context}"]
38
42
== Quick start
39
43
40
44
OpenShift Logging supports two data models:
41
45
42
46
* ViaQ (General Availability)
43
47
* OpenTelemetry (Technology Preview)
44
48
45
-
You can select either of these data models based on your requirement by configuring the `lokiStack.dataModel` field in the `ClusterLogForwarder`. ViaQ is the default data model when forwarding logs to LokiStack. For more information on OpenTelemetry, see xref:../../../observability/logging/logging-6.1/log6x-opentelemetry-data-model-6.1.adoc#log6x-opentelemetry-data-model-6.1[OpenTelemetry data model].
49
+
You can select either of these data models based on your requirement by configuring the `lokiStack.dataModel` field in the `ClusterLogForwarder`. ViaQ is the default data model when forwarding logs to LokiStack.
46
50
47
51
[NOTE]
48
52
====
@@ -51,15 +55,4 @@ In future releases of OpenShift Logging, the default data model will change from
* xref:../../../observability/logging/logging-6.0/log6x-upgrading-to-6.adoc#secrets-and-tls-configuration[Secrets and TLS Configuration]
65
-
* xref:../../../observability/logging/logging-6.1/log6x-configuring-lokistack-otlp-6.1.adoc#log6x-configuring-lokistack-otlp-data-ingestion_log6x-configuring-lokistack-otlp-6.1[Configuring LokiStack for OTLP data ingestion]
To modify the log level in the collector, you can set the `observability.openshift.io/log-level` annotation to `trace`, `debug`, `info`, `warn`, `error`, and `off`.
@@ -32,6 +33,7 @@ metadata:
32
33
# ...
33
34
----
34
35
36
+
[id="managing-the-operator_6-1_{context}"]
35
37
== Managing the Operator
36
38
37
39
The `ClusterLogForwarder` resource has a `managementState` field that controls whether the operator actively manages its resources or leaves them Unmanaged:
@@ -42,6 +44,7 @@ Unmanaged:: The operator will not take any action related to the logging compone
42
44
43
45
This allows administrators to temporarily pause log forwarding by setting `managementState` to `Unmanaged`.
44
46
47
+
[id="clf-structure_6-1_{context}"]
45
48
== Structure of the ClusterLogForwarder
46
49
47
50
The CLF has a `spec` section that contains the following key components:
@@ -54,6 +57,7 @@ Pipelines:: Define the path logs take from inputs, through filters, to outputs.
54
57
55
58
Filters:: Transform or drop log messages in the pipeline. Users can define filters that match certain log fields and drop or modify the messages. Filters are applied in the order specified in the pipeline.
56
59
60
+
[id="clf-inputs_6-1_{context}"]
57
61
=== Inputs
58
62
59
63
Inputs are configured in an array under `spec.inputs`. There are three built-in input types:
@@ -66,6 +70,7 @@ audit:: Selects logs from the OpenShift API server audit logs, Kubernetes API se
66
70
67
71
Users can define custom inputs of type `application` that select logs from specific namespaces or using pod labels.
68
72
73
+
[id="clf-outputs_6-1_{context}"]
69
74
=== Outputs
70
75
71
76
Outputs are configured in an array under `spec.outputs`. Each output must have a unique name and a type. Supported types are:
@@ -86,6 +91,7 @@ Each output type has its own configuration fields.
Pipelines are configured in an array under `spec.pipelines`. Each pipeline must have a unique name and consists of:
@@ -96,6 +102,7 @@ filterRefs:: (optional) Names of filters to apply.
96
102
97
103
The order of filterRefs matters, as they are applied sequentially. Earlier filters can drop messages that will not be processed by later filters.
98
104
105
+
[id="clf-filters_6-1_{context}"]
99
106
=== Filters
100
107
101
108
Filters are configured in an array under `spec.filters`. They can match incoming log messages based on the value of structured fields and modify or drop them.
Copy file name to clipboardExpand all lines: observability/logging/logging-6.1/log6x-configuring-lokistack-otlp-6.1.adoc
+10-13Lines changed: 10 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,12 @@
1
1
:_mod-docs-content-type: ASSEMBLY
2
-
[id="log6x-configuring-lokistack-otlp-6.1"]
2
+
[id="log6x-configuring-lokistack-otlp-6-1"]
3
3
= OTLP data ingestion in Loki
4
4
include::_attributes/common-attributes.adoc[]
5
-
:context: log6x-configuring-lokistack-otlp-6.1
5
+
:context: log6x-configuring-lokistack-otlp-6-1
6
6
7
7
toc::[]
8
8
9
-
Loki 3.0 introduces an API endpoint using the OpenTelemetry Protocol (OTLP), providing a new way to ingest log entries into Loki. This endpoint complements the standard Push API, which is available in Loki from its initial release. Because OTLP is a standardized format not specifically designed for Loki, it requires additional configuration on Loki's side to map OpenTelemetry's data format to Loki's data model. OTLP lacks concepts such as _stream labels_ or _structured metadata_. Instead, OTLP provides metadata about log entries as *attributes*, grouped into three categories:
9
+
Logging 6.1 enables an API endpoint using the OpenTelemetry Protocol (OTLP). As OTLP is a standardized format not specifically designed for Loki, it requires additional configuration on Loki's side to map OpenTelemetry's data format to Loki's data model. OTLP lacks concepts such as _stream labels_ or _structured metadata_. Instead, OTLP provides metadata about log entries as *attributes*, grouped into three categories:
When the {loki-op} is set to `openshift-logging` mode, it automatically applies a default set of xref:../../../observability/logging/logging-6.1/log6x-opentelemetry-data-model-6.1.adoc#log6x-opentelemetry-data-model-6.1[attribute mappings]. These mappings align specific OTLP attributes with Loki's stream labels and structured metadata.
22
+
When the {loki-op} is set to `openshift-logging` mode, it automatically applies a default set of attribute mappings. These mappings align specific OTLP attributes with Loki's stream labels and structured metadata.
23
23
24
24
For typical setups, these default mappings should be sufficient. However, you might need to customize attribute mapping in the following cases:
25
25
26
26
* Using a custom Collector: If your setup includes a custom collector that generates additional attributes, consider customizing the mapping to ensure these attributes are retained in Loki.
27
27
* Adjusting attribute detail levels: If the default attribute set is more detailed than necessary, you can reduce it to essential attributes only. This can avoid excessive data storage and streamline the {logging} process.
28
28
29
-
For more infomation on configuring custom attribute mappings, see xref:../../../observability/logging/logging-6.1/log6x-configuring-lokistack-otlp-6.1.adoc#custom-attribute-mapping-for-openshift_log6x-configuring-lokistack-otlp-6.1[Custom attribute mapping for OpenShift].
30
-
31
29
[IMPORTANT]
32
30
====
33
31
Attributes that are not mapped to either stream labels or structured metadata are not stored in Loki.
@@ -36,9 +34,9 @@ Attributes that are not mapped to either stream labels or structured metadata ar
When using the {loki-op} in `openshift-logging` mode, attribute mapping follow OpenShift defaults, but custom mappings can be configured to adjust these. Custom mappings allow further configurations to meet specific needs.
37
+
When using the {loki-op} in `openshift-logging` mode, attribute mapping follow OpenShift defaults, but custom mappings can be configured to adjust these. Custom mappings allow further configurations to meet specific needs.
40
38
41
-
In `openshift-logging` mode, custom attribute mappings can be configured globally for all tenants or for individual tenants as needed. When custom mappings are defined, they are appended to the OpenShift defaults. If default recommended labels are not required, they can be disabled in the tenant configuration. For more information, see xref:../../../observability/logging/logging-6.1/log6x-configuring-lokistack-otlp-6.1.adoc#customizing-openshift-defaults_log6x-configuring-lokistack-otlp-6.1[Customizing OpenShift defaults].
39
+
In `openshift-logging` mode, custom attribute mappings can be configured globally for all tenants or for individual tenants as needed. When custom mappings are defined, they are appended to the OpenShift defaults. If default recommended labels are not required, they can be disabled in the tenant configuration.
42
40
43
41
[NOTE]
44
42
====
@@ -53,10 +51,10 @@ Within `LokiStack`, attribute mapping configuration is managed through the `limi
53
51
spec:
54
52
limits:
55
53
global:
56
-
otlp: {} # <1>
54
+
otlp: {} # <1>
57
55
tenants:
58
56
application:
59
-
otlp: {} # <2>
57
+
otlp: {} # <2>
60
58
----
61
59
<1> Global OTLP attribute configuration.
62
60
<2> OTLP attribute configuration for the `application` tenant within `openshift-logging` mode.
@@ -105,7 +103,7 @@ spec:
105
103
106
104
[TIP]
107
105
====
108
-
Use regular expressions by setting `regex: true` for attributes names when mapping similar attributes in Loki.
106
+
Use regular expressions by setting `regex: true` for attributes names when mapping similar attributes in Loki.
109
107
====
110
108
111
109
[IMPORTANT]
@@ -116,7 +114,7 @@ Avoid using regular expressions for stream labels, as this can increase data vol
116
114
[id="customizing-openshift-defaults_{context}"]
117
115
=== Customizing OpenShift defaults
118
116
119
-
In `openshift-logging` mode, certain attributes are required and cannot be removed from the configuration due to their role in OpenShift functions. Other attributes, labeled *recommended*, might be disabled if performance is impacted. For a complete attribute list, see xref:../../../observability/logging/logging-6.1/log6x-opentelemetry-data-model-6.1.adoc#log6x-opentelemetry-data-model-6.1[OpenShift default attributes].
117
+
In `openshift-logging` mode, certain attributes are required and cannot be removed from the configuration due to their role in OpenShift functions. Other attributes, labeled *recommended*, might be disabled if performance is impacted.
120
118
121
119
When using the `openshift-logging` mode without custom attributes, you can achieve immediate compatibility with OpenShift tools. If additional attributes are needed as stream labels or structured metadata, use custom configuration. Custom configurations can merge with default configurations.
122
120
@@ -148,4 +146,3 @@ This option is beneficial if the default attributes causes performance or storag
0 commit comments