Skip to content

Commit 90e14a8

Browse files
authored
Merge pull request #81212 from max-cx/OBSDOCS-957
OBSDOCS-957: Add Technology Preview docs for Loki Exporter to RH Collector
2 parents 25b8a58 + ddc03a5 commit 90e14a8

File tree

6 files changed

+231
-8
lines changed

6 files changed

+231
-8
lines changed

_topic_maps/_topic_map.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2964,8 +2964,8 @@ Topics:
29642964
File: otel-sending-traces-and-metrics-to-otel-collector
29652965
- Name: Configuring metrics for the monitoring stack
29662966
File: otel-configuring-metrics-for-monitoring-stack
2967-
- Name: Forwarding traces to a TempoStack
2968-
File: otel-forwarding
2967+
- Name: Forwarding telemetry data
2968+
File: otel-forwarding-telemetry-data
29692969
- Name: Configuring the Collector metrics
29702970
File: otel-configuring-otelcol-metrics
29712971
- Name: Gathering the observability data from multiple clusters
Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
//Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-forwarding-data.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="otel-forwarding-logs-to-tempostack_{context}"]
7+
= Forwarding logs to a LokiStack instance
8+
9+
You can deploy the OpenTelemetry Collector with Collector components to forward logs to a LokiStack instance.
10+
11+
This use of the Loki Exporter is a temporary Technology Preview feature that is planned to be replaced with the publication of an improved solution in which the Loki Exporter is replaced with the OTLP HTTP Exporter.
12+
13+
:FeatureName: The Loki Exporter
14+
include::snippets/technology-preview.adoc[leveloffset=+1]
15+
16+
.Prerequisites
17+
18+
* The {OTELOperator} is installed.
19+
* The {loki-op} is installed.
20+
* A supported LokiStack instance is deployed on the cluster.
21+
22+
.Procedure
23+
24+
. Create a service account for the OpenTelemetry Collector.
25+
+
26+
.Example `ServiceAccount` object
27+
[source,yaml]
28+
----
29+
apiVersion: v1
30+
kind: ServiceAccount
31+
metadata:
32+
name: otel-collector-deployment
33+
namespace: openshift-logging
34+
----
35+
36+
. Create a cluster role that grants the Collector's service account the permissions to push logs to the LokiStack application tenant.
37+
+
38+
.Example `ClusterRole` object
39+
[source,yaml]
40+
----
41+
apiVersion: rbac.authorization.k8s.io/v1
42+
kind: ClusterRole
43+
metadata:
44+
name: otel-collector-logs-writer
45+
rules:
46+
- apiGroups: ["loki.grafana.com"]
47+
resourceNames: ["logs"]
48+
resources: ["application"]
49+
verbs: ["create"]
50+
- apiGroups: [""]
51+
resources: ["pods", "namespaces", "nodes"]
52+
verbs: ["get", "watch", "list"]
53+
- apiGroups: ["apps"]
54+
resources: ["replicasets"]
55+
verbs: ["get", "list", "watch"]
56+
- apiGroups: ["extensions"]
57+
resources: ["replicasets"]
58+
verbs: ["get", "list", "watch"]
59+
----
60+
61+
. Bind the cluster role to the service account.
62+
+
63+
.Example `ClusterRoleBinding` object
64+
[source,yaml]
65+
----
66+
apiVersion: rbac.authorization.k8s.io/v1
67+
kind: ClusterRoleBinding
68+
metadata:
69+
name: otel-collector-logs-writer
70+
roleRef:
71+
apiGroup: rbac.authorization.k8s.io
72+
kind: ClusterRole
73+
name: otel-collector-logs-writer
74+
subjects:
75+
- kind: ServiceAccount
76+
name: otel-collector-deployment
77+
namespace: openshift-logging
78+
----
79+
80+
. Create an `OpenTelemetryCollector` custom resource (CR) object.
81+
+
82+
.Example `OpenTelemetryCollector` CR object
83+
[source,yaml]
84+
----
85+
apiVersion: opentelemetry.io/v1beta1
86+
kind: OpenTelemetryCollector
87+
metadata:
88+
name: otel
89+
namespace: openshift-logging
90+
spec:
91+
serviceAccount: otel-collector-deployment
92+
config:
93+
extensions:
94+
bearertokenauth:
95+
filename: "/var/run/secrets/kubernetes.io/serviceaccount/token"
96+
receivers:
97+
otlp:
98+
protocols:
99+
grpc: {}
100+
http: {}
101+
processors:
102+
k8sattributes:
103+
auth_type: "serviceAccount"
104+
passthrough: false
105+
extract:
106+
metadata:
107+
- k8s.pod.name
108+
- k8s.container.name
109+
- k8s.namespace.name
110+
labels:
111+
- tag_name: app.label.component
112+
key: app.kubernetes.io/component
113+
from: pod
114+
pod_association:
115+
- sources:
116+
- from: resource_attribute
117+
name: k8s.pod.name
118+
- from: resource_attribute
119+
name: k8s.container.name
120+
- from: resource_attribute
121+
name: k8s.namespace.name
122+
- sources:
123+
- from: connection
124+
resource:
125+
attributes: # <1>
126+
- key: loki.format # <2>
127+
action: insert
128+
value: json
129+
- key: kubernetes_namespace_name
130+
from_attribute: k8s.namespace.name
131+
action: upsert
132+
- key: kubernetes_pod_name
133+
from_attribute: k8s.pod.name
134+
action: upsert
135+
- key: kubernetes_container_name
136+
from_attribute: k8s.container.name
137+
action: upsert
138+
- key: log_type
139+
value: application
140+
action: upsert
141+
- key: loki.resource.labels # <3>
142+
value: log_type, kubernetes_namespace_name, kubernetes_pod_name, kubernetes_container_name
143+
action: insert
144+
transform:
145+
log_statements:
146+
- context: log
147+
statements:
148+
- set(attributes["level"], ConvertCase(severity_text, "lower"))
149+
exporters:
150+
loki:
151+
endpoint: https://logging-loki-gateway-http.openshift-logging.svc.cluster.local:8080/api/logs/v1/application/loki/api/v1/push # <4>
152+
tls:
153+
ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt"
154+
auth:
155+
authenticator: bearertokenauth
156+
debug:
157+
verbosity: detailed
158+
service:
159+
extensions: [bearertokenauth] # <5>
160+
pipelines:
161+
logs:
162+
receivers: [otlp]
163+
processors: [k8sattributes, transform, resource]
164+
exporters: [loki] # <6>
165+
logs/test:
166+
receivers: [otlp]
167+
processors: []
168+
exporters: [debug]
169+
----
170+
<1> Provides the following resource attributes to be used by the web console: `kubernetes_namespace_name`, `kubernetes_pod_name`, `kubernetes_container_name`, and `log_type`. If you specify them as values for this `loki.resource.labels` attribute, then the Loki Exporter processes them as labels.
171+
<2> Configures the format of Loki logs. Supported values are `json`, `logfmt` and `raw`.
172+
<3> Configures which resource attributes are processed as Loki labels.
173+
<4> Points the Loki Exporter to the gateway of the LokiStack `logging-loki` instance and uses the `application` tenant.
174+
<5> Enables the BearerTokenAuth Extension that is required by the Loki Exporter.
175+
<6> Enables the Loki Exporter to export logs from the Collector.
176+
177+
[TIP]
178+
====
179+
You can deploy `telemetrygen` as a test:
180+
181+
[source,yaml]
182+
----
183+
apiVersion: batch/v1
184+
kind: Job
185+
metadata:
186+
name: telemetrygen
187+
spec:
188+
template:
189+
spec:
190+
containers:
191+
- name: telemetrygen
192+
image: ghcr.io/open-telemetry/opentelemetry-collector-contrib/telemetrygen:v0.106.1
193+
args:
194+
- logs
195+
- --otlp-endpoint=otel-collector.openshift-logging.svc.cluster.local:4317
196+
- --otlp-insecure
197+
- --duration=180s
198+
- --workers=1
199+
- --logs=10
200+
- --otlp-attributes=k8s.container.name="telemetrygen"
201+
restartPolicy: Never
202+
backoffLimit: 4
203+
----
204+
====

observability/otel/otel-forwarding.adoc renamed to modules/otel-forwarding-traces.adoc

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1-
:_mod-docs-content-type: ASSEMBLY
2-
[id="otel-forwarding-traces"]
1+
//Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-forwarding-data.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="otel-forwarding-traces_{context}"]
37
= Forwarding traces to a TempoStack instance
4-
include::_attributes/common-attributes.adoc[]
5-
:context: otel-forwarding-traces
68

79
To configure forwarding traces to a TempoStack instance, you can deploy and configure the OpenTelemetry Collector. You can deploy the OpenTelemetry Collector in the deployment mode by using the specified processors, receivers, and exporters. For other modes, see the OpenTelemetry Collector documentation linked in _Additional resources_.
810

observability/distr_tracing/distr-tracing-rn.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ toc::[]
88

99
include::modules/distr-tracing-product-overview.adoc[leveloffset=+1]
1010

11-
You can use the {TempoName} xref:../otel/otel-forwarding.adoc#otel-forwarding-traces[in combination with] the xref:../otel/otel-rn.adoc#otel_rn[{OTELName}].
11+
You can use the {TempoName} xref:../otel/otel-forwarding-telemetry-data.adoc#otel-forwarding-telemetry-data[in combination with] the xref:../otel/otel-rn.adoc#otel_rn[{OTELName}].
1212

1313
include::snippets/distr-tracing-and-otel-disclaimer-about-docs-for-supported-features-only.adoc[]
1414

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="otel-forwarding-telemetry-data"]
3+
= Forwarding telemetry data
4+
include::_attributes/common-attributes.adoc[]
5+
:context: otel-forwarding-telemetry-data
6+
7+
toc::[]
8+
9+
You can use the OpenTelemetry Collector to forward your telemetry data.
10+
11+
include::modules/otel-forwarding-traces.adoc[leveloffset=+1]
12+
include::modules/otel-forwarding-logs-to-tempostack.adoc[leveloffset=+1]
13+
14+
[role="_additional-resources"]
15+
.Additional resources
16+
17+
* xref:../logging/log_storage/installing-log-storage.adoc#installing-log-storage[Installing LokiStack log storage]

observability/otel/otel-rn.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ toc::[]
88

99
include::modules/otel-product-overview.adoc[leveloffset=+1]
1010

11-
You can use the {OTELName} xref:otel-forwarding.adoc#otel-forwarding-traces[in combination with] the xref:../distr_tracing/distr-tracing-rn.adoc#distr-tracing-rn[{TempoName}].
11+
You can use the {OTELName} xref:otel-forwarding-telemetry-data.adoc#otel-forwarding-telemetry-data[in combination with] the xref:../distr_tracing/distr-tracing-rn.adoc#distr-tracing-rn[{TempoName}].
1212

1313
include::snippets/distr-tracing-and-otel-disclaimer-about-docs-for-supported-features-only.adoc[]
1414

0 commit comments

Comments
 (0)