Skip to content

Commit 00b251e

Browse files
kelbrown20kelbrown20
authored
Merge pull request #73318 from shreyasiddhartha/OBSDOCS-821
OBSDOCS-821: Docs for log filtering by metadata
2 parents 6532038 + 6427151 commit 00b251e

7 files changed

+204
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2779,6 +2779,8 @@ Topics:
27792779
File: logging-flow-control-mechanisms
27802780
# - Name: Filtering logs by content
27812781
# File: logging-content-filtering
2782+
# - Name: Filtering logs by metadata
2783+
# File: logging-input-spec-filtering
27822784
- Name: Scheduling resources
27832785
Dir: scheduling_resources
27842786
Topics:

‎_topic_maps/_topic_map_osd.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1212,6 +1212,8 @@ Topics:
12121212
File: logging-flow-control-mechanisms
12131213
# - Name: Filtering logs by content
12141214
# File: logging-content-filtering
1215+
# - Name: Filtering logs by metadata
1216+
# File: logging-input-spec-filtering
12151217
- Name: Scheduling resources
12161218
Dir: scheduling_resources
12171219
Topics:

‎_topic_maps/_topic_map_rosa.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1454,6 +1454,8 @@ Topics:
14541454
File: logging-flow-control-mechanisms
14551455
# - Name: Filtering logs by content
14561456
# File: logging-content-filtering
1457+
# - Name: Filtering logs by metadata
1458+
# File: logging-input-spec-filtering
14571459
- Name: Scheduling resources
14581460
Dir: scheduling_resources
14591461
Topics:

‎logging/performance_reliability/logging-input-spec-filtering.adoc

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
include::_attributes/common-attributes.adoc[]
3+
include::_attributes/attributes-openshift-dedicated.adoc[]
4+
[id="logging-input-spec-filtering"]
5+
= Filtering logs by metadata
6+
:context: logging-input-spec-filtering
7+
8+
toc::[]
9+
10+
11+
You can filter logs in the `ClusterLogForwarder` CR to select or ignore an entire log stream based on the metadata by using the `input` selector. As an administrator or developer, you can include or exclude the log collection to reduce the memory and CPU load on the collector.
12+
13+
[IMPORTANT]
14+
====
15+
You can use this feature only if the Vector collector is set up in your logging deployment.
16+
====
17+
18+
[NOTE]
19+
====
20+
`input` spec filtering is different from content filtering. `input` selectors select or ignore entire log streams based on the source metadata. Content filters edit the log streams to remove and modify the records based on the record content.
21+
====
22+
23+
include::modules/logging-input-spec-filter-namespace-container.adoc[leveloffset=+1]
24+
include::modules/logging-input-spec-filter-labels-expressions.adoc[leveloffset=+1]
25+
include::modules/logging-input-spec-filter-audit-infrastructure.adoc[leveloffset=+1]

‎modules/logging-input-spec-filter-audit-infrastructure.adoc

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/performance_reliability/logging-input-spec-filtering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="logging-input-spec-filter-audit-infrastructure_{context}"]
7+
= Filtering the audit and infrastructure log inputs by source
8+
9+
You can define the list of `audit` and `infrastructure` sources to collect the logs by using the `input` selector.
10+
11+
.Prerequisites
12+
13+
* You have installed the {clo}.
14+
* You have administrator permissions.
15+
* You have created a `ClusterLogForwarder` custom resource (CR).
16+
17+
.Procedure
18+
19+
. Add a configuration to define the `audit` and `infrastructure` sources in the `ClusterLogForwarder` CR.
20+
21+
+
22+
The following example shows how to configure the `ClusterLogForwarder` CR to define `aduit` and `infrastructure` sources:
23+
+
24+
.Example `ClusterLogForwarder` CR
25+
+
26+
[source,yaml]
27+
----
28+
apiVersion: "logging.openshift.io/v1"
29+
kind: ClusterLogForwarder
30+
# ...
31+
spec:
32+
inputs:
33+
- name: mylogs1
34+
infrastructure:
35+
sources: # <1>
36+
- node
37+
- name: mylogs2
38+
audit:
39+
sources: # <2>
40+
- kubeAPI
41+
- openshiftAPI
42+
- ovn
43+
# ...
44+
----
45+
<1> Specifies the list of infrastructure sources to collect. The valid sources include:
46+
** `node`: Journal log from the node
47+
** `container`: Logs from the workloads deployed in the namespaces
48+
<2> Specifies the list of audit sources to collect. The valid sources include:
49+
** `kubeAPI`: Logs from the Kubernetes API servers
50+
** `openshiftAPI`: Logs from the OpenShift API servers
51+
** `auditd`: Logs from a node auditd service
52+
** `ovn`: Logs from an open virtual network service
53+
54+
. Apply the `ClusterLogForwarder` CR by running the following command:
55+
56+
+
57+
[source,terminal]
58+
----
59+
$ oc apply -f <filename>.yaml
60+
----

‎modules/logging-input-spec-filter-labels-expressions.adoc

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/performance_reliability/logging-input-spec-filtering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="logging-input-spec-filter-labels-expressions_{context}"]
7+
= Filtering application logs at input ny including the label expressions or a matching label key and values
8+
9+
You can include the application logs based on the label expressions or a matching label key and its values by using the `input` selector.
10+
11+
.Prerequisites
12+
13+
* You have installed the {clo}.
14+
* You have administrator permissions.
15+
* You have created a `ClusterLogForwarder` custom resource (CR).
16+
17+
.Procedure
18+
19+
. Add a configuration for a filter to the `input` spec in the `ClusterLogForwarder` CR.
20+
+
21+
The following example shows how to configure the `ClusterLogForwarder` CR to include logs based on label expressions or matched label key/values:
22+
+
23+
.Example `ClusterLogForwarder` CR
24+
[source,yaml]
25+
----
26+
apiVersion: "logging.openshift.io/v1"
27+
kind: ClusterLogForwarder
28+
# ...
29+
spec:
30+
inputs:
31+
- name: mylogs
32+
application:
33+
selector:
34+
matchExpressions:
35+
- key: env # <1>
36+
operator: In # <2>
37+
values: [“prod”, “qa”] # <3>
38+
- key: zone
39+
operator: NotIn
40+
values: [“east”, “west”]
41+
matchLabels: # <4>
42+
app: one
43+
name: app1
44+
# ...
45+
----
46+
<1> Specifies the label key to match.
47+
<2> Specifies the operator. Valid values include: `In`, `NotIn`, `Exists`, and `DoesNotExist`.
48+
<3> Specifies an array of string values. If the `operator` value is either `Exists` or `DoesNotExist`, the value array must be empty.
49+
<4> Specifies an exact key or value mapping.
50+
51+
. Apply the `ClusterLogForwarder` CR by running the following command:
52+
53+
+
54+
[source,terminal]
55+
----
56+
$ oc apply -f <filename>.yaml
57+
----

‎modules/logging-input-spec-filter-namespace-container.adoc

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/performance_reliability/logging-input-spec-filtering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="logging-input-spec-filter-namespace-container_{context}"]
7+
= Filtering application logs at input by including or excluding the namespace or container name
8+
9+
You can include or exclude the application logs based on the namespace and container name by using the `input` selector.
10+
11+
.Prerequisites
12+
13+
* You have installed the {clo}.
14+
* You have administrator permissions.
15+
* You have created a `ClusterLogForwarder` custom resource (CR).
16+
17+
.Procedure
18+
19+
. Add a configuration to include or exclude the namespace and container names in the `ClusterLogForwarder` CR.
20+
+
21+
The following example shows how to configure the `ClusterLogForwarder` CR to include or exclude namespaces and container names:
22+
+
23+
.Example `ClusterLogForwarder` CR
24+
[source,yaml]
25+
----
26+
apiVersion: "logging.openshift.io/v1"
27+
kind: ClusterLogForwarder
28+
# ...
29+
spec:
30+
inputs:
31+
- name: mylogs
32+
application:
33+
includes:
34+
- namespace: "my-project" # <1>
35+
container: "my-container" # <2>
36+
excludes:
37+
- container: "other-container*" # <3>
38+
namespace: "other-namespace" # <4>
39+
# ...
40+
----
41+
<1> Specifies that the logs are only collected from these namespaces.
42+
<2> Specifies that the logs are only collected from these containers.
43+
<3> Specifies the pattern of namespaces to ignore when collecting the logs.
44+
<4> Specifies the set of containers to ignore when collecting the logs.
45+
46+
. Apply the `ClusterLogForwarder` CR by running the following command:
47+
48+
+
49+
[source,terminal]
50+
----
51+
$ oc apply -f <filename>.yaml
52+
----
53+
[NOTE]
54+
====
55+
The `excludes` option takes precedence over `includes`.
56+
====

0 commit comments

Comments
 (0)