Merge pull request #95467 from eromanova97/OBSDOCS-2053

OBSDOCS-2053: fix 'Configuring content filters to drop unwanted log records' and remove duplicate

Author: kcarmichael08

configuring/configuring-log-forwarding.adoc

@@ -116,7 +116,7 @@ Administrators can configure the following types of filters:
 include::modules/enabling-multi-line-exception-detection.adoc[leveloffset=+2]
 include::modules/logging-http-forward.adoc[leveloffset=+2]
 include::modules/cluster-logging-collector-log-forward-syslog.adoc[leveloffset=+2]
-include::modules/content-filter-drop-records.adoc[leveloffset=+2]
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+2]
 include::modules/logging-audit-log-filtering.adoc[leveloffset=+2]
 include::modules/input-spec-filter-labels-expressions.adoc[leveloffset=+2]
 include::modules/logging-content-filter-prune-records.adoc[leveloffset=+2]

modules/content-filter-drop-records.adoc

@@ -6,103 +6,123 @@
 [id="logging-content-filter-drop-records_{context}"]
 = Configuring content filters to drop unwanted log records
 
-When the `drop` filter is configured, the log collector evaluates log streams according to the filters before forwarding. The collector drops unwanted log records that match the specified configuration.
+Collecting all cluster logs produces a large amount of data, which can be expensive to move and store. To reduce volume, you can configure the `drop` filter to exclude unwanted log records before forwarding. The log collector evaluates log streams against the filter and drops records that match specified conditions.
+
+The `drop` filter uses the `test` field to define one or more conditions for evaluating log records.  
+The filter applies the following rules to check whether to drop a record:
+
+* A test passes if all its specified conditions evaluate to true.
+* If a test passes, the filter drops the log record.
+* If you define several tests in the `drop` filter configuration, the filter drops the log record if any of the tests pass.
+* If there is an error evaluating a condition, for example, the referenced field is missing, that condition evaluates to false.
 
 .Prerequisites
 
 * You have installed the {clo}.
 * You have administrator permissions.
 * You have created a `ClusterLogForwarder` custom resource (CR).
+* You have installed the {oc-first}.
 
 .Procedure
 
-. Add a configuration for a filter to the `filters` spec in the `ClusterLogForwarder` CR.
+. Extract the existing `ClusterLogForwarder` configuration and save it as a local file.
++
+[source,terminal]
+----
+$ oc get clusterlogforwarder <name> -n <namespace> -o yaml > <filename>.yaml
+----
++
+Where:
 +
-The following example shows how to configure the `ClusterLogForwarder` CR to drop log records based on regular expressions:
+* `<name>` is the name of the `ClusterLogForwarder` instance you want to configure.
+* `<namespace>` is the namespace where you created the `ClusterLogForwarder` instance, for example `openshift-logging`.
+* `<filename>` is the name of the local file where you save the configuration.
+
+. Add a configuration to drop unwanted log records to the `filters` spec in the `ClusterLogForwarder` CR.
 +
+--
 .Example `ClusterLogForwarder` CR
 [source,yaml]
 ----
-apiVersion: logging.openshift.io/v1
+apiVersion: observability.openshift.io/v1
 kind: ClusterLogForwarder
 metadata:
-# ...
+  name: instance
+  namespace: openshift-logging
 spec:
+  # ...
   filters:
-  - name: <filter_name>
+  - name: drop-filter
     type: drop # <1>
     drop: # <2>
     - test: # <3>
-      - field: .kubernetes.labels."foo-bar/baz" # <4>
+      - field: .kubernetes.labels."app.version-1.2/beta" # <4>
         matches: .+ # <5>
       - field: .kubernetes.pod_name
         notMatches: "my-pod" # <6>
   pipelines:
-  - name: <pipeline_name> # <7>
-    filterRefs: ["<filter_name>"]
-# ...
+  - name: my-pipeline # <7>
+    filterRefs:
+    - drop-filter
+  # ...
 ----
-<1> Specifies the type of filter. The `drop` filter drops log records that match the filter configuration.
-<2> Specifies configuration options for applying the `drop` filter.
-<3> Specifies the configuration for tests that are used to evaluate whether a log record is dropped.
-** If all the conditions specified for a test are true, the test passes and the log record is dropped.
-** When multiple tests are specified for the `drop` filter configuration, if any of the tests pass, the record is dropped.
-** If there is an error evaluating a condition, for example, the field is missing from the log record being evaluated, that condition evaluates to false.
-<4> Specifies a dot-delimited field path, which is a path to a field in the log record. The path can contain alpha-numeric characters and underscores (`a-zA-Z0-9_`), for example, `.kubernetes.namespace_name`. If segments contain characters outside of this range, the segment must be in quotes, for example, `.kubernetes.labels."foo.bar-bar/baz"`. You can include multiple field paths in a single `test` configuration, but they must all evaluate to true for the test to pass and the `drop` filter to be applied.
-<5> Specifies a regular expression. If log records match this regular expression, they are dropped. You can set either the `matches` or `notMatches` condition for a single `field` path, but not both.
-<6> Specifies a regular expression. If log records do not match this regular expression, they are dropped. You can set either the `matches` or `notMatches` condition for a single `field` path, but not both.
-<7> Specifies the pipeline that the `drop` filter is applied to.
-
-. Apply the `ClusterLogForwarder` CR by running the following command:
+<1> Specify the type of filter. The `drop` filter drops log records that match the filter configuration.
+<2> Specify configuration options for the `drop` filter.
+<3> Specify conditions for tests to evaluate whether the filter drops a log record.
+<4> Specify dot-delimited paths to fields in log records. 
+** Each path segment can contain alphanumeric characters and underscores, `a-z`, `A-Z`, `0-9`, `_`, for example, `.kubernetes.namespace_name`. 
+** If segments contain different characters, the segment must be in quotes, for example, `.kubernetes.labels."app.version-1.2/beta"`. 
+** You can include several field paths in a single `test` configuration, but they must all evaluate to true for the test to pass and the `drop` filter to apply.
+<5> Specify a regular expression. If log records match this regular expression, they are dropped.
+<6> Specify a regular expression. If log records do not match this regular expression, they are dropped. 
+<7> Specify the pipeline that uses the `drop` filter.
+--
 +
-[source,terminal]
-----
-$ oc apply -f <filename>.yaml
-----
-
-.Additional examples
-
-The following additional example shows how you can configure the `drop` filter to only keep higher priority log records:
-
+[NOTE]
+====
+You can set either the `matches` or `notMatches` condition for a single `field` path, but not both.
+====
++
+.Example configuration that keeps only high-priority log records
 [source,yaml]
 ----
-apiVersion: logging.openshift.io/v1
-kind: ClusterLogForwarder
-metadata:
 # ...
-spec:
-  filters:
-  - name: important
-    type: drop
-    drop:
-      test:
-      - field: .message
-        notMatches: "(?i)critical|error"
-      - field: .level
-        matches: "info|warning"
+filters:
+- name: important
+  type: drop
+  drop:
+  - test:
+    - field: .message
+      notMatches: "(?i)critical|error"
+    - field: .level
+      matches: "info|warning"
 # ...
 ----
-
-In addition to including multiple field paths in a single `test` configuration, you can also include additional tests that are treated as _OR_ checks. In the following example, records are dropped if either `test` configuration evaluates to true. However, for the second `test` configuration, both field specs must be true for it to be evaluated to true:
-
++
+.Example configuration with several tests
 [source,yaml]
 ----
-apiVersion: logging.openshift.io/v1
-kind: ClusterLogForwarder
-metadata:
 # ...
-spec:
-  filters:
-  - name: important
-    type: drop
-    drop:
-      test:
-      - field: .kubernetes.namespace_name
-        matches: "^open"
-      test:
-      - field: .log_type
-        matches: "application"
-      - field: .kubernetes.pod_name
-        notMatches: "my-pod"
+filters:
+- name: important
+  type: drop
+  drop:
+  - test: # <1>
+    - field: .kubernetes.namespace_name
+      matches: "openshift.*"
+  - test: # <2>
+    - field: .log_type
+      matches: "application"
+    - field: .kubernetes.pod_name
+      notMatches: "my-pod"
 # ...
 ----
+<1> The filter drops logs that contain a namespace that starts with `openshift`.
+<2> The filter drops application logs that do not have `my-pod` in the pod name.
+
+. Apply the `ClusterLogForwarder` CR by running the following command:
++
+[source,terminal]
+----
+$ oc apply -f <filename>.yaml
+----