OSDOCS-14941 [NETOBSERV] Update API/CLI references

Author: gwynnemonahan

modules/network-observability-con_filter-network-flows-at-ingestion.adoc

@@ -78,4 +78,6 @@ spec:
         sampling: 10  <2>
 ----
 <1> Sends matching flows to a specific output, such as Loki, Prometheus, or an external system. When omitted, sends to all configured outputs.
-<2> Optional. Applies a sampling ratio to limit the number of matching flows to be stored or exported. For example, `sampling: 10` means 1/10 of the flows are kept.
+<2> Optional. Applies a sampling ratio to limit the number of matching flows to be stored or exported. For example, `sampling: 10` means 1/10 of the flows are kept.
+
+

modules/network-observability-flowcollector-api-specifications.adoc

@@ -154,7 +154,7 @@ is set to `eBPF`.
 
 | `type`
 | `string`
-| `type` [deprecated *] selects the flows tracing agent. Previously, this field allowed to select between `eBPF` or `IPFIX`.
+| `type` [deprecated (*)] selects the flows tracing agent. Previously, this field allowed to select between `eBPF` or `IPFIX`.
 Only `eBPF` is allowed now, so this field is deprecated and is planned for removal in a future version of the API.
 
 |===
@@ -180,7 +180,8 @@ Type::
 | `object`
 | `advanced` allows setting some aspects of the internal configuration of the eBPF agent.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk. You can also
+override the default Linux capabilities from there.
 
 | `cacheActiveTimeout`
 | `string`
@@ -205,25 +206,28 @@ Otherwise it is matched as a case-sensitive string.
 | List of additional features to enable. They are all disabled by default. Enabling additional features might have performance impacts. Possible values are: +
 
 - `PacketDrop`: Enable the packets drop flows logging feature. This feature requires mounting
-the kernel debug filesystem, so the eBPF agent pods must run as privileged.
-If the `spec.agent.ebpf.privileged` parameter is not set, an error is reported. +
+the kernel debug filesystem, so the eBPF agent pods must run as privileged via `spec.agent.ebpf.privileged`. +
 
 - `DNSTracking`: Enable the DNS tracking feature. +
 
 - `FlowRTT`: Enable flow latency (sRTT) extraction in the eBPF agent from TCP traffic. +
 
 - `NetworkEvents`: Enable the network events monitoring feature, such as correlating flows and network policies.
-This feature requires mounting the kernel debug filesystem, so the eBPF agent pods must run as privileged.
-It requires using the OVN-Kubernetes network plugin with the Observability feature. +
-IMPORTANT: This feature is available as a Technology Preview.
+This feature requires mounting the kernel debug filesystem, so the eBPF agent pods must run as privileged via `spec.agent.ebpf.privileged`.
+It requires using the OVN-Kubernetes network plugin with the Observability feature.
+IMPORTANT: This feature is available as a Technology Preview. +
 
 - `PacketTranslation`: Enable enriching flows with packet translation information, such as Service NAT. +
 
-- `EbpfManager`: Unsupported * . Use eBPF Manager to manage Network Observability eBPF programs. Pre-requisite: the eBPF Manager operator (or upstream bpfman operator) must be installed. +
+- `EbpfManager`: [Unsupported (*)]. Use eBPF Manager to manage Network Observability eBPF programs. Pre-requisite: the eBPF Manager operator (or upstream bpfman operator) must be installed. +
+
+- `UDNMapping`: Enable interfaces mapping to User Defined Networks (UDN).  +
+
+This feature requires mounting the kernel debug filesystem, so the eBPF agent pods must run as privileged via `spec.agent.ebpf.privileged`.
+It requires using the OVN-Kubernetes network plugin with the Observability feature.  +
+
+- `IPSec`, to track flows between nodes with IPsec encryption.  +
 
-- `UDNMapping`: Unsupported *. Enable interfaces mapping to User Defined Networks (UDN).  +
-This feature requires mounting the kernel debug filesystem, so the eBPF agent pods must run as privileged.
-It requires using the OVN-Kubernetes network plugin with the Observability feature.
 
 | `flowFilter`
 | `object`
@@ -255,7 +259,7 @@ Otherwise it is matched as a case-sensitive string.
 | `privileged`
 | `boolean`
 | Privileged mode for the eBPF Agent container. When ignored or set to `false`, the operator sets
-granular capabilities (BPF, PERFMON, NET_ADMIN, SYS_RESOURCE) to the container.
+granular capabilities (BPF, PERFMON, NET_ADMIN) to the container.
 If for some reason these capabilities cannot be set, such as if an old kernel version not knowing CAP_BPF
 is in use, then you can turn on this mode for more global privileges.
 Some agent features require the privileged mode, such as packet drops tracking (see `features`) and SR-IOV support.
@@ -267,7 +271,7 @@ For more information, see https://kubernetes.io/docs/concepts/configuration/mana
 
 | `sampling`
 | `integer`
-| Sampling rate of the flow reporter. 100 means one flow on 100 is sent. 0 or 1 means all flows are sampled.
+| Sampling ratio of the eBPF probe. 100 means one packet on 100 is sent. 0 or 1 means all packets are sampled.
 
 |===
 == .spec.agent.ebpf.advanced
@@ -276,7 +280,8 @@ Description::
 --
 `advanced` allows setting some aspects of the internal configuration of the eBPF agent.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk. You can also
+override the default Linux capabilities from there.
 --
 
 Type::
@@ -289,6 +294,10 @@ Type::
 |===
 | Property | Type | Description
 
+| `capOverride`
+| `array (string)`
+| Linux capabilities override, when not running as privileged. Default capabilities are BPF, PERFMON and NET_ADMIN.
+
 | `env`
 | `object (string)`
 | `env` allows passing custom environment variables to underlying components. Useful for passing
@@ -447,7 +456,7 @@ To change the default, you can define a rule that accepts everything: `{ action:
 
 | `sampling`
 | `integer`
-| `sampling` sampling rate for the matched flows, overriding the global sampling defined at `spec.agent.ebpf.sampling`.
+| `sampling` is the sampling ratio for the matched packets, overriding the global sampling defined at `spec.agent.ebpf.sampling`.
 
 | `sourcePorts`
 | `integer-or-string`
@@ -481,7 +490,7 @@ Type::
 Description::
 +
 --
-`EBPFFlowFilterRules` defines the desired eBPF agent configuration regarding flow filtering rules.
+`EBPFFlowFilterRule` defines the desired eBPF agent configuration regarding flow filtering rule.
 --
 
 Type::
@@ -549,7 +558,7 @@ To filter two ports, use a "port1,port2" in string format. For example, `ports:
 
 | `sampling`
 | `integer`
-| `sampling` sampling rate for the matched flows, overriding the global sampling defined at `spec.agent.ebpf.sampling`.
+| `sampling` is the sampling ratio for the matched packets, overriding the global sampling defined at `spec.agent.ebpf.sampling`.
 
 | `sourcePorts`
 | `integer-or-string`
@@ -661,7 +670,7 @@ If set to `true`, the `providedCaFile` field is ignored.
 | Select the type of TLS configuration: +
 
 - `Disabled` (default) to not configure TLS for the endpoint.
-- `Provided` to manually provide cert file and a key file. Unsupported *.
+- `Provided` to manually provide cert file and a key file. [Unsupported (*)].
 - `Auto` to use {product-title} auto generated certificate using annotations.
 
 |===
@@ -791,7 +800,7 @@ Type::
 | `object`
 | `advanced` allows setting some aspects of the internal configuration of the console plugin.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk.
 
 | `autoscaler`
 | `object`
@@ -833,7 +842,7 @@ Description::
 --
 `advanced` allows setting some aspects of the internal configuration of the console plugin.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk.
 --
 
 Type::
@@ -1165,7 +1174,7 @@ Required::
 
 | `sasl`
 | `object`
-| SASL authentication configuration. Unsupported *.
+| SASL authentication configuration. [Unsupported (*)].
 
 | `tls`
 | `object`
@@ -1180,7 +1189,7 @@ Required::
 Description::
 +
 --
-SASL authentication configuration. Unsupported *.
+SASL authentication configuration. [Unsupported (*)].
 --
 
 Type::
@@ -1676,7 +1685,7 @@ Required::
 
 | `sasl`
 | `object`
-| SASL authentication configuration. Unsupported *.
+| SASL authentication configuration. [Unsupported (*)].
 
 | `tls`
 | `object`
@@ -1691,7 +1700,7 @@ Required::
 Description::
 +
 --
-SASL authentication configuration. Unsupported *.
+SASL authentication configuration. [Unsupported (*)].
 --
 
 Type::
@@ -2077,7 +2086,7 @@ Type::
 
 - `Forward` forwards the user token for authorization. +
 
-- `Host` [deprecated *] - uses the local pod service account to authenticate to Loki. +
+- `Host` [deprecated (*)] - uses the local pod service account to authenticate to Loki. +
 
 When using the Loki Operator, this must be set to `Forward`.
 
@@ -2693,7 +2702,7 @@ This feature requires the "topology.kubernetes.io/zone" label to be set on nodes
 | `object`
 | `advanced` allows setting some aspects of the internal configuration of the flow processor.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk.
 
 | `clusterName`
 | `string`
@@ -2702,14 +2711,12 @@ such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
 | `deduper`
 | `object`
 | `deduper` allows you to sample or drop flows identified as duplicates, in order to save on resource usage.
-Unsupported *.
 
 | `filters`
 | `array`
 | `filters` lets you define custom filters to limit the amount of generated flows.
 These filters provide more flexibility than the eBPF Agent filters (in `spec.agent.ebpf.flowFilter`), such as allowing to filter by Kubernetes namespace,
 but with a lesser improvement in performance.
-Unsupported *.
 
 | `imagePullPolicy`
 | `string`
@@ -2743,9 +2750,9 @@ This setting is ignored when Kafka is disabled.
 
 - `Flows` to export regular network flows. This is the default. +
 
-- `Conversations` to generate events for started conversations, ended conversations as well as periodic "tick" updates. +
+- `Conversations` to generate events for started conversations, ended conversations as well as periodic "tick" updates. Note that in this mode, Prometheus metrics are not accurate on long-standing conversations. +
 
-- `EndedConversations` to generate only ended conversations events. +
+- `EndedConversations` to generate only ended conversations events. Note that in this mode, Prometheus metrics are not accurate on long-standing conversations. +
 
 - `All` to generate both network flows and all conversations events. It is not recommended due to the impact on resources footprint. +
 
@@ -2775,7 +2782,7 @@ Description::
 --
 `advanced` allows setting some aspects of the internal configuration of the flow processor.
 This section is aimed mostly for debugging and fine-grained performance optimizations,
-such as `GOGC` and `GOMAXPROCS` env vars. Set these values at your own risk.
+such as `GOGC` and `GOMAXPROCS` environment vars. Set these values at your own risk.
 --
 
 Type::
@@ -2803,7 +2810,7 @@ This delay is ignored when a FIN packet is collected for TCP flows (see `convers
 
 | `dropUnusedFields`
 | `boolean`
-| `dropUnusedFields` [deprecated *] this setting is not used anymore.
+| `dropUnusedFields` [deprecated (*)] this setting is not used anymore.
 
 | `enableKubeProbes`
 | `boolean`
@@ -2910,7 +2917,8 @@ Description::
 +
 --
 Defines secondary networks to be checked for resources identification.
-To guarantee a correct identification, indexed values must form an unique identifier across the cluster. If the same index is used by several resources, those resources might be incorrectly labeled.
+To guarantee a correct identification, indexed values must form an unique identifier across the cluster.
+If the same index is used by several resources, those resources might be incorrectly labeled.
 --
 
 Type::
@@ -2955,7 +2963,6 @@ Description::
 +
 --
 `deduper` allows you to sample or drop flows identified as duplicates, in order to save on resource usage.
-Unsupported *.
 --
 
 Type::
@@ -2970,7 +2977,7 @@ Type::
 
 | `mode`
 | `string`
-| Set the Processor de-duplication mode. It comes in addition to the Agent-based deduplication because the Agent cannot de-duplicate same flows reported from different nodes. +
+| Set the Processor de-duplication mode. It comes in addition to the Agent-based deduplication, since the Agent cannot de-duplicate same flows reported from different nodes. +
 
 - Use `Drop` to drop every flow considered as duplicates, allowing saving more on resource usage but potentially losing some information such as the network interfaces used from peer, or network events. +
 
@@ -2981,7 +2988,7 @@ Type::
 
 | `sampling`
 | `integer`
-| `sampling` is the sampling rate when deduper `mode` is `Sample`.
+| `sampling` is the sampling ratio when deduper `mode` is `Sample`. For example, a value of `50` means that 1 flow in 50 is sampled.
 
 |===
 == .spec.processor.filters
@@ -2991,7 +2998,6 @@ Description::
 `filters` lets you define custom filters to limit the amount of generated flows.
 These filters provide more flexibility than the eBPF Agent filters (in `spec.agent.ebpf.flowFilter`), such as allowing to filter by Kubernetes namespace,
 but with a lesser improvement in performance.
-Unsupported *.
 --
 
 Type::
@@ -3017,64 +3023,17 @@ Type::
 |===
 | Property | Type | Description
 
-| `allOf`
-| `array`
-| `filters` is a list of matches that must be all satisfied in order to remove a flow.
-
 | `outputTarget`
 | `string`
-| If specified, these filters only target a single output: `Loki`, `Metrics` or `Exporters`. By default, all outputs are targeted.
-
-| `sampling`
-| `integer`
-| `sampling` is an optional sampling rate to apply to this filter.
-
-|===
-== .spec.processor.filters[].allOf
-Description::
-+
---
-`filters` is a list of matches that must be all satisfied in order to remove a flow.
---
-
-Type::
-  `array`
-
-
-
-
-== .spec.processor.filters[].allOf[]
-Description::
-+
---
-`FLPSingleFilter` defines the desired configuration for a single FLP-based filter.
---
-
-Type::
-  `object`
-
-Required::
-  - `field`
-  - `matchType`
-
-
-
-[cols="1,1,1",options="header"]
-|===
-| Property | Type | Description
-
-| `field`
-| `string`
-| Name of the field to filter on.
-Refer to the documentation for the list of available fields: https://github.com/netobserv/network-observability-operator/blob/main/docs/flows-format.adoc.
+| If specified, these filters target a single output: `Loki`, `Metrics` or `Exporters`. By default, all outputs are targeted.
 
-| `matchType`
+| `query`
 | `string`
-| Type of matching to apply.
+| A query that selects the network flows to keep. More information about this query language in https://github.com/netobserv/flowlogs-pipeline/blob/main/docs/filtering.md.
 
-| `value`
-| `string`
-| Value to filter on. When `matchType` is `Equal` or `NotEqual`, you can use field injection with `$(SomeField)` to refer to any other field of the flow.
+| `sampling`
+| `integer`
+| `sampling` is an optional sampling ratio to apply to this filter. For example, a value of `50` means that 1 matching flow in 50 is sampled.
 
 |===
 == .spec.processor.kafkaConsumerAutoscaler
@@ -3199,7 +3158,7 @@ If set to `true`, the `providedCaFile` field is ignored.
 | Select the type of TLS configuration: +
 
 - `Disabled` (default) to not configure TLS for the endpoint.
-- `Provided` to manually provide cert file and a key file. Unsupported *.
+- `Provided` to manually provide cert file and a key file. [Unsupported (*)].
 - `Auto` to use {product-title} auto generated certificate using annotations.
 
 |===
@@ -3593,4 +3552,4 @@ If the namespace is different, the config map or the secret is copied so that it
 | `string`
 | Type for the certificate reference: `configmap` or `secret`.
 
-|===
+|===