Skip to content

Commit ca44410

Browse files
gabriel-rhgabriel-rh
authored
Merge pull request #81463 from gabriel-rh/OBSDOCS-1246
OBSDOCS-1246 troubleshooting UI plugin
2 parents 21d770e + ab6d685 commit ca44410

8 files changed

+183
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3072,6 +3072,8 @@ Topics:
30723072
File: dashboard-ui-plugin
30733073
- Name: Distributed tracing UI plugin
30743074
File: distributed-tracing-ui-plugin
3075+
- Name: Troubleshooting UI plugin
3076+
File: troubleshooting-ui-plugin
30753077
---
30763078
Name: Scalability and performance
30773079
Dir: scalability_and_performance

‎images/coo-troubleshooting-experimental.png

31.1 KB
Loading

‎images/coo-troubleshooting-panel-graph.png

49.7 KB
Loading

‎images/coo-troubleshooting-panel-link.png

72.6 KB
Loading

‎modules/coo-troubleshooting-ui-plugin-creating-alert.adoc

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
// Module included in the following assemblies:
2+
3+
// * observability/cluster_observability_operator/ui_plugins/troubleshooting-ui-plugin.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coo-troubleshooting-ui-plugin-creating-alert_{context}"]
7+
= Creating the example alert
8+
9+
10+
To trigger an alert as a starting point to use in the troubleshooting UI panel, you can deploy a container that is deliberately misconfigured.
11+
12+
.Procedure
13+
14+
. Use the following YAML, either from the command line or in the web console, to create a broken deployment in a system namespace:
15+
+
16+
[source,yaml]
17+
----
18+
apiVersion: apps/v1
19+
kind: Deployment
20+
metadata:
21+
name: bad-deployment
22+
namespace: default <1>
23+
spec:
24+
selector:
25+
matchLabels:
26+
app: bad-deployment
27+
template:
28+
metadata:
29+
labels:
30+
app: bad-deployment
31+
spec:
32+
containers: <2>
33+
- name: bad-deployment
34+
image: quay.io/openshift-logging/vector:5.8
35+
----
36+
<1> The deployment must be in a system namespace (such as `default`) to cause the desired alerts.
37+
<2> This container deliberately tries to start a `vector` server with no configuration file. The server logs a few messages, and then exits with an error. Alternatively, you can deploy any container you like that is badly configured, causing it to trigger an alert.
38+
39+
. View the alerts:
40+
.. Go to *Observe* -> *Alerting* and click *clear all filters*. View the `Pending` alerts.
41+
+
42+
[IMPORTANT]
43+
====
44+
Alerts first appear in the `Pending` state. They do not start `Firing` until the container has been crashing for some time. By viewing `Pending` alerts, you do not have to wait as long to see them occur.
45+
====
46+
.. Choose one of the `KubeContainerWaiting`, `KubePodCrashLooping`, or `KubePodNotReady` alerts and open the troubleshooting panel by clicking on the link. Alternatively, if the panel is already open, click the "Focus" button to update the graph.

‎modules/coo-troubleshooting-ui-plugin-install.adoc

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
// Module included in the following assemblies:
2+
3+
// * observability/cluster_observability_operator/ui_plugins/troubleshooting-ui-plugin.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coo-troubleshooting-ui-plugin-install_{context}"]
7+
= Installing the {coo-full} Troubleshooting UI plugin
8+
9+
.Prerequisites
10+
* You have access to the cluster as a user with the `cluster-admin` cluster role.
11+
* You have logged in to the {product-title} web console.
12+
* You have installed the {coo-full}
13+
14+
.Procedure
15+
. In the {product-title} web console, click *Operators* -> *Installed Operators* and select {coo-full}
16+
. Choose the *UI Plugin* tab (at the far right of the tab list) and press *Create UIPlugin*
17+
. Select *YAML view*, enter the following content, and then press *Create*:
18+
+
19+
[source,yaml]
20+
----
21+
apiVersion: observability.openshift.io/v1alpha1
22+
kind: UIPlugin
23+
metadata:
24+
name: troubleshooting-panel
25+
spec:
26+
type: TroubleshootingPanel
27+
----

‎modules/coo-troubleshooting-ui-plugin-using.adoc

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
// Module included in the following assemblies:
2+
3+
// * observability/cluster_observability_operator/ui_plugins/troubleshooting-ui-plugin.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coo-troubleshooting-ui-plugin-using_{context}"]
7+
= Using the {coo-full} troubleshooting UI plugin
8+
9+
.Prerequisites
10+
* You have access to the {product-title} cluster as a user with the `cluster-admin` cluster role. If your cluster version is 4.17+, you can access the troubleshooting UI panel from the Application Launcher {launch}.
11+
* You have logged in to the {product-title} web console.
12+
* You have installed {product-title} Logging, if you want to visualize correlated logs.
13+
* You have installed {product-title} Network Observability, if you want to visualize correlated netflows.
14+
* You have installed the {coo-full}.
15+
* You have installed the {coo-full} troubleshooting UI plugin.
16+
+
17+
[NOTE]
18+
====
19+
The troubleshooting panel relies on the observability signal stores installed in your cluster.
20+
Kuberenetes resources, alerts and metrics are always available by default in an {product-title} cluster.
21+
Other signal types require optional components to be installed:
22+
23+
* **Logs:** Red Hat Openshift Logging (collection) and Loki Operator provided by Red Hat (store)
24+
* **Network events:** Network observability provided by Red Hat (collection) and Loki Operator provided by Red Hat (store)
25+
====
26+
.Procedure
27+
28+
. In the admin perspective of the web console, navigate to *Observe* -> *Alerting* and then select an alert. If the alert has correlated items, a **Troubleshooting Panel** link will appear above the chart on the alert detail page.
29+
+
30+
image::coo-troubleshooting-panel-link.png[Troubleshooting Panel link]
31+
+
32+
Click on the **Troubleshooting Panel** link to display the panel.
33+
. The panel consists of query details and a topology graph of the query results. The selected alert is converted into a Korrel8r query string and sent to the `korrel8r` service.
34+
The results are displayed as a graph network connecting the returned signals and resources. This is a _neighbourhood_ graph, starting at the current resource and including related objects up to 3 steps away from the starting point.
35+
Clicking on nodes in the graph takes you to the corresponding web console pages for those resouces.
36+
. You can use the troubleshooting panel to find resources relating to the chosen alert.
37+
+
38+
[NOTE]
39+
====
40+
Clicking on a node may sometimes show fewer results than indicated on the graph. This is a known issue that will be addressed in a future release.
41+
====
42+
+
43+
image::coo-troubleshooting-panel-graph.png[Troubleshooting panel]
44+
[arabic]
45+
.. **Alert (1):** This node is the starting point in the graph and represents the `KubeContainerWaiting` alert displayed in the web console.
46+
47+
.. **Pod (1):** This node indicates that there is a single `Pod` resource associated with this alert. Clicking on this node will open a console search showing the related pod directly.
48+
49+
.. **Event (2):** There are two Kuberenetes events associated with the pod. Click this node to see the events.
50+
51+
.. **Logs (74):** This pod has 74 lines of logs, which you can access by clicking on this node.
52+
53+
.. **Metrics (105):** There are many metrics associated with the pod.
54+
55+
.. **Network (6):** There are network events, meaning the pod has communicated over the network. The remaining nodes in the graph represent the `Service`, `Deployment` and `DaemonSet` resources that the pod has communicated with.
56+
57+
.. **Focus:** Clicking this button updates the graph. By default, the graph itself does not change when you click on nodes in the graph. Instead, the main web console page changes, and you can then navigate to other resources using links on the page, while the troubleshooting panel itself stays open and unchanged. To force an update to the graph in the troubleshooting panel, click **Focus**. This draws a new graph, using the current resource in the web console as the starting point.
58+
59+
.. **Show Query:** Clicking this button enables some experimental features:
60+
+
61+
image::coo-troubleshooting-experimental.png[Experimental features]
62+
[arabic]
63+
... **Hide Query** hides the experimental features.
64+
65+
... The query that identifies the starting point for the graph.
66+
The query language, part of the link:https://korrel8r.github.io/korrel8r[Korrel8r] correlation engine used to create the graphs, is experimental and may change in future.
67+
The query is updated by the **Focus** button to correspond to the resources in the main web console window.
68+
69+
... **Neighbourhood depth** is used to display a smaller or larger neighbourhood.
70+
+
71+
[NOTE]
72+
====
73+
Setting a large value in a large cluster might cause the query to fail, if the number of results is too big.
74+
====
75+
... **Goal class** results in a goal directed search instead of a neighbourhood search. A goal directed search shows all paths from the starting point to the goal class, which indicates a type of resource or signal. The format of the goal class is experimental and may change. Currently, the following goals are valid:
76+
**** `k8s:__RESOURCE[VERSION.[GROUP]]__` identifying a kind of kuberenetes resource. For example `k8s:Pod` or `k8s:Deployment.apps.v1`.
77+
**** `alert:alert` representing any alert.
78+
79+
**** `metric:metric` representing any metric.
80+
81+
**** `netflow:network` representing any network observability network event.
82+
83+
**** `log:__LOG_TYPE__` representing stored logs, where `__LOG_TYPE__` must be one of `application`, `infrastructure` or `audit`.

‎observability/cluster_observability_operator/ui_plugins/troubleshooting-ui-plugin.adoc

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="troubleshooting-ui-plugin"]
3+
= Troubleshooting UI plugin
4+
include::_attributes/common-attributes.adoc[]
5+
:context: troubleshooting-ui-plugin
6+
7+
toc::[]
8+
9+
:FeatureName: The Cluster Observability Operator
10+
include::snippets/technology-preview.adoc[leveloffset=+2]
11+
12+
The troubleshooting UI plugin provides observability signal correlation, powered by the open source Korrel8r project.
13+
With the troubleshooting panel that is available under *Observe* -> *Alerting*, you can easily correlate metrics, logs, alerts, netflows, and additional observability signals and resources, across different data stores. Users of {product-title} version 4.17+ can also access the troubleshooting UI panel from the Application Launcher {launch}.
14+
15+
When you install the troubleshooting UI plugin, a link:https://github.com/korrel8r/korrel8r[Korrel8r] service named `korrel8r` is deployed in the same namespace, and it is able to locate related observability signals and Kubernetes resources from its correlation engine.
16+
17+
The output of Korrel8r is displayed in the form of an interactive node graph in the {product-title} web console.
18+
Nodes in the graph represent a type of resource or signal, while edges represent relationships.
19+
When you click on a node, you are automatically redirected to the corresponding web console page with the specific information for that node, for example, metric, log, pod.
20+
21+
include::modules/coo-troubleshooting-ui-plugin-install.adoc[leveloffset=+1]
22+
23+
include::modules/coo-troubleshooting-ui-plugin-using.adoc[leveloffset=+1]
24+
25+
include::modules/coo-troubleshooting-ui-plugin-creating-alert.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)