Skip to content

Commit 61a8143

Browse files
mburke5678mburke5678
committed
EO - ClusterLogging Status messages when cluster logging is not functional
1 parent f9b3e2b commit 61a8143

File tree

2 files changed

+214
-46
lines changed

2 files changed

+214
-46
lines changed

‎logging/config/efk-logging-elasticsearch.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ include::modules/efk-logging-elasticsearch-exposing.adoc[leveloffset=+1]
4141

4242
include::modules/efk-logging-elasticsearch-rules.adoc[leveloffset=+1]
4343

44+
include::modules/efk-logging-elasticsearch-status.adoc[leveloffset=+1]
45+
4446
////
4547
modules/efk-logging-elasticsearch-persistent-storage-persistent.adoc[leveloffset=+2]
4648

‎modules/efk-logging-elasticsearch-status.adoc

Lines changed: 212 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -13,59 +13,225 @@ You can view the status of your Elasticsearch cluster.
1313

1414
.Procedure
1515

16-
Run the following command to view the Elasticsearch status:
16+
. Change to the `openshift-logging` project.
17+
+
18+
----
19+
$ oc project openshift-logging
20+
----
21+
22+
. To view the Elasticsearch cluster status:
1723

24+
.. Get the name of the Elasticsearch instance:
25+
+
26+
----
27+
$ oc get Elasticsearch
28+
29+
NAME AGE
30+
elasticsearch 5h9m
1831
----
19-
$ oc get Elasticsearch elasticsearch -n openshift-logging -o yaml
2032

21-
status:
22-
clusterHealth: green <1>
23-
conditions: <2>
33+
.. Get the Elasticsearch status:
34+
+
35+
----
36+
$ oc get Elasticsearch <Elasticsearch-instance> -o yaml
37+
----
38+
+
39+
For example:
40+
+
41+
----
42+
$ oc get Elasticsearch elasticsearch -n openshift-logging -o yaml
43+
----
44+
+
45+
The output includes information similar to the following:
46+
+
47+
----
48+
status: <1>
49+
clusterHealth: green <2>
50+
conditions: <3>
2451
.....
25-
nodes: <2>
52+
nodes: <4>
2653
.....
27-
pods: <3>
54+
pods: <5>
2855
.....
29-
shardAllocationEnabled: "True" <4>
30-
----
31-
<1> The status of the Elasticsearch cluster, `green`, `red`, `yellow`.
32-
<2> Conditions
33-
<3> The Elsticsearch nodes in the cluster, with `upgradeStatus`.
34-
<4> The Elasticsearch client, data, and master pods in the cluster, listed under 'failed`, `notReady` or `ready` state.
35-
36-
oc get deployment --selector component=elasticsearch -n openshift-logging -o name
37-
38-
https://github.com/openshift/elasticsearch-operator/pull/92
39-
40-
----
41-
conditions:
42-
- lastTransitionTime: 2019-03-06T23:33:00Z
43-
message: Invalid master nodes count. Please ensure there are no more than 3 total
44-
nodes with master roles
45-
reason: Invalid Settings
46-
status: "True"
47-
type: InvalidMasters
48-
- lastTransitionTime: 2019-03-06T23:28:37Z
49-
status: "False"
50-
type: InvalidData
51-
- lastTransitionTime: 2019-03-06T23:28:38Z
52-
status: "False"
53-
type: InvalidRedundancy
54-
- lastTransitionTime: 2019-03-06T23:30:35Z
55-
message: Config Map is different
56-
reason: ConfigChange
57-
status: "True"
58-
type: UpdatingSettings
59-
- lastTransitionTime: 2019-03-06T23:28:52Z
60-
status: "False"
61-
type: ScalingUp
62-
- lastTransitionTime: 2019-03-06T23:28:52Z
63-
status: "False"
64-
type: ScalingDown
65-
- lastTransitionTime: 2019-03-06T23:28:52Z
66-
status: "False"
67-
type: Restarting
56+
shardAllocationEnabled: "True" <6>
57+
----
58+
<1> In the output, the cluster status fields appear in the `status` stanza.
59+
<2> The status of the Elasticsearch cluster, `green`, `red`, `yellow`.
60+
<3> Any status conditions, if present.
61+
<4> The Elsticsearch nodes in the cluster, with `upgradeStatus`.
62+
<5> The Elasticsearch client, data, and master pods in the cluster, listed under 'failed`, `notReady` or `ready` state.
63+
64+
. To view the status of the Elasticsearch deployment configuration:
65+
66+
.. Get the name of a deployment configuration:
67+
+
68+
----
69+
$ oc get deployment --selector component=elasticsearch -o name
70+
71+
deployment.extensions/elasticsearch-cdm-1gon-1
72+
deployment.extensions/elasticsearch-cdm-1gon-2
73+
deployment.extensions/elasticsearch-cdm-1gon-3
74+
----
75+
76+
.. Get the deployment configuration status:
77+
+
6878
----
79+
$ oc describe deployment elasticsearch-cdm-1gon-1
80+
----
81+
+
82+
The output includes the following status information:
83+
+
84+
----
85+
....
86+
Containers:
87+
elasticsearch:
88+
Image: quay.io/openshift/origin-logging-elasticsearch5:latest
89+
Readiness: exec [/usr/share/elasticsearch/probe/readiness.sh] delay=10s timeout=30s period=5s #success=1 #failure=3
90+
91+
....
92+
93+
Conditions:
94+
Type Status Reason
95+
---- ------ ------
96+
Progressing Unknown DeploymentPaused
97+
Available True MinimumReplicasAvailable
98+
99+
....
100+
101+
Events: <none>
102+
----
103+
104+
. To view the status of the Elasticsearch Replica Set:
105+
106+
.. Get the name of a replica set:
107+
+
108+
----
109+
$ oc get replicaSet --selector component=elasticsearch -o name
110+
111+
replicaset.extensions/elasticsearch-cdm-1gon-1-6f8495
112+
replicaset.extensions/elasticsearch-cdm-1gon-2-5769cf
113+
replicaset.extensions/elasticsearch-cdm-1gon-3-f66f7d
114+
----
115+
116+
.. Get the status of the replica set:
117+
+
118+
----
119+
$ oc describe replicaSet elasticsearch-cdm-1gon-1-6f8495
120+
----
121+
+
122+
The output includes the following status information:
123+
+
124+
----
125+
....
126+
Containers:
127+
elasticsearch:
128+
Image: quay.io/openshift/origin-logging-elasticsearch5:latest
129+
Readiness: exec [/usr/share/elasticsearch/probe/readiness.sh] delay=10s timeout=30s period=5s #success=1 #failure=3
130+
131+
....
132+
133+
Events: <none>
134+
----
135+
136+
. To view the status of the Elasticsearch pods:
137+
138+
.. Get the name of a pod:
139+
+
140+
----
141+
$ oc get pods --selector component=elasticsearch -o name
142+
143+
pod/elasticsearch-cdm-1godmszn-1-6f8495-vp4lw
144+
pod/elasticsearch-cdm-1godmszn-2-5769cf-9ms2n
145+
pod/elasticsearch-cdm-1godmszn-3-f66f7d-zqkz7
146+
----
147+
148+
.. Get the status of a pod:
149+
+
150+
----
151+
oc describe pod elasticsearch-cdm-1godmszn-1-6f8495-vp4lw
152+
----
153+
+
154+
The output includes the following status information:
155+
+
156+
----
157+
....
158+
Status: Running
159+
160+
....
161+
162+
Containers:
163+
elasticsearch:
164+
Container ID: cri-o://b7d44e0a9ea486e27f47763f5bb4c39dfd2
165+
State: Running
166+
Started: Mon, 08 Apr 2019 10:17:56 -0400
167+
Ready: True
168+
Restart Count: 0
169+
Readiness: exec [/usr/share/elasticsearch/probe/readiness.sh] delay=10s timeout=30s period=5s #success=1 #failure=3
170+
171+
....
172+
173+
proxy:
174+
Container ID: cri-o://3f77032abaddbb1652c116278652908dc01860320b8a4e741d06894b2f8f9aa1
175+
State: Running
176+
Started: Mon, 08 Apr 2019 10:18:38 -0400
177+
Ready: True
178+
Restart Count: 0
179+
180+
....
181+
182+
Conditions:
183+
Type Status
184+
Initialized True
185+
Ready True
186+
ContainersReady True
187+
PodScheduled True
188+
189+
....
190+
191+
Events: <none>
192+
----
193+
194+
== Viewing Elasticsearch cluster status
195+
196+
197+
198+
// https://github.com/openshift/elasticsearch-operator/pull/92
199+
200+
201+
[cols="3,4",options="header"]
202+
|===
203+
| Condition
204+
| Reason
205+
206+
207+
| Invalid master nodes count. Please ensure there are no more than 3 total nodes with master roles
208+
|
209+
210+
| No data nodes requested. Please ensure there is at least 1 node with data roles
211+
|
212+
213+
| Wrong RedundancyPolicy selected. Choose different RedundancyPolicy or add more nodes with data roles
214+
|
215+
216+
| Config Map is different
217+
|
218+
219+
| ScalingUp
220+
|
221+
222+
| ScalingDown
223+
|
224+
225+
| Restarting
226+
|
227+
228+
| Disk Watermark High
229+
| Shards will be relocated from this node
230+
231+
| Disk Watermark Low
232+
| Shards will be not be allocated on this node.
233+
234+
|===
69235

70236
Example conditions
71237

0 commit comments

Comments
 (0)