Skip to content

Commit 4f7cb78

Browse files
thegridmanthegridman
authored
Update installation documentation (#733)
1 parent f571641 commit 4f7cb78

21 files changed

+1293
-1026
lines changed

‎docs/images/openshift-coherence-install-done.png

617 KB
Loading

‎docs/images/openshift-coherence-install-progress.png

709 KB
Loading

‎docs/images/openshift-coherence-install.png

1.32 MB
Loading

‎docs/images/openshift-coherence-operator-details.png

1.69 MB
Loading

‎docs/images/openshift-coherence.png

1.68 MB
Loading

‎docs/images/openshift-operatorhub-coherence.png

1000 KB
Loading

‎docs/installation/001_installation.adoc

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
///////////////////////////////////////////////////////////////////////////////
2+
3+
Copyright (c) 2020, 2025, Oracle and/or its affiliates.
4+
Licensed under the Universal Permissive License v 1.0 as shown at
5+
http://oss.oracle.com/licenses/upl.
6+
7+
///////////////////////////////////////////////////////////////////////////////
8+
9+
= Coherence Operator Installation
10+
11+
The Coherence Operator is available as an image from the GitHub container registry
12+
`container-registry.oracle.com/middleware/coherence-operator:{operator-version}` that can
13+
easily be installed into a Kubernetes cluster.
14+
15+
== Coherence Operator Installation
16+
17+
*Contents*
18+
19+
* <<prereq,Prerequisites before installation>>
20+
* <<ha,Operator High Availability>>
21+
* <<images,Coherence Operator Images>>
22+
* <<scope,Operator Scope - monitoring all or a fixed set of namespaces>>
23+
24+
[#prereq]
25+
=== Prerequisites
26+
The prerequisites apply to all installation methods.
27+
28+
* Access to Oracle Coherence Operator images.
29+
* Access to a Kubernetes cluster. The Operator test pipeline is run using against all the currently supported Kubernetes versions.
30+
* A Coherence application image using Coherence version 12.2.1.3 or later. Note that some functionality (e.g. metrics) is only
31+
available in Coherence 12.2.1.4 and later.
32+
33+
34+
[NOTE]
35+
====
36+
Istio (or similar service meshes)
37+
38+
When installing the Operator and Coherence into Kubernetes cluster that use Istio or similar meshes there are a
39+
number of pre-requisites that must be understood.
40+
See the <<examples/400_Istio/README.adoc,Istio example>> for more details.
41+
====
42+
43+
=== Installation Options
44+
45+
There are a number of ways to install the Coherence Operator.
46+
47+
* <<installation/011_install_manifests.adoc,Install using the yaml manifest file>>
48+
* <<installation/012_install_helm.adoc,Install using Helm>>
49+
* <<installation/013_install_kustomize.adoc,Install using Kustomize>>
50+
* <<installation/014_install_openshift.adoc,Install on OpenShift>>
51+
* <<installation/015_install_olm.adoc,Install using the Operator Lifecycle Manager (OLM)>>
52+
* <<installation/016_install_tanzu.adoc,Install on VMWare Tanzu>>
53+
54+
55+
[#ha]
56+
=== High Availability
57+
58+
The Coherence Operator runs in HA mode by default. The `Deployment` created by the installation will have a replica count of 3.
59+
In reduced capacity Kubernetes clusters, for example, local laptop development and test, the replica count can be reduced. It is recommended to leave the default of 3 for production environments.
60+
Instructions on how to change the replica count for the different install methods are included below.
61+
62+
The Coherence Operator runs a REST server that the Coherence cluster members will query to discover the site and rack names that should be used by Coherence. If the Coherence Operator is not running when a Coherence Pod starts, then the Coherence member in that Pod will be unable to properly configure its site and rack names, possibly leading to data distribution that is not safely distributed over sites. In production, and in Kubernetes clusters that are spread over multiple availability zones and failure domains, it is important to run the Operator in HA mode.
63+
64+
The Operator yaml files and Helm chart include a default Pod scheduling configuration that uses anti-affinity to distribute the three replicas onto nodes that have different `topology.kubernetes.io/zone` labels. This label is a standard Kubernetes label used to describe the zone the node is running in, and is typically applied by Kubernetes cloud vendors.
65+
66+
67+
=== Notes
68+
69+
NOTE: Installing the Coherence Operator using the methods below will create a number of `ClusterRole` RBAC resources.
70+
Some corporate security policies do not like to give cluster wide roles to third-party products.
71+
To help in this situation the operator can be installed without cluster roles, but with caveats
72+
(see the <<docs/installation/09_RBAC.adoc,RBAC>> documentation) for more details.
73+
74+
NOTE: OpenShift - the Coherence Operator works without modification on OpenShift, but some versions
75+
of the Coherence images will not work out of the box.
76+
See the <<docs/installation/06_openshift.adoc,OpensShift>> section of the documentation that explains how to
77+
run Coherence clusters with the Operator on OpenShift.
78+
79+
NOTE: Whilst Coherence works out of the box on many Kubernetes installations, some Kubernetes
80+
installations may configure iptables in a way that causes Coherence to fail to create clusters.
81+
See the <<docs/installation/08_networking.adoc,O/S Network Configuration>> section of the documentation
82+
for more details if you have well-known-address issues when Pods attempt to form a cluster.
83+
84+
[#images]
85+
== Coherence Operator Images
86+
87+
The Coherence Operator uses a single image, the Operator also runs as an init-container in the Coherence cluster Pods.
88+
89+
* `{operator-image}` - The Operator image.
90+
91+
If no image is specified in the `Coherence` yaml, then the default Coherence image will also be used,
92+
93+
* `{coherence-image}` - The default Coherence image.
94+
95+
If using a private image registry then these images will all need to be pushed to that registry for the Operator to work. The default Coherence image may be omitted if all Coherence applications will use custom Coherence images.
96+
97+
[#scope]
98+
== Operator Scope
99+
100+
The recommended way to install the Coherence Operator is to install a single instance of the operator into a namespace
101+
and where it will then control `Coherence` resources in all namespaces across the Kubernetes cluster.
102+
Alternatively it may be configured to watch a sub-set of namespaces by setting the `WATCH_NAMESPACE` environment variable.
103+
The watch namespace(s) does not have to include the installation namespace.
104+
105+
[CAUTION]
106+
====
107+
In theory, it is possible to install multiple instances of the Coherence Operator into different namespaces, where
108+
each instance monitors a different set of namespaces. There are a number of potential issues with this approach, so
109+
it is not recommended.
110+
111+
* Only one version of a CRD can be installed - There is currently only a single version of the CRD, but different
112+
releases of the Operator may use slightly different specs of this CRD version, for example
113+
a new Operator release may introduce extra fields not in the previous releases.
114+
As the CRD version is fixed at `v1` there is no guarantee which CRD version has actually installed, which could lead to
115+
subtle issues.
116+
* The operator creates and installs defaulting and validating web-hooks. A web-hook is associated to a CRD resource so
117+
installing multiple web-hooks for the same resource may lead to issues. If an operator is uninstalled, but the web-hook
118+
configuration remains, then Kubernetes will not accept modifications to resources of that type as it will be
119+
unable to contact the web-hook.
120+
121+
It is possible to run the Operator without web-hooks, but this has its own
122+
caveats see the <<docs/installation/07_webhooks.adoc,Web Hooks>> documentation for how to do this.
123+
====
124+
125+
[IMPORTANT]
126+
====
127+
If multiple instances of the Operator are installed, where they are monitoring the same namespaces, this can cause issues.
128+
For example, when a `Coherence` resource is then changed, all the Operator deployments will receive the same events
129+
from Etcd and try to apply the same changes. Sometimes this may work, sometimes there may be errors, for example multiple
130+
Operators trying to remove finalizers and delete a Coherence cluster.
131+
====
132+

‎docs/installation/011_install_manifests.adoc

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
///////////////////////////////////////////////////////////////////////////////
2+
3+
Copyright (c) 2020, 2025, Oracle and/or its affiliates.
4+
Licensed under the Universal Permissive License v 1.0 as shown at
5+
http://oss.oracle.com/licenses/upl.
6+
7+
///////////////////////////////////////////////////////////////////////////////
8+
9+
= Install Using Manifests
10+
11+
== Install Using Manifests
12+
13+
If you want the default Coherence Operator installation then the simplest solution is use `kubectl` to
14+
apply the manifests from the Operator release.
15+
16+
[NOTE]
17+
====
18+
As of v3.5.0 of the Operator the manifest yaml also installs the two CRDs that the Operator uses.
19+
In previous releases the Operator would install the CRDs when it started but this behaviour is disabled by default
20+
when installing with the manifest yaml.
21+
====
22+
23+
The following command will install the Operator. This assumes that the Kubernetes account being used to perform
24+
the installation has all the RBAC permissions required to install all the resource types in the yaml file.
25+
26+
[source,bash]
27+
----
28+
kubectl apply -f https://github.com/oracle/coherence-operator/releases/download/v3.5.0/coherence-operator.yaml
29+
----
30+
31+
This will create a namespace called `coherence` and install the CRDs and the Operator into the namespace,
32+
along with all the required `ClusterRole` and `RoleBinding` resources. The `coherence` namespace can be changed by
33+
downloading and editing the yaml file.
34+
35+
In some restricted environments, a Kubernetes user might not have RBAC permissions to install CRDs.
36+
In this case the `coherence-operator.yaml` file will need to be edited to remove the two CRDs from the
37+
beginning of the file. The CRDs *_must be manually installed before the Operator is installed_*, as described
38+
below in <<manual-crd,Manually Install the CRDs>>.
39+
40+
[NOTE]
41+
====
42+
Because the `coherence-operator.yaml` manifest also creates the namespace, the corresponding `kubectl delete`
43+
command will _remove the namespace and everything deployed to it_! If you do not want this behaviour you should edit
44+
the `coherence-operator.yaml` to remove the namespace section from the start of the file.
45+
====
46+
47+
Instead of using a hard coded version in the command above you can find the latest Operator version using `curl`:
48+
49+
[source,bash]
50+
----
51+
export VERSION=$(curl -s \
52+
https://api.github.com/repos/oracle/coherence-operator/releases/latest \
53+
| grep '"name": "v' \
54+
| cut -d '"' -f 4 \
55+
| cut -b 2-10)
56+
----
57+
58+
Then download with:
59+
[source,bash]
60+
----
61+
kubectl apply -f https://github.com/oracle/coherence-operator/releases/download/${VERSION}/coherence-operator.yaml
62+
----
63+
64+
[#manifest-restrict]
65+
== Installing Without Cluster Roles
66+
67+
The default install for the Operator is to have one Operator deployment that manages all Coherence resources across
68+
all the namespaces in a Kubernetes cluster. This requires the Operator to have cluster role RBAC permissions
69+
to manage and monitor all the resources.
70+
71+
Sometimes, for security reasons or for example in a shared Kubernetes cluster this is not desirable.
72+
The Operator can therefore be installed with plain namespaced scoped roles and role bindings.
73+
The Operator release includes a single yaml file named `coherence-operator-restricted.yaml` that may be used to install
74+
the Operator into a single namespace without any cluster roles.
75+
76+
The Operator installed with this yaml
77+
78+
* will not use WebHooks
79+
* will not look-up Node labels for Coherence site and rack configurations
80+
81+
[NOTE]
82+
====
83+
As of v3.5.0 of the Operator the `coherence-operator-restricted.yaml` also installs the two CRDs that the Operator uses.
84+
In previous releases the Operator would install the CRDs when it started but this behaviour is disabled by default
85+
when installing with the manifest yaml.
86+
====
87+
88+
The following command will install the Operator. This assumes that the Kubernetes account being used to perform
89+
the installation has all the RBAC permissions required to install all the resource types in the yaml file.
90+
91+
[source,bash]
92+
----
93+
kubectl apply -f https://github.com/oracle/coherence-operator/releases/download/v3.5.0/coherence-operator-restricted.yaml
94+
----
95+
[IMPORTANT]
96+
====
97+
In some restricted environments, a Kubernetes user might not have RBAC permissions to install CRDs.
98+
In this case the `coherence-operator.yaml` file will need to be edited to remove the two CRDs from the
99+
beginning of the file. The CRDs *_must be manually installed before the Operator is installed_*, as described
100+
below in <<manual-crd,Manually Install the CRDs>>.
101+
====
102+
103+
[#manual-crd]
104+
== Manually Install the CRDs
105+
106+
Although by default the Operator will install its CRDs, they can be manually installed into Kubernetes.
107+
This may be required where the Operator is running with restricted permissions as described above.
108+
109+
The Operator release artifacts include small versions of the two CRDs which can be installed with the following commands:
110+
111+
[source,bash]
112+
----
113+
kubectl apply -f https://github.com/oracle/coherence-operator/releases/download/v3.5.0/coherence.oracle.com_coherence_small.yaml
114+
kubectl apply -f https://github.com/oracle/coherence-operator/releases/download/v3.5.0/coherencejob.oracle.com_coherence_small.yaml
115+
----
116+
117+
The small versions of the CRDs are identical to the full versions but hav a cut down OpenAPI spec with a lot of comments
118+
removed so that the CRDs are small enough to be installed with `kubectl apply`
119+
120+
== Change the Operator Replica Count
121+
122+
When installing with single manifest yaml file, the replica count can be changed by editing the yaml file itself
123+
to change the occurrence of `replicas: 3` in the manifest yaml to `replicas: 1`
124+
125+
For example, this could be done using `sed`
126+
[source,bash]
127+
----
128+
sed -i -e 's/replicas: 3/replicas: 1/g' coherence-operator.yaml
129+
----
130+
131+
Or on MacOS, where `sed` is slightly different:
132+
[source,bash]
133+
----
134+
sed -i '' -e 's/replicas: 3/replicas: 1/g' coherence-operator.yaml
135+
----

0 commit comments

Comments
 (0)