Document multitenancy support in GitOps

Addressing SME review comments

IMplementing SME review comments

pdating and adding info to the docinfo.xml file

Second round of SME review comments incorporation

Fixing nitpick

Incorporating SME + peer review comments

updating docinfo.xml file

Author: Srivaralakshmi

_topic_maps/_topic_map.yml

@@ -96,6 +96,13 @@ Topics:
 - Name: Managing the application set resources in non-control plane namespaces
   File: managing-app-sets-in-non-control-plane-namespaces
 ---
+Name: Multitenancy
+Dir: multitenancy
+Distros: openshift-gitops
+Topics:
+- Name: Multitenancy support in GitOps
+  File: multitenancy-support-in-gitops
+---
 Name: Declarative cluster configuration
 Dir: declarative_clusterconfig
 Distros: openshift-gitops

argocd_instance/setting-up-argocd-instance.adoc

@@ -21,6 +21,12 @@ For {gitops-shortname} version 1.13 and later:
 
 To manage cluster configurations or deploy applications, you can install and deploy a new user-defined Argo CD instance. By default, any new user-defined instance has permissions to manage resources only in the namespace where it is deployed.
 
+[WARNING]
+====
+* A Kubernetes user with access to the Argo CD namespace is an Argo CD administrator and can bypass any role-based access control (RBAC) restrictions configured in Argo CD. Never grant non-administrator users any read or write access to the Argo CD namespace.
+* If non-administrator users create applications, do not allow them to be bound to the default `AppProject` custom resource (CR) because it has no restrictions. Otherwise, the Kubernetes permissions of the Argo CD instances and the default `AppProject` CR allow deployment of everything everywhere. To prevent this situation, lock down the default `AppProject` CR so that no one can use it accidentally, even if someone misconfigures the Argo CD RBAC.
+====
+
 You can create a user-defined Argo CD instance in any namespace, other than the `openshift-gitops` namespace.
 
 [IMPORTANT]

declarative_clusterconfig/configuring-an-openshift-cluster-by-deploying-an-application-with-cluster-configurations.adoc

@@ -67,4 +67,5 @@ include::modules/gitops-installing-olm-operators-using-gitops.adoc[leveloffset=+
 == Additional resources
 
 * xref:../installing_gitops/installing-argocd-gitops-cli.adoc#installing-argocd-gitops-cli[Installing the {gitops-shortname} CLI]
-* xref:../gitops_cli_argocd/argocd-gitops-cli-reference.adoc#argocd-gitops-cli-reference[Basic {gitops-shortname} argocd commands]
+* xref:../gitops_cli_argocd/argocd-gitops-cli-reference.adoc#argocd-gitops-cli-reference[Basic {gitops-shortname} argocd commands]
+* xref:../multitenancy/multitenancy-support-in-gitops.adoc#multitenancy-support-in-gitops[Multitenancy support in {gitops-shortname}]

modules/gitops-argocd-projects.adoc

@@ -0,0 +1,20 @@
+// Module included in the following assemblies:
+//
+// * multitenancy/multitenancy-support-in-gitops.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="gitops-argocd-projects_{context}"]
+= Argo CD projects
+
+Argo CD projects, not to be confused with {OCP} projects, provide a way to group applications together. With Argo CD projects, you can specify restrictions on Applications concerning what resources can be deployed and where they can be deployed. Additionally, you can enable Argo CD role-based access control (RBAC) rules and permissions by defining them more granularly at the project level versus the global level in the Argo CD custom resource (CR).
+
+While you can define tenant RBAC globally in the Operator’s Argo CD CR, you should define tenant RBAC along with restrictions in the `AppProject` CR.
+
+If you have a large number of tenants, attempting to manage all tenants with global RBAC might lead to a lot of repetition. 
+
+If you have many instances of tenants, some of the project configurations might be common across tenant projects. To reduce duplication and minimize maintenance, use global projects for common configuration and inherit them in tenant projects.
+
+[NOTE]
+====
+Always define your projects and never use the default project created with an Argo CD installation by the Operator.
+====

modules/gitops-argocd-rbac.adoc

@@ -0,0 +1,14 @@
+// Module included in the following assemblies:
+//
+// * multitenancy/multitenancy-support-in-gitops.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="gitops-argocd-rbac_{context}"]
+= Argo CD role-based access control (RBAC)
+
+Access controls in {gitops-title} are managed at two distinct levels as follows:
+
+* At the Kubernetes level, the {gitops-shortname} Argo CD Application Controller interacts with one or more clusters to deploy various resources by using a single Kubernetes service account per cluster. This service account must have sufficient permissions to deploy resources for all tenants and use cases that this instance of Argo CD is managing.
+* At the Argo CD level, the {gitops-shortname} Argo CD Application Controller includes its own RBAC permissions model independent of Kubernetes. To manage the user-level access, you can use this RBAC model independently of the underlying Kubernetes permissions. You can use this capability to provide access to Argo CD applications that a user would not have access to from a purely Kubernetes perspective.
+
+Because these two access controls and the interaction between components are distinct and separate, privilege escalation becomes a concern when designing a multitenant solution with Argo CD. Privilege escalation is when a tenant leverages the increased privileges of the application controller’s service account to perform actions they would not typically be permitted to perform. You can mitigate the privilege escalation in an Argo CD instance by using the Argo CD RBAC or separate instances of Argo CD.

modules/gitops-cluster-scoped-instance.adoc

@@ -0,0 +1,32 @@
+// Module included in the following assemblies:
+//
+// * multitenancy/multitenancy-support-in-gitops.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="gitops-cluster-scoped-instance_{context}"]
+= Cluster-scoped instance 
+
+A cluster-scoped instance is intended to deploy and manage resources across a cluster.
+
+[NOTE]
+====
+If you intend to use the _Applications in any namespace_ feature, choose the mode of the Argo CD instance scope as cluster-scoped instance.
+====
+
+Cluster-scoped instances have access to cluster-level resources and thus are typically, but not always, used for cluster configuration. You can choose to elevate certain namespace-scoped Argo CD instances to become cluster-scoped. To elevate the instances, you must modify the `Subscription` resource of the {gitops-shortname} Operator.
+
+[IMPORTANT]
+====
+* Ensure you give careful consideration while elevating instances.
+* Do not elevate instances that are self-managed by application delivery teams. Elevating such instances would be a severe security risk to the cluster because this action makes users of self-managed instances cluster administrators, and gives them full control over the permissions. 
+====
+
+Much care must be taken when setting up multitenancy configuration within Argo CD. For example, in a use case where cluster administrators want to set up an Argo CD instance that is shared among multiple application delivery teams and is managed by the cluster administrators, having a custom cluster-scoped instance might be what you need.
+
+.How does this method work?
+
+To prevent users from deploying Argo CD instances with `cluster-admin` privileges, you must identify the namespaces with cluster privileges by using the `ARGOCD_CLUSTER_CONFIG_NAMESPACES` environment variable in the `Subscription` resource. 
+
+Because non-cluster administrators do not have access to the `Subscription` resource, they cannot elevate the privileges of their instance and bypass cluster security.
+
+When an instance is designated as cluster-scoped, the Operator automatically creates a set of cluster role and cluster role bindings for the Argo CD Application Controller and server service accounts in that namespace. This default role is not intended to be the equivalent of the standard `cluster-admin` role. It is given a much smaller set of permissions. These permissions can be extended by creating additional cluster roles or cluster role bindings as needed.

modules/gitops-default-cluster-scoped-instance.adoc

@@ -0,0 +1,22 @@
+// Module included in the following assemblies:
+//
+// * multitenancy/multitenancy-support-in-gitops.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="gitops-default-cluster-scoped-instance_{context}"]
+= Default cluster-scoped instance 
+
+After you install the {gitops-shortname} Operator, by default, it instantiates a cluster-scoped instance in the `openshift-gitops` namespace. This instance is set up in a very opinionated way and is intended to allow cluster administrators to manage certain cluster configuration resources.
+
+[IMPORTANT]
+====
+* Do not use the default cluster-scoped instance for anything else, such as application delivery. 
+* Do not grant permission to users who do not have `cluster-admin` roles to access the default cluster-scoped instance.
+====
+
+The default instance does not have full `cluster-admin` privileges. It has read access to all resources in the cluster, but it can only deploy a limited set of resources.
+
+[NOTE]
+====
+Use the default Argo CD instance in the `openshift-gitops` namespace for cluster configuration and delegate tenant use cases to one or many separate instances in other namespaces.
+====

modules/gitops-inbuilt-permissions-for-cluster-config.adoc

@@ -10,7 +10,8 @@ By default, the Argo CD instance has permissions to manage specific cluster-scop
 
 [NOTE]
 ====
-Argo CD does not have cluster-admin permissions.
+* Argo CD does not have `cluster-admin` permissions.
+* You can extend the permissions bound to any Argo CD instances managed by the {gitops-shortname} Operator. However, you must not modify the permission resources, such as roles or cluster roles created by the {gitops-shortname} Operator, because the Operator might reconcile them back to their initial state. Instead, create dedicated role and cluster role objects and bind them to the appropriate service account that the application controller uses.
 ====
 
 Permissions for the Argo CD instance:

modules/gitops-namespace-scoped-instance.adoc

@@ -0,0 +1,35 @@
+// Module included in the following assemblies:
+//
+// * multitenancy/multitenancy-support-in-gitops.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="gitops-namespace-scoped-instance_{context}"]
+= Namespace-scoped instance (Application delivery instance)
+
+After you create an Argo CD custom resource (CR) in one of your namespaces, the {gitops-shortname} Operator starts an Argo CD in this namespace, which you can use for application delivery. In its initial state, this instance has permission to deploy resources only in the same namespace in which it is installed. However, you may need to configure the instance to meet your specific requirements.
+
+With the {gitops-shortname} Operator, you can extend the permissions of the Argo CD instance to enable the Argo CD Application Controller to deploy resources into other namespaces apart from where it is installed. 
+
+[NOTE]
+====
+The roles that the {gitops-shortname} Operator in the namespace creates are namespace-scoped and only have access to namespace resources. The Operator cannot perform any actions outside of the namespaces it manages.
+====
+
+.How does this method work?
+
+The {gitops-shortname} Operator allows OpenShift users to instantiate Argo CD instances in their namespaces, as long as they have permission to create Argo CD resources in the `argoproj.io/v1alpha1` or `argoproj.io/v1beta1` API in their namespace. Currently, a namespace-scoped instance has full administrative permissions for one or many namespaces it manages, similar to allowing all verbs for all resources within that namespace.
+
+For the Argo CD Application Controller to deploy resources into any other namespace, you need Kubernetes roles and role bindings to label and indicate these namespaces that you want a namespace-scoped instance to manage. The {gitops-shortname} Operator supports the use of the `argocd.argoproj.io/managed-by` label to automatically create these roles and role bindings. Use this label and set the value to indicate the managed namespaces. Then, with the `argocd.argoproj.io/managed-by` label, when you deploy the {gitops-shortname} Operator in the namespace-scoped instance mode, it creates a set of roles and role bindings in every namespace that the instance manages. 
+
+[IMPORTANT]
+====
+The `argocd.argoproj.io/managed-by` label only works for namespaces in the same cluster as the {gitops-shortname} Operator. For remote clusters, you must define the permissions manually.
+====
+
+By default, when labeling the namespace using the `managed-by` label, the {gitops-shortname} Operator gives the Argo CD Application Controller permissions equivalent to the Kubernetes default `admin` cluster role for the labeled namespace. However, you can opt to define an alternate cluster role that is used for the controller and server components by using the `CONTROLLER_CLUSTER_ROLE` and `CONTROLLER_SERVER_ROLE` environment variables respectively in the Operator's `Subscription` resource.  When you provide these variables, the Operator will not create a default role in the namespace but rather only create a role binding in the namespace of the corresponding cluster role. It is up to the administrator to create the cluster role, having full control over the permissions.
+
+[NOTE]
+====
+* When defining your role, Argo CD will attempt to interact with all resources. Therefore, you must either provide permission to `view`, `get`, and `watch` all resources in your custom cluster role, or configure the Argo CD CR to include or exclude specific resources through the `resourceInclusions` or `resourceExclusions` fields defined in the role.
+* It is not possible to manage cluster-scoped resources, such as namespaces, custom resource definitions (CRDs), or cluster roles with a namespace-scoped instance.
+====

modules/gitops-using-argo-cd-instance-to-manage-cluster-scoped-resources.adoc

@@ -6,6 +6,11 @@
 [id="using-argo-cd-instance-to-manage-cluster-scoped-resources_{context}"]
 = Using an Argo CD instance to manage cluster-scoped resources
 
+[WARNING]
+====
+Do not elevate the permissions of Argo CD instances to be cluster-scoped unless you have a distinct use case that requires it. Only users with `cluster-admin` privileges should manage the instances you elevate. Anyone with access to the namespace of a cluster-scoped instance can elevate their privileges on the cluster to become a cluster administrator themselves.
+====
+
 To manage cluster-scoped resources, update the existing `Subscription` object for the {gitops-title} Operator and add the namespace of the Argo CD instance to the `ARGOCD_CLUSTER_CONFIG_NAMESPACES` environment variable in the `spec` section.
 
 .Procedure