add docs about resource groups

Author: abrennan89

modules/configuring-resourceflavors.adoc

@@ -8,9 +8,9 @@
 
 After you have configured a `ClusterQueue` object, you can configure a `ResourceFlavor` object.
 
-Resources in a cluster are typically not homogeneous. If the resources in your cluster are homogeneous, you can use an empty `ResourceFlavor` instead of adding labels to custom resource flavors.
+You can use a custom `ResourceFlavor` object to represent different resource types that are associated with cluster nodes through labels, taints, and tolerations. You can then associate workloads with specific node types to enable fine-grained resource management.
 
-You can use a custom `ResourceFlavor` object to represent different resource variations that are associated with cluster nodes through labels, taints, and tolerations. You can then associate workloads with specific node types to enable fine-grained resource management.
+Resources in a cluster are typically not homogeneous. If the resources in your cluster are homogeneous, you can use an empty `ResourceFlavor` instead of adding labels to custom resource flavors.
 
 .Prerequisites
 

modules/resource-groups-flavors.adoc

@@ -0,0 +1,55 @@
+// Module included in the following assemblies:
+//
+// * configure/using-cohorts.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="resource-groups-flavors_{context}"]
+= Resource groups and flavors
+
+You can configure resource groups within cluster queues to define a list of resources and a list of flavors that provide quotas for these resources.
+Each resource type and flavor can only belong to one resource group.
+
+.Example of a basic `ClusterQueue` object with resource groups defined
+[source,yaml]
+----
+apiVersion: kueue.x-k8s.io/v1beta1
+kind: ClusterQueue
+metadata:
+  name: example-queue
+spec:
+  namespaceSelector: {}
+  resourceGroups: # <1>
+  - coveredResources: ["cpu", "memory"] # <2>
+    flavors:
+    - name: "resource-flavor-a" # <3>
+      resources:
+      - name: "cpu"
+        nominalQuota: 9
+      - name: "memory"
+        nominalQuota: 36Gi
+  - coveredResources: ["pods", "foo.com/gpu"] # <4>
+    flavors:
+    - name: "resource-flavor-b" # <5>
+      resources: # <4>
+      - name: "pods"
+        nominalQuota: 5
+      - name: "foo.com/gpu"
+        nominalQuota: 100
+----
+<1> You can define up to 16 resource groups for your cluster.
+<2> Defines the resource types governed the first resource group. This resource group governs CPU and memory resources.
+<3> Defines the resource flavor that is applied to the resource types listed in the first resource group. In this example, the `resource-flavor-a` resource flavor is applied to CPU and memory.
+<4> Defines the resource types governed the second resource group. This resource group governs pods and GPU resources.
+<5> Defines the resource flavor that is applied to the resource types listed in the second resource group. In this example, the `resource-flavor-b` resource flavor is applied to pods and GPU.
+
+
+
+When a ClusterQueue is part of a cohort, Kueue satisfies the following admission semantics:
+
+When assigning flavors, Kueue goes through the list of flavors in the relevant ResourceGroup inside ClusterQueue’s (.spec.resourceGroups[*].flavors). For each flavor, Kueue attempts to fit a Workload’s pod set according to the quota defined in the ClusterQueue for the flavor and the unused quota in the cohort. If the Workload doesn’t fit, Kueue evaluates the next flavor in the list.
+A Workload’s pod set resource fits in a flavor defined for a ClusterQueue resource if the sum of requests for the resource:
+Is less than or equal to the unused nominalQuota for the flavor in the ClusterQueue; or
+Is less than or equal to the sum of unused nominalQuota for the flavor in the ClusterQueues in the cohort, and
+Is less than or equal to the unused nominalQuota + borrowingLimit for the flavor in the ClusterQueue. In Kueue, when (2) and (3) are satisfied, but not (1), this is called borrowing quota.
+A ClusterQueue can only borrow quota for flavors that the ClusterQueue defines.
+For each pod set resource in a Workload, a ClusterQueue can only borrow quota for one flavor.