Merge branch 'v2.17' into v2.17-RUN-15154-inference-workloads

Author: jasonnovichRunAI

docs/Researcher/Walkthroughs/quickstart-inference.md

@@ -2,44 +2,43 @@
 
 ## Introduction
 
-Machine learning (ML) inference is the process of running live data points into a machine-learning algorithm to calculate an output. 
+Machine learning (ML) inference is the process of running live data points into a machine-learning algorithm to calculate an output.
 
-With Inference, you are taking a trained _Model_ and deploying it into a production environment. The deployment must align with the organization's production standards such as average and 95% response time as well as up-time. 
+With Inference, you are taking a trained *Model* and deploying it into a production environment. The deployment must align with the organization's production standards such as average and 95% response time as well as up-time.
 
-## Prerequisites 
+## Prerequisites
 
 To complete this Quickstart you must have:
 
-* Run:ai software installed on your Kubernetes cluster. See: [Installing Run:ai on a Kubernetes Cluster](../../admin/runai-setup/installation-types.md). There are additional prerequisites for running inference. See [cluster installation prerequisites](../../admin/runai-setup/cluster-setup/cluster-prerequisites.md#inference) for more information. 
+* Run:ai software installed on your Kubernetes cluster. See: [Installing Run:ai on a Kubernetes Cluster](../../admin/runai-setup/installation-types.md). There are additional prerequisites for running inference. See [cluster installation prerequisites](../../admin/runai-setup/cluster-setup/cluster-prerequisites.md#inference) for more information.
 * Run:ai CLI installed on your machine. See: [Installing the Run:ai Command-Line Interface](../../admin/researcher-setup/cli-install.md)
-* You must have _ML Engineer_ access rights. See [Adding, Updating and Deleting Users](../../admin/admin-ui-setup/admin-ui-users.md) for more information. 
+* You must have *ML Engineer* access rights. See [Adding, Updating and Deleting Users](../../admin/admin-ui-setup/admin-ui-users.md) for more information.
 
 ## Step by Step Walkthrough
 
 ### Setup
 
-*  Login to the Projects area of the Run:ai user interface.
-*  Add a Project named "team-a".
-*  Allocate 2 GPUs to the Project.
+* Login to the Projects area of the Run:ai user interface.
+* Add a Project named "team-a".
+* Allocate 2 GPUs to the Project.
 
-### Run an Inference Workload 
+### Run an Inference Workload
 
-*   In the Run:ai user interface go to `Deployments`. If you do not see the `Deployments` section you may not have the required access control, or the inference module is disabled. 
+* In the Run:ai user interface go to `Deployments`. If you do not see the `Deployments` section you may not have the required access control, or the inference module is disabled.
 * Select `New Deployment` on the top right.
 * Select `team-a` as a project and add an arbitrary name. Use the image `gcr.io/run-ai-demo/example-triton-server`.
 * Under `Resources` add 0.5 GPUs.
-* Under `Auto Scaling` select a minimum of 1, a maximum of 2. Use the `concurrency` autoscaling threshold method. Add a threshold of 3.
+* Under `Autoscaling` select a minimum of 1, a maximum of 2. Use the `concurrency` autoscaling threshold method. Add a threshold of 3.
 * Add a `Container port` of `8000`.
 
-
 This would start an inference workload for team-a with an allocation of a single GPU. Follow up on the Job's progress using the [Deployment list](../../admin/admin-ui-setup/deployments.md) in the user interface or by running `runai list jobs`
 
 ### Query the Inference Server
 
 The specific inference server we just created is accepting queries over port 8000. You can use the Run:ai Triton demo client to send requests to the server:
 
 * Find an IP address by running `kubectl get svc -n runai-team-a`. Use the `inference1-00001-private` Cluster IP.
-* Replace `<IP>` below and run: 
+* Replace `<IP>` below and run:
 
 ```
  runai submit inference-client  -i gcr.io/run-ai-demo/example-triton-client \
@@ -52,11 +51,10 @@ The specific inference server we just created is accepting queries over port 800
 runai logs inference-client
 ```
 
-
 ### View status on the Run:ai User Interface
 
 * Open the Run:ai user interface.
-* Under _Deployments_ you can view the new Workload. When clicking the workload, note the utilization graphs go up. 
+* Under *Deployments* you can view the new Workload. When clicking the workload, note the utilization graphs go up.
 
 ### Stop Workload
 
@@ -66,4 +64,3 @@ Use the user interface to delete the workload.
 
 * You can also create Inference deployments via API. For more information see [Submitting Workloads via YAML](../../developer/cluster-api/submit-yaml.md).
 * See [Deployment](../../admin/admin-ui-setup/deployments.md) user interface.
-

docs/Researcher/cli-reference/runai-submit-dist-TF.md

@@ -273,6 +273,10 @@ runai submit-dist tf --name distributed-job --workers=2 -g 1 \
 > 
 > Mount /root/data to NFS path /public/data on NFS server nfs.example.com for read-write access.
 
+#### --configmap-volume name=<name of configmap>,path=<path to mount> ...'
+
+> Mount a `ConfigMap` object for use as a data volume.
+
 ### Network
 
 #### --address `<string>`

docs/Researcher/cli-reference/runai-submit-dist-mpi.md

@@ -276,6 +276,10 @@ You can start an unattended mpi training Job of name dist1, based on Project *te
 > 
 > Mount /root/data to NFS path /public/data on NFS server nfs.example.com for read-write access.
 
+#### --configmap-volume name=<name of configmap>,path=<path to mount> ...'
+
+> Mount a `ConfigMap` object for use as a data volume.
+
 ### Network
 
 #### --address `<string>`

docs/Researcher/cli-reference/runai-submit-dist-pytorch.md

@@ -280,6 +280,10 @@ runai submit-dist pytorch --name distributed-job --workers=2 -g 1 \
 > 
 > Mount /root/data to NFS path /public/data on NFS server nfs.example.com for read-write access.
 
+#### --configmap-volume name=<name of configmap>,path=<path to mount> ...'
+
+> Mount a `ConfigMap` object for use as a data volume.
+
 ### Network
 
 #### --address `<string>`

docs/Researcher/cli-reference/runai-submit-dist-xgboost.md

@@ -268,6 +268,10 @@ runai submit-dist xgboost --name distributed-job --workers=2 -g 1 \
 > 
 > Mount /root/data to NFS path /public/data on NFS server nfs.example.com for read-write access.
 
+#### --configmap-volume name=<name of configmap>,path=<path to mount> ...'
+
+> Mount a `ConfigMap` object for use as a data volume.
+
 ### Network
 
 #### --address `<string>`

docs/Researcher/cli-reference/runai-submit.md

@@ -346,6 +346,10 @@ runai submit --job-name-prefix -i gcr.io/run-ai-demo/quickstart -g 1
 >
 > Mount /root/data to NFS path /public/data on NFS server nfs.example.com for read-write access.
 
+#### --configmap-volume name=<name of configmap>,path=<path to mount> ...'
+
+> Mount a `ConfigMap` object for use as a data volume.
+
 ### Network
 
 <!-- 

docs/Researcher/user-interface/workspaces/create/create-compute.md

@@ -7,6 +7,13 @@ You can select one or more resources. For example, one compute resource may cons
 !!! Note
     Selecting resources more than the cluster can supply will result in a permanently failed workspace.
 
+Use the *Cluster* filter at the top of the table to see compute resources that are assigned to specific clusters.
+
+!!! Note
+    The cluster filter will be in the top bar when there are clusters that are installed with version 2.16 or lower.
+
+Use the *Add filter* to add additional filters to the table.
+
 ## Set GPU resources
 
 GPU resources can be expressed in various ways:
@@ -33,7 +40,7 @@ A CPU resource consists of cores and memory. When GPU resources are requested th
 To create a compute resource:
 
 1. Select the `New Compute Resource` button.
-2. In the *Scope* pane, choose one item from the tree. The compute resource is assigned to that item and all its subsidiaries.
+2. In the *Scope* pane, choose a cluster, department, or project from the tree. The compute resource is assigned to that item and all its subsidiaries.
 3. Give the resource a meaningful name.
 4. In the resources pane, set the resource request.
       1. To add GPU resources, enter the number of GPUs to request. You can then enter the amount of GPU memory by selecting a percentage of the GPU, memory size in MB or GB, or multi-instance GPUs.

docs/Researcher/user-interface/workspaces/create/create-ds.md

@@ -7,7 +7,7 @@ When you select `New Compute Resource` you will be presented with various data s
 To create an NFS data source, provide:
 
 * A data source name.
-* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+* A Run:ai scope (cluster, department, or project) which is assigned to that item and all its subsidiaries.
 * An NFS server.
 * The path to the data within the server.
 * The path within the container where the data will be mounted.
@@ -19,58 +19,94 @@ The data can be set as read-write or limited to read-only permission regardless
 To create an PVC data source, provide:
 
 * A data source name
-* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+* A Run:ai scope (cluster, department, or project) which is assigned to that item and all its subsidiaries.
 * Select an existing PVC or create a new one by providing:
 
-    * A claim name
-    * A storage class
-    * Access mode
-    * Required storage size
-    * Volume system mode
+  * A claim name
+  * A storage class
+  * Access mode
+  * Required storage size
+  * Volume system mode
 
 * The path within the container where the data will be mounted.
 
+You can see the status of the resources created in the [Data sources table](#data-sources-table).
+
 ## Create an S3 data source
 
 S3 storage saves data in *buckets*. S3 is typically attributed to AWS cloud service but can also be used as a separate service unrelated to Amazon.
 
 To create an S3 data source, provide
 
 * A data source name
-* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+* A Run:ai scope (cluster, department, or project) which is assigned to that item and all its subsidiaries.
 * The relevant S3 service URL server
 * The bucket name of the data.
 * The path within the container where the data will be mounted.
 
-Note that an S3 data source can be public or private. For the latter option, please select the relevant credentials associated with the project to allow access to the data.
+An S3 data source can be public or private. For the latter option, please select the relevant credentials associated with the project to allow access to the data. S3 buckets that use credentials will have a status associated with it. For more information, see [Data sources table](#data-sources-table).
 
 ## Create a Git data source
 
 To create a Git data source, provide:
 
 * A data source name.
-* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+* A Run:ai scope (cluster, department, or project) which is assigned to that item and all its subsidiaries.
 * The relevant repository URL.
 * The path within the container where the data will be mounted.
 
-The Git data source can be public or private. To allow access to a private Git data source, you must select the relevant credentials associated with the project.
+The Git data source can be public or private. To allow access to a private Git data source, you must select the relevant credentials associated with the project. Git data sources that use credentials will have a status associated with it. For more information, see [Data sources table](#data-sources-table).
 
 ## Create a host path data source
 
 To create a host path data source, provide:
 
 * A data source name.
-* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+* A Run:ai scope (cluster, department, or project) which is assigned to that item and all its subsidiaries.
 * The relevant path on the host.
 * The path within the container where the data will be mounted.
 
 !!! Note
     The data can be limited to read-only permission regardless of any other user privileges.
 
-### Download Data Sources Table
+## Create a ConfigMap data source
+
+* A Run:ai project scope which is assigned to that item and all its subsidiaries.
+  
+    !!! Note
+        You can only choose a project as a scope.
+
+* A data source name.
+* A data mount consisting of:
+
+  * A ConfigMap name—select from the drop down.
+  * A target location—the path to the container.
+
+## Data sources table
+
+The *Data sources* table contains a column for the status of the data source. The following statuses are supported:
+
+| Status |  Description |
+| -- | -- |
+| **No issues found** | No issues were found when propagating the data source to the *PROJECTS*. |
+| **Issues found** | Failed to create the data source for some or all of the *PROJECTS*. |
+| **Issues found** | Failed to access the cluster. |
+| **Deleting** | The data source is being removed. |
+
+!!! Note
+
+    * The *Status* column in the table shows statuses based on your level of permissions. For example, a user that has create permissions for the scope, will see statuses that are calculated from the entire scope, while users who have only view and use permissions, will only be able to see statuses from a subset of the scope (assets that they have permissions to).
+    * The status of “-” indicates that there is no status because this asset is not cluster-syncing.
 
 You can download the Data Sources table to a CSV file. Downloading a CSV can provide a snapshot history of your Data Sources over the course of time, and help with compliance tracking. All the columns that are selected (displayed) in the table will be downloaded to the file.
 
+Use the *Cluster* filter at the top of the table to see data sources that are assigned to specific clusters.
+
+!!! Note
+    The cluster filter will be in the top bar when there are clusters that are installed with version 2.16 or lower.
+
+Use the *Add filter* to add additional filters to the table.
+
 To download the Data Sources table to a CSV:
 
 1. Open *Data Sources*.

docs/Researcher/user-interface/workspaces/create/create-env.md

@@ -3,30 +3,32 @@
 To create an environment:
 
 1. In the left menu, press *New Environment*.
-2. In the *Scope* pane, choose one item from the tree. The compute resource is assigned to that item and all its subsidiaries.
+2. In the *Scope* pane, choose a cluster, department, or project from the tree. The environment is assigned to that item and all its subsidiaries.
 3. Enter an *Environment name*.
 4. Enter the image URL path and an image pull policy.
 5. Choose a supported workload type. Configure this section based on the type of workload you expect to run in this environment. Choose from:
 
-      * `Single node`—use for running workloads on a single node.
-      * `Multi-node`—use for running distributed workloads on multiple nodes.
+      * *Standard*—use for running workloads on a single node.
+      * *Distributed*—use for running distributed workloads on multiple nodes.
 
     Then choose the workload that can use the environment:
 
-      * `Workspace`
-      * `Training`
-6. In the *Supported workload types* pane select either `Single node` or `Multi-node (Distributed)`.
+      * *Workspace*
+      * *Training*
+      * *Inference*
 
-      1. If you selected `Single node`, select `Workspace`, or `Training` or both.
-      2. If you selected `Multi-node (Distributed)`, select a framework from the dropdown, then select `Workspace`, or `Training` or both.
+    If you selected *Inference*, in the *endpoint* pane, select a *Protocol* from the dropdown, then enter the *Container port*.
 
-7. Select a tool from the list. You can add multiple tools by pressing *+ Tool*. Selecting a tool is optional.
+6. Select a tool from the list. You can add multiple tools by pressing *+ Tool*. Selecting a tool is optional.
 
     Tools can be:
 
       * Different applications such as Code editor IDEs (for example, VS Code), Experiment tracking (for example, Weight and Biases), visualization tools (for example, Tensor Board), and more.
       * Open source tools (for example, Jupyter notebook) or commercial 3rd party tools (for example,. MATLAB)
 
+    !!! Note
+        Tool configuration is not supported with *Inference* environments.
+
     It is also possible to set up a custom tool used by the organization.
 
     For each tool, you must set the type of connection interface and port. If not set, default values are provided. The supported connection types are:
@@ -35,12 +37,14 @@ To create an environment:
       * External node port: A [NodePort](../../../../admin/runai-setup/config/allow-external-access-to-containers.md) exposes your application externally on every host of the cluster, access the tool using `http://<HOST_IP>:<NODEPORT>` (for example, http://203.0.113.20:30556).
 
     !!! Note
-        Selecting a tool requires configuration to be up and running. To configure a tool:
+        Selecting a tool requires a configuration to be up and running.
+
+    To configure a tool:
 
     * The container image needs to support the tool.
     * The administrator must configure a DNS record and certificate. For more information, see [Workspaces configuration](../../../../admin/runai-setup/config/allow-external-access-to-containers.md#workspaces-configuration).
 
-8. Configure runtime settings with:
+7. Configure runtime settings with:
 
        1. Commands and arguments&mdash;visible, but not editable in the workspace creation form.
        2. Environment variables&mdash;visible and editable in the workspace creation form.
@@ -49,22 +53,29 @@ To create an environment:
     !!! Note
         The value of an environment variable can remain empty for the researcher to fill in when creating a workspace.
 
-9. Configure the security settings from:
+8. Configure the security settings from:
 
        1. Settings in the image&mdash;security settings that come with the image file. 
        2. Custom settings:
 
-            1. User ID.
-            2. Group ID.
-            3. Supplementary Groups.
-            4. Values modification settings.
+          1. User ID.
+          2. Group ID.
+          3. Supplementary Groups.
+          4. Values modification settings.
 
        3. Add linux capabilities.
 
 ## Download Environments Table
 
 You can download the Environments table to a CSV file. Downloading a CSV can provide a snapshot history of your environments over the course of time, and help with compliance tracking. All the columns that are selected (displayed) in the table will be downloaded to the file.
 
+Use the *Cluster* filter at the top of the table to see environments that are assigned to specific clusters.
+
+!!! Note
+    The cluster filter will be in the top bar when there are clusters that are installed with version 2.16 or lower.
+
+Use the *Add filter* to add additional filters to the table.
+
 To download the Environments table to a CSV:
 
 1. In the left menu, press *Environments*.