|
1 | | -## Intel AI Inference E2E Solution for OpenShift |
| 1 | +# Overview |
| 2 | +Intel AI inference end-to-end solution with RHOCP is based on the Intel® Data Center GPU Flex Series provisioning, Intel® OpenVINO™, and [Red Hat OpenShift Data Science](https://www.redhat.com/en/technologies/cloud-computing/openshift/openshift-data-science) (RHODS) on RHOCP. There are two AI inference modes verified with Intel® Xeon® processors and Intel Data Center GPU Flex Series with RHOCP-4.12. |
| 3 | +* Interactive mode – RHODS provides OpenVINO based Jupyter Notebooks for users to interactively debug the inference applications or [optimize the models](https://docs.openvino.ai/2023.0/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) on RHOCP using data center GPU cards or Intel Xeon processors. |
| 4 | +* Deployment mode – [OpenVINO Model Sever](https://github.com/openvinotoolkit/model_server) (OVMS) can be used to deploy the inference workloads in data center and edge computing environments on RHOCP. |
| 5 | +## Prerequisites |
| 6 | +* Provisioned RHOCP 4.12 cluster. Follow steps [here](https://github.com/intel/intel-technology-enabling-for-openshift/tree/main#provisioning-rhocp-cluster). |
| 7 | +* Provisioning Intel Data Center GPU Flex Series [link](https://github.com/intel/intel-technology-enabling-for-openshift/tree/main#provisioning-intel-hardware-features-on-rhocp) |
| 8 | +* Setup node feature discovery (NFD). Follow the steps [here](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/nfd/README.md). |
| 9 | +* Setup machine configuration. Follow the steps [here](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/machine_configuration/README.md). |
| 10 | +* Setup out of tree drivers for Intel GPU provisioning. Follow the steps [here](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/machine_configuration/README.md). |
| 11 | +* Setup Intel device plugins operator and create Intel GPU device plugin [CR]([link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/device_plugins/deploy_gpu.md)) |
| 12 | +* Using Intel Data Center GPU Resource Exclusively [link](https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/device_plugins/deploy_gpu.md#using-intel-data-center-gpu-resource-exclusively) |
| 13 | +## Install RHODS |
| 14 | +The Red Hat certified RHODS operator is published at [Red Hat Ecosystem Catalog](https://catalog.redhat.com/software/container-stacks/detail/63b85b573112fe5a95ee9a3a). You can use the command line interface (CLI) or web console to install it. |
| 15 | +### Install using CLI (To be added) |
| 16 | +### Install using web console |
| 17 | +1. On the RHOCP web console, click Operators → OperatorHub. |
| 18 | +2. Search RedHat OpenShift Data Science Operator and click Install. The operator is installed in the namespace redhat-ods-operator. |
| 19 | +### Verification |
| 20 | +1. Navigate to Operators → Installed Operators page. |
| 21 | +2. Ensure that in the redhat-ods-operator namespace, RedHat OpenShift Data Science Status is InstallSucceeded |
| 22 | +3. Click on ```Search -> Routes -> rhods-dashboard``` from the web console and access the RHODS UI link. |
| 23 | +Note: When installing the operator, the default ```kfdef``` Custom Resource (CR) is created. This CR enables the RHODS dashboard for users to browse and launch Jupyter Notebooks projects on an RHOCP cluster. Please refer to this [link](https://github.com/red-hat-data-services/odh-deployer) for more details about kfdef. |
| 24 | +## Install OpenVINO operator |
| 25 | +The OpenVINO operator is published at [Red Hat Ecosystem Catalog](https://catalog.redhat.com/software/container-stacks/detail/60649a56209af65d24b7ca9e). You can use the CLI or web console to install it. |
| 26 | +### Install using CLI (To be added) |
| 27 | +### Install using web console |
| 28 | +Follow this [link](https://github.com/openvinotoolkit/operator/blob/v1.1.0/docs/operator_installation.md) to install the operator via the web console. |
| 29 | +## Work with interactive mode |
| 30 | +To enable the interactive mode, the OpenVINO notebook CR needs to be created and integrated with RHODS. |
| 31 | +1. Click on the ```create Notebook``` option in this [link](https://github.com/red-hat-data-services/odh-deployer) from the web console and follow these [steps](https://github.com/openvinotoolkit/operator/blob/main/docs/notebook_in_rhods.md) to create the notebook CR. |
| 32 | +2. Enable Intel Data Center GPU |
| 33 | +**Note:** The Intel Data Center GPU option is not visible in the RHODS UI at this time of release. For more details, please refer to this [issue](https://github.com/opendatahub-io/odh-dashboard/issues/956). Until this issue is resolved, please follow the steps below to enable the Intel Data Center GPU. |
| 34 | +a. Search for the OpenVINO Notebook Server from web console ```Search -> Notebook -> Jupyter-nb-<ocp-user>``` in the namespace ```rhods-notebooks```. |
| 35 | +b. Navigate to notebook yaml and modify the yaml file according to the example shown below. |
| 36 | + ``` |
| 37 | + containers: |
| 38 | + name: jupyter-nb-<ocp-user-name> |
| 39 | + resources: |
| 40 | + limits: |
| 41 | + cpu: '14' |
| 42 | + gpu.intel.com/i915: '1' |
| 43 | + memory: 56Gi |
| 44 | + requests: |
| 45 | + cpu: '7' |
| 46 | + gpu.intel.com/i915: '1' |
| 47 | + memory: 56Gi |
| 48 | + ``` |
| 49 | +c. This operation respawns the notebook server to use the Intel Data Center GPU. |
2 | 50 |
|
3 | | -### Overview |
4 | | -Intel AI inference e2e solution for OCP is built upon Intel® dGPU provisioning for OpenShift and Intel® Xeon® processors. The two following AI inference modes are used to test with the Intel Data Center GPU Card provisioning: |
5 | | -* **Interactive Mode** |
6 | | -[Open Data Hub (ODH)](https://github.com/opendatahub-io) and [Red Hat OpenShift Data Science (RHODS)](https://www.redhat.com/en/technologies/cloud-computing/openshift/openshift-data-science) provides Intel® OpenVINO™ based [Jupiter Notebook](https://jupyter.org/) to help users interactively debug the inferencing applications or optimize the models with OCP using Intel Data Center GPU cards and Intel Xeon processors. |
7 | | -* **Deployment Mode** |
8 | | -[Intel OpenVINO™ Toolkit](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html) and [Operator](https://github.com/openvinotoolkit/operator) provide the [OpenVINO Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) for users to deploy their inferencing workload using Intel Data Center GPU cards and Intel Xeon processors on OCP cluster in cloud or edge environment. |
| 51 | +3. Run sample Jupyter Notebooks. |
| 52 | +Follow the [link](https://github.com/openvinotoolkit/operator/blob/main/docs/notebook_in_rhods.md) to execute the sample Jupyter Notebook There are 60+ sample notebooks available with this notebook image. For the details on the notebooks with Intel Data Center GPU, please check this [link](https://github.com/openvinotoolkit/openvino_notebooks). |
| 53 | +## Work with deployment mode |
| 54 | +1. From the web console, click on the ModelServer option in this [link](https://github.com/openvinotoolkit/operator/blob/v1.1.0/docs/operator_installation.md) and follow the [steps](https://github.com/openvinotoolkit/operator/blob/v1.1.0/docs/modelserver.md) to start the OVMS instance. |
| 55 | +2. To enable the Intel Data Center GPU, make sure to modify the OVMS instance options according to the screenshot below. |
9 | 56 |
|
10 | | - `note: The verification on this mode is ongoing` |
| 57 | +* Add 2 screenshots |
11 | 58 |
|
12 | | -### Deploy Intel AI Inference E2E Solution |
13 | | - |
14 | | -* **Install RHODS on OpenShift** |
15 | | -* **Install Intel OpenVINO Toolkit Operator** |
16 | | - |
17 | | -### Run Interactive Mode Demo |
18 | | - |
19 | | -### Run Deployment Mode Demo |
| 59 | +## See Also |
| 60 | +[GPU accelerated demo with OpenVINO](https://www.youtube.com/watch?v=3fTz_k4JT2A) |
0 commit comments