ODSC-36358. [ads_docs] Add raw-request examples for Deployment section (#31)

oci cli raw-request examples to model deployment prediction

Author: liudmylaru

docs/source/user_guide/cli/opctl/localdev/condapack.rst

@@ -24,6 +24,14 @@ create
 
 Build conda packs from your workstation using ``ads opctl conda create`` subcommand.
 
+.. admonition:: Tip
+
+    To publish a conda pack that is natively installed on a oracle linux host (compute or laptop), use ``NO_CONTAINER`` environment variable to remove dependency on the ml-job container image:
+
+    .. code-block:: shell
+
+        NO_CONTAINER=1 ads opctl conda publish -s <slug> --auth <api_key/instance_principal/resource_principal>
+
 -------
 publish
 -------
@@ -32,8 +40,7 @@ publish
 
   ads opctl conda publish -s <slug>
 
-Publish conda pack to the object storage bucket from your laptop or workstation. You can use this conda pack inside ``OCI Data Science Jobs``, ``OCI Data Science Notebooks`` and ``OCI Data Science Model Deployment``
-
+Publish conda pack to the object storage bucket from your laptop or workstation. You can use this conda pack inside ``OCI Data Science Service`` or ``Data Flow service``.
 
 -------
 install
@@ -43,4 +50,4 @@ Install conda pack using its URI. The conda pack can be used inside the docker i
 
 .. code-block:: shell
 
-  ads opctl conda install -u "oci://mybucket@namespace/conda_environment/path/to/my/conda"
+  ads opctl conda install -u "oci://mybucket@namespace/conda_environment/path/to/my/conda"

docs/source/user_guide/model_registration/frameworks/lightgbmmodel.rst

@@ -121,6 +121,128 @@ Run Prediction against Endpoint
 
     [1,0,...,1]
 
+Run Prediction with oci raw-request command
+===========================================
+
+Model deployment endpoints can be invoked with the OCI-CLI. The below examples invoke a model deployment with the CLI with different types of payload: ``json``,  ``numpy.ndarray``, ``pandas.core.frame.DataFrame`` or ``dict``.
+
+`json` payload example
+----------------------
+
+.. code-block:: python3
+
+    >>> # Prepare data sample for prediction
+    >>> data = testx[[11]]
+    >>> data
+    array([[ 0.59990614,  0.95516275, -1.22366985, -0.89270887, -1.14868768,
+        -0.3506047 ,  0.28529227,  2.00085413,  0.31212668,  0.39411511,
+         0.87301082, -0.01743923, -1.15089633,  1.03461823,  1.50228029]])
+
+Use printed output of the data and endpoint to invoke prediction with raw-request command in terminal:
+
+.. code-block:: bash
+
+    export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict
+    export data='{"data": [[ 0.59990614,  0.95516275, ... , 1.50228029]]}'
+    oci raw-request \
+        --http-method POST \
+        --target-uri $uri \
+        --request-body "$data"
+
+`numpy.ndarray` payload example
+-------------------------------
+
+.. code-block:: python3
+
+    >>> # Prepare data sample for prediction
+    >>> from io import BytesIO
+    >>> import base64
+    >>> import numpy as np
+
+    >>> data = testx[[10]]
+    >>> np_bytes = BytesIO()
+    >>> np.save(np_bytes, data, allow_pickle=True)
+    >>> data = base64.b64encode(np_bytes.getvalue()).decode("utf-8")
+    >>> print(data)
+    k05VTVBZAQB2AHsnZGVzY......D1cJ+D8=
+
+Use printed output of `base64` data and endpoint to invoke prediction with raw-request command in terminal:
+
+.. code-block:: bash
+
+    export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict
+    export data='{"data":"k05VTVBZAQB2AHsnZGVzY......4UdN0L8=", "data_type": "numpy.ndarray"}'
+    oci raw-request \
+        --http-method POST \
+        --target-uri $uri \
+        --request-body "$data"
+
+`pandas.core.frame.DataFrame` payload example
+---------------------------------------------
+
+.. code-block:: python3
+
+    >>> # Prepare data sample for prediction
+    >>> import pandas as pd
+
+    >>> df = pd.DataFrame(testx[[10]])
+    >>> print(json.dumps(df.to_json())
+    "{\"0\":{\"0\":0.4133005141},\"1\":{\"0\":0.676589266},...,\"14\":{\"0\":-0.2547168443}}"
+
+Use printed output of ``DataFrame`` data and endpoint to invoke prediction with raw-request command in terminal:
+
+.. code-block:: bash
+
+    export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict
+    export data='{"data":"{\"0\":{\"0\":0.4133005141},...,\"14\":{\"0\":-0.2547168443}}","data_type":"pandas.core.frame.DataFrame"}'
+    oci raw-request \
+        --http-method POST \
+        --target-uri $uri \
+        --request-body "$data"
+
+`dict` payload example
+----------------------
+
+    >>> # Prepare data sample for prediction
+    >>> import pandas as pd
+
+    >>> df = pd.DataFrame(testx[[11]])
+    >>> print(json.dumps(df.to_dict()))
+    {"0": {"0": 0.5999061426438217}, "1": {"0": 0.9551627492226553}, ...,"14": {"0": 1.5022802918908846}}
+
+Use printed output of ``dict`` data and endpoint to invoke prediction with raw-request command in terminal:
+
+.. code-block:: bash
+
+    export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict
+    export data='{"data": {"0": {"0": 0.5999061426438217}, ...,"14": {"0": 1.5022802918908846}}}'
+    oci raw-request \
+        --http-method POST \
+        --target-uri $uri \
+        --request-body "$data"
+
+Expected output of raw-request command
+--------------------------------------
+
+.. code-block:: bash
+
+    {
+      "data": {
+        "prediction": [
+          1
+        ]
+      },
+      "headers": {
+        "Connection": "keep-alive",
+        "Content-Length": "18",
+        "Content-Type": "application/json",
+        "Date": "Wed, 07 Dec 2022 18:31:05 GMT",
+        "X-Content-Type-Options": "nosniff",
+        "opc-request-id": "B67EED723ADD43D7BBA1AD5AFCCBD0C6/03F218FF833BC8A2D6A5BDE4AB8B7C12/6ACBC4E5C5127AC80DA590568D628B60",
+        "server": "uvicorn"
+      },
+      "status": "200 OK"
+    }
 
 Example
 =======

docs/source/user_guide/model_registration/frameworks/pytorchmodel.rst

@@ -267,17 +267,151 @@ Predict with Image
 
 Predict Image by passing a uri, which can be http(s), local path, or other URLs
 (e.g. starting with “oci://”, “s3://”, and “gcs://”), of the image or a PIL.Image.Image object
-using the `image` argument in `predict()` to predict a single image.
+using the ``image`` argument in ``predict()`` to predict a single image.
 The image will be converted to a tensor and then serialized so it can be passed to the endpoint.
-You can catch the tensor in `score.py` to perform further transformation.
+You can catch the tensor in ``score.py`` to perform further transformation.
+
+.. note::
+    The payload size limit is 10 MB. Read more about invoking a model deployment `here <https://docs.oracle.com/iaas/data-science/using/model-dep-invoke.htm#model_dep_invoke>`_.
+
+Given the size limitation, the example below is with resized image. To pass an image and invoke prediction, additional code inside score.py is required to preprocess the data. Open ``pytorch_model_artifact/score.py`` and update the ``pre_inference()`` method. The edits are highlighted:
+
+.. code-block:: python3
+    :emphasize-lines: 15-26
+
+    def pre_inference(data, input_schema_path):
+        """
+        Preprocess json-serialized data to feed into predict function.
+
+        Parameters
+        ----------
+        data: Data format as expected by the predict API of the core estimator.
+        input_schema_path: path of input schema.
+
+        Returns
+        -------
+        data: Data format after any processing.
+        """
+        data = deserialize(data, input_schema_path)
+        import torchvision.transforms as transforms
+        preprocess = transforms.Compose([
+            transforms.Resize(256),
+            transforms.CenterCrop(224),
+            transforms.Normalize(
+                mean=[0.485, 0.456, 0.406],
+                std=[0.229, 0.224, 0.225]
+            ),
+        ])
+        input_tensor = preprocess(data)
+        input_batch = input_tensor.unsqueeze(0)
+        return input_batch
+
+Save ``score.py`` and verify prediction works:
+
+.. code-block:: python3
+
+    >>> uri = ("https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/model_deploy_examples/images/dog_resized.jpg")
+    >>> prediction = pytorch_model.verify(image=uri)["prediction"]
+    >>> import numpy as np
+    >>> np.argmax(prediction)
+    258
+
+Re-deploy model with updated ``score.py``:
+
+.. code-block:: python3
+
+    pytorch_model.deploy(
+        display_name="PyTorch Model For Classification",
+        deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxxx",
+        deployment_access_log_id="ocid1.log.oc1.xxx.xxxxxx",
+        deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxxx",
+    )
+
+Run prediction with the image provided:
 
 .. code-block:: python3
 
-    uri = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg")
+    uri = ("https://github.com/oracle-samples/oci-data-science-ai-samples/tree/master/model_deploy_examples/images/dog_resized.jpg")
 
     # Generate prediction by invoking the deployed endpoint
     prediction = pytorch_model.predict(image=uri)['prediction']
 
+Run Prediction with oci raw-request command
+===========================================
+
+Model deployment endpoints can be invoked with the OCI-CLI. This example invokes a model deployment with the CLI with a ``torch.Tensor`` payload:
+
+`torch.Tensor` payload example
+------------------------------
+
+.. code-block:: python3
+
+    >>> # Prepare data sample for prediction and save it to file 'data-payload'
+    >>> from io import BytesIO
+    >>> import base64
+
+    >>> buffer = BytesIO()
+    >>> torch.save(input_batch, buffer)
+    >>> data = base64.b64encode(buffer.getvalue()).decode("utf-8")
+    >>> with open('data-payload', 'w') as f:
+    >>>     f.write('{"data": "' + data + '", "data_type": "torch.Tensor"}')
+
+File ``data-payload`` will have this information:
+
+.. code-block:: bash
+
+    {"data": "UEsDBAAACAgAAAAAAAAAAAAAAAAAAAAAAAAQ ........................
+    .......................................................................
+    ...AAAAEAAABQSwUGAAAAAAMAAwC3AAAA0jEJAAAA", "data_type": "torch.Tensor"}
+
+Use file ``data-payload`` with data and endpoint to invoke prediction with raw-request command in terminal:
+
+.. code-block:: bash
+
+    export uri=https://modeldeployment.{region}.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.xxx.xxxxx/predict
+    oci raw-request \
+        --http-method POST \
+        --target-uri $uri \
+        --request-body file://data-payload
+
+Expected output of raw-request command
+--------------------------------------
+
+.. code-block:: bash
+
+    {
+      "data": {
+        "prediction": [
+          [
+            0.0159152802079916,
+            -1.5496551990509033,
+            .......
+            2.5116958618164062
+          ]
+        ]
+      },
+      "headers": {
+        "Connection": "keep-alive",
+        "Content-Length": "19398",
+        "Content-Type": "application/json",
+        "Date": "Thu, 08 Dec 2022 18:28:41 GMT",
+        "X-Content-Type-Options": "nosniff",
+        "opc-request-id": "BD80D931A6EA4C718636ECE00730B255/86111E71C1B33C24988C59C27F15ECDE/E94BBB27AC3F48CB68F41135073FF46B",
+        "server": "uvicorn"
+      },
+      "status": "200 OK"
+    }
+
+Copy prediction output into `argmax` to retrieve result of the image prediction:
+
+.. code-block:: python3
+
+    >>> print(np.argmax([[0.0159152802079916,
+    >>>                   -1.5496551990509033,
+    >>>                   .......
+    >>>                   2.5116958618164062]]))
+    258
+
 Example
 =======