docs/odsc 41635/update doc for dataflow pool support (#224)

Author: mingkang111

docs/source/user_guide/apachespark/dataflow-spark-magic.rst

@@ -191,6 +191,26 @@ Example path : ``oci://<your-bucket>@<your-tenancy-namespace>/conda_environments
     "configuration":{\
       "spark.archives": "oci://<your-bucket>@<your-tenancy-namespace>/conda_environments/cpu/PySpark 3.2 and Data Flow/2.0/pyspark32_p38_cpu_v2#conda>"}}'
 
+**Example command with the Data Flow Pools**
+
+.. versionadded:: 2.8.7
+
+The `Data Flow Pools <https://docs.oracle.com/en-us/iaas/data-flow/using/pools.htm>`__  achieve fast job startup, resource isolation, manage budgets, and prioritize your Spark workloads. Use the `poolId` to use the Pool resources.
+
+.. code-block:: python
+
+  %create_session -l python -c '{\
+    "compartmentId":"<compartment_id>",\
+    "displayName":"TestDataFlowSession",\
+    "sparkVersion":"3.2.1",\
+    "driverShape":"VM.Standard.E4.Flex",\
+    "executorShape":"VM.Standard.E4.Flex",\
+    "numExecutors":1,\
+    "driverShapeConfig":{"ocpus":1,"memoryInGBs":16},\
+    "executorShapeConfig":{"ocpus":1,"memoryInGBs":16},\
+    "poolId": "<ocid1.dataflowpool...>",\
+    "logsBucketUri" : "oci://<bucket_name>@<namespace>/"}'
+
 
 Update Session
 **************
@@ -296,4 +316,4 @@ Check the result:
 .. code-block:: python
 
   print(type(df_nyc_tlc))
-  df_nyc_tlc.head()
+  df_nyc_tlc.head()

docs/source/user_guide/apachespark/dataflow.rst

@@ -37,6 +37,8 @@ Define config. If you have not yet configured your dataflow setting, or would li
   dataflow_config.spark_version = "3.2.1"
   dataflow_config.configuration = {"spark.driver.memory": "512m"}
   dataflow_config.private_endpoint_id = "ocid1.dataflowprivateendpoint.oc1.iad.<your private endpoint ocid>"
+  # For using Data Flow Pools
+  # dataflow_config.poolId = "ocid1.dataflowpool.oc1..<unique_ocid>"
 
 Use the config defined above to submit the cell.
 
@@ -207,6 +209,7 @@ You can set them using the ``with_{property}`` functions:
 - ``with_spark_version``
 - ``with_warehouse_bucket_uri``
 - ``with_private_endpoint_id`` (`doc <https://docs.oracle.com/en-us/iaas/data-flow/using/pe-allowing.htm#pe-allowing>`__)
+- ``with_pool_id`` (`doc <https://docs.oracle.com/en-us/iaas/data-flow/using/pools.htm>`__)
 - ``with_defined_tags``
 - ``with_freeform_tags``
 
@@ -274,6 +277,8 @@ accepted. In the next example, the prefix is given for ``script_bucket``.
             .with_executor_shape("VM.Standard.E4.Flex")
             .with_executor_shape_config(ocpus=4, memory_in_gbs=64)
             .with_spark_version("3.0.2")
+            # For using Data Flow Pool
+            # .with_pool_id("ocid1.dataflowpool.oc1..<unique_ocid>")
             .with_defined_tag(
                 **{"Oracle-Tags": {"CreatedBy": "test_name@oracle.com"}}
             )
@@ -576,6 +581,7 @@ into the ``Job.from_yaml()`` function to build a Data Flow job:
         numExecutors: 1
         sparkVersion: 3.2.1
         privateEndpointId: <private_endpoint_ocid>
+        poolId: <dataflow_pool_ocid>
         definedTags:
           Oracle-Tags:
             CreatedBy: test_name@oracle.com
@@ -659,6 +665,9 @@ into the ``Job.from_yaml()`` function to build a Data Flow job:
             privateEndpointId:
                 required: false
                 type: string
+            poolId:
+                required: false
+                type: string
             configuration:
                 required: false
                 type: dict