Merge branch 'main' into fix_aqua_vision_model

Author: mrDzurb

ads/opctl/operator/lowcode/forecast/MLoperator

@@ -2,12 +2,12 @@ type: forecast
 version: v1
 name: Forecasting Operator
 conda_type: service
-conda: forecast_p38_cpu_v1
+conda: forecast_p310_cpu_x86_64_v4
 gpu: no
 jobs_default_params:
   shape_name: VM.Standard.E4.Flex
-  ocpus: 32
-  memory_in_gbs: 512
+  ocpus: 16
+  memory_in_gbs: 256
   block_storage_size_in_GBs: 512
 keywords:
   - Prophet

ads/opctl/operator/lowcode/forecast/model/automlx.py

@@ -49,7 +49,7 @@ def set_kwargs(self):
         time_budget = model_kwargs_cleaned.pop("time_budget", -1)
         model_kwargs_cleaned[
             "preprocessing"
-        ] = self.spec.preprocessing or model_kwargs_cleaned.get("preprocessing", True)
+        ] = self.spec.preprocessing.enabled or model_kwargs_cleaned.get("preprocessing", True)
         return model_kwargs_cleaned, time_budget
 
     def preprocess(self, data, series_id=None):  # TODO: re-use self.le for explanations

ads/opctl/operator/lowcode/forecast/model/forecast_datasets.py

@@ -68,7 +68,7 @@ def __init__(self, spec, historical_data):
             add_dates.sort()
             if historical_data.get_max_time() > add_dates[-spec.horizon]:
                 raise DataMismatchError(
-                    f"The Historical Data ends on {historical_data.get_max_time()}. The additional data horizon starts on {add_dates[-spec.horizon]}. The horizon should have exactly {spec.horizon} dates after the Hisotrical at a frequency of {historical_data.freq}"
+                    f"The Historical Data ends on {historical_data.get_max_time()}. The additional data horizon starts on {add_dates[-spec.horizon]}. The horizon should have exactly {spec.horizon} dates after the Historical at a frequency of {historical_data.freq}"
                 )
             elif historical_data.get_max_time() != add_dates[-(spec.horizon + 1)]:
                 raise DataMismatchError(

docs/source/index.rst

@@ -20,15 +20,15 @@ Oracle Accelerated Data Science (ADS)
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Getting Started:
+   :caption: Getting Started
 
    release_notes
    user_guide/quick_start/quick_start
 
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Installation and Configuration:
+   :caption: Installation and Configuration
 
    user_guide/cli/quickstart
    user_guide/cli/authentication
@@ -38,19 +38,19 @@ Oracle Accelerated Data Science (ADS)
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Low-Code AI Operators:
+   :caption: Low-Code AI Operators
 
    user_guide/operators/index
-   user_guide/operators/common/index
-   user_guide/operators/forecasting_operator/index
+   user_guide/operators/forecast_operator/index
    user_guide/operators/anomaly_detection_operator/index
    user_guide/operators/pii_operator/index
    user_guide/operators/recommender_operator/index
+   user_guide/operators/common/index
 
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Tasks:
+   :caption: Tasks
 
    user_guide/loading_data/connect
    user_guide/data_labeling/index
@@ -62,7 +62,7 @@ Oracle Accelerated Data Science (ADS)
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Integrations:
+   :caption: Integrations
 
    user_guide/apachespark/spark
    user_guide/big_data_service/index
@@ -76,7 +76,7 @@ Oracle Accelerated Data Science (ADS)
 .. toctree::
    :hidden:
    :maxdepth: 5
-   :caption: Classes:
+   :caption: Classes
 
    modules
 

docs/source/release_notes.rst

@@ -2,6 +2,12 @@
 Release Notes
 =============
 
+2.12.0
+-------
+Release date: October 1, 2024
+
+* Update LangChain Integration to add support for chat model, streaming and async calls.
+
 2.11.19
 -------
 Release date: September 24, 2024
@@ -140,7 +146,7 @@ Release date: March 20, 2024
 Release date: February 7, 2024
 
 * Releasing v1 of the Anomaly Detection Operator! The Anomaly Detection Operator is a no-code Anomaly or Outlier Detection solution through the OCI Data Science Platform. It uses dozens of models from Oracle’s own proprietary research and the best of open source. See the ``Anomaly Detection`` Section of the ``AI Operators`` tab for full details (:doc:`link <./user_guide/operators/anomaly_detection_operator/index>`).
-* Releasing a new version of the Forecast Operator. This release has faster explainability, improved support for reading from databases, upgrades to the automatic reporting, improved parallelization across all models, and an ability to save models for deferred inference. See the ``Forecast`` Section of the ``AI Operators`` tab for full details (:doc:`link <./user_guide/operators/forecasting_operator/index>`).
+* Releasing a new version of the Forecast Operator. This release has faster explainability, improved support for reading from databases, upgrades to the automatic reporting, improved parallelization across all models, and an ability to save models for deferred inference. See the ``Forecast`` Section of the ``AI Operators`` tab for full details (:doc:`link <./user_guide/operators/forecast_operator/index>`).
 * Change to the default signer such that it now defaults to ``resource_prinicpal`` on any OCI Data Science resource (for example, jobs, notebooks, model deployments, dataflow).
 
 2.10.0

docs/source/user_guide/large_language_model/langchain_models.rst

@@ -6,12 +6,12 @@ LangChain Integration
 .. admonition:: LangChain Community
   :class: note
 
-  While the stable integrations (such as ``OCIModelDeploymentVLLM`` and ``OCIModelDeploymentTGI``) are also available from `LangChain Community <https://python.langchain.com/docs/integrations/llms/oci_model_deployment_endpoint>`_, integrations from ADS may provide additional or experimental features in the latest updates, .
+  While the stable integrations (such as ``ChatOCIModelDeploymentVLLM`` and ``OCIModelDeploymentVLLM``) are also available from `LangChain Community <https://python.langchain.com/docs/integrations/llms/oci_model_deployment_endpoint>`_, integrations from ADS may provide additional or experimental features in the latest updates.
 
 .. admonition:: Requirements
   :class: note
 
-  The LangChain integration requires ``python>=3.9`` and ``langchain>=0.3``
+  The LangChain integration requires ``python>=3.9`` and ``langchain>=0.3``. Chat model also requires ``langchain-openai``.
 
 
 LangChain compatible models/interfaces are needed for LangChain applications to invoke LLMs deployed on OCI data science model deployment service.

docs/source/user_guide/operators/anomaly_detection_operator/faq.rst

@@ -9,7 +9,3 @@ More details in the documentation here: https://docs.oracle.com/en-us/iaas/tools
 **How do I learn More about AutoTS?**
 
 More details in the documentation here: https://winedarksea.github.io/AutoTS/build/html/source/tutorial.html
-
-**Pip Install Failing with "ERROR: No matching distribution found for oracle-automlx==23.4.1; extra == "forecast""**
-
-Automlx only supports Python<=3.8, != 3.9, <= 3.10.7 . If you are builing in Python 3.9 or 3.10.8+, no automlx distribution will be available. It's recommended to use the conda pack available through Notebook Sessions, or to use Python 3.8

docs/source/user_guide/operators/anomaly_detection_operator/install.rst

@@ -12,7 +12,7 @@ Installing Through PyPi
 
 If you are running the operator from outside of a Notebook Session, you may download ``oracle_ads[anomaly]`` from pypi. 
 
-*Note: Due to our dependence on Automlx, ``oracle_ads[anomaly]``  only supports Python<=3.8, != 3.9, <= 3.10.7 . Python 3.8 is the recommended version.*
+*Note: Python 3.10 is recommended.*
 
 .. code-block:: bash
 

docs/source/user_guide/operators/common/index.rst

@@ -1,6 +1,6 @@
-===============
-Getting Started
-===============
+====================
+More About Operators
+====================
 
 Welcome to the world of operators! Getting started with operators is a breeze, and this section will guide you through the process step by step. Whether you're a seasoned data scientist or a newcomer, you'll find that harnessing the power of operators is both accessible and rewarding.
 

docs/source/user_guide/operators/forecast_operator/data_sources.rst

@@ -0,0 +1,98 @@
+=================
+Data Integration
+=================
+
+Supported Data Sources
+----------------------
+
+The Operator can read data from the following sources:
+
+- Oracle RDBMS
+- OCI Object Storage
+- OCI Data Lake
+- HTTPS
+- S3
+- Azure Blob Storage
+- Google Cloud Storage
+- Local file systems
+
+Additionally, the operator supports any data source supported by `fsspec <https://filesystem-spec.readthedocs.io/en/latest/_modules/fsspec/registry.html>`_.
+
+Examples
+--------
+
+Reading from OCI Object Storage
+===============================
+
+Below is an example of reading data from OCI Object Storage using the operator:
+
+.. code-block:: yaml
+
+    kind: operator
+    type: forecast
+    version: v1
+    spec:
+        datetime_column:
+            name: ds
+        historical_data:
+            url: oci://<bucket_name>@<namespace_name>/example_yosemite_temps.csv
+        horizon: 3
+        target_column: y
+
+Reading from Oracle Database
+============================
+
+Below is an example of reading data from an Oracle Database:
+
+.. code-block:: yaml
+
+    kind: operator
+    type: forecast
+    version: v1
+    spec:
+        historical_data:
+            connect_args:
+                user: XXX
+                password: YYY
+                dsn: "localhost/orclpdb"
+            sql: 'SELECT Store_ID, Sales, Date FROM live_data'
+        datetime_column:
+            name: ds
+        horizon: 1
+        target_column: y
+
+
+Data Preprocessing
+------------------
+
+The forecasting operator simplifies powerful data preprocessing. By default, it includes several preprocessing steps to ensure dataset compliance with each framework. However, users can disable one or more of these steps if needed, though doing so may cause the model to fail. Proceed with caution.
+
+Default preprocessing steps:
+- Missing value imputation
+- Outlier treatment
+
+To disable ``outlier_treatment``, modify the YAML file as shown below:
+
+.. code-block:: yaml
+
+    kind: operator
+    type: forecast
+    version: v1
+    spec:
+        datetime_column:
+            name: ds
+        historical_data:
+            url: https://raw.githubusercontent.com/facebook/prophet/main/examples/example_yosemite_temps.csv
+        horizon: 3
+        target_column: y
+        preprocessing: 
+            enabled: true
+            steps:
+                missing_value_imputation: True
+                outlier_treatment: False
+
+
+Real-Time Trigger
+-----------------
+
+The Operator can be run locally or on an OCI Data Science Job. The resultant model can be saved and deployed for future use if needed. For questions regarding this integration, please reach out to the OCI Data Science team.