Merge pull request #52 from GoogleCloudPlatform/feature/monitoring

Feature/monitoring

Author: srastatter

AutoMLOps_User_Guide.pdf

@@ -1,6 +1,29 @@
 # Change Log
 All notable changes to this project will be documented in this file.
 
+## [1.2.9] - 2024-1-18
+
+### Added
+
+- Added `setup_model_monitoring` parameter to `AutoMLOps.generate` and orchestration/configs.py
+- Added the creation of a model_monitoring/ directory and `create_model_monitoring_job.sh` script to generate
+- Added a new `AutoMLOps.monitor` function, along with relevant templates and tests
+- Added in a monitoring section into the generated `config/defaults.yaml` file
+- Added `logging.googleapis.com` to the list of potentially required apis.
+- Created 2 new functions for giving warnings when running `AutoMLOps.monitor`: `get_model_monitoring_min_permissions` and `get_model_monitoring_recommended_roles` in utils.py
+
+### Changed
+
+- Updated the `services/submission_service/main.py.j2` to include elements for automatic retraining based on monitoring anomaly logs, and adding in labels to the submit.
+- Updated `services/submission_service/requirements.txt` to include google-cloud-storage.
+- Updated the `README.md.j2` to reflect the optional creation of the new model_monitoring/ directory.
+- Changed the format for how `config/defaults.yaml` file gets written; this file is now written using the `write_yaml_file` function (yaml.safe_dump) in utils.py.
+- Updated the `account_permissions_warning` function in utils.py to include a new operation: `operation='model_monitoring'`
+- Updated the introductory example, readme, and user guide to reflect the new monitoring capabilities.
+- Renamed `validate_schedule` in utils.py to `validate_use_ci` to reflect new requirements for model monitoring.
+
+### Fixed
+
 ## [1.2.8] - 2024-1-09
 
 ### Added

CHANGELOG.md

@@ -7,7 +7,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Copyright 2023 Google LLC\n",
+    "# Copyright 2024 Google LLC\n",
     "#\n",
     "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
     "# you may not use this file except in compliance with the License.\n",
@@ -103,6 +103,7 @@
     "- [compute.googleapis.com](https://cloud.google.com/compute/docs/reference/rest/v1)\n",
     "- [iam.googleapis.com](https://cloud.google.com/iam/docs/reference/rest)\n",
     "- [iamcredentials.googleapis.com](https://cloud.google.com/iam/docs/reference/credentials/rest)\n",
+    "- [logging.googleapis.com](https://cloud.google.com/logging/docs/reference/v2/rest)\n",
     "- [pubsub.googleapis.com](https://cloud.google.com/pubsub/docs/reference/rest)\n",
     "- [run.googleapis.com](https://cloud.google.com/run/docs/reference/rest)\n",
     "- [storage.googleapis.com](https://cloud.google.com/storage/docs/apis)\n",
@@ -337,13 +338,14 @@
     "- `AutoMLOps.component(...)`: Defines a component, which is a containerized python function.\n",
     "- `AutoMLOps.pipeline(...)`: Defines a pipeline, which is a series of components.\n",
     "\n",
-    "AutoMLOps provides 5 functions for building and maintaining MLOps pipelines:\n",
+    "AutoMLOps provides 6 functions for building and maintaining MLOps pipelines:\n",
     "\n",
     "- `AutoMLOps.generate(...)`: Generates the MLOps codebase. Users can specify the tooling and technologies they would like to use in their MLOps pipeline.\n",
     "- `AutoMLOps.provision(...)`: Runs provisioning scripts to create and maintain necessary infra for MLOps.\n",
     "- `AutoMLOps.deprovision(...)`: Runs deprovisioning scripts to tear down MLOps infra created using AutoMLOps.\n",
     "- `AutoMLOps.deploy(...)`: Builds and pushes component container, then triggers the pipeline job.\n",
-    "- `AutoMLOps.launchAll(...)`: Runs `generate()`, `provision()`, and `deploy()` all in succession. \n",
+    "- `AutoMLOps.launchAll(...)`: Runs `generate()`, `provision()`, and `deploy()` all in succession.\n",
+    "- `AutoMLOps.monitor(...)`: Creates model monitoring jobs on deployed endpoints.\n",
     "\n",
     "Please see the [readme](https://github.com/GoogleCloudPlatform/automlops/blob/main/README.md) for more information."
    ]
@@ -521,9 +523,7 @@
    "outputs": [],
    "source": [
     "@AutoMLOps.pipeline #(name='automlops-pipeline', description='This is an optional description')\n",
-    "def pipeline(bq_table: str,\n",
-    "             data_path: str,\n",
-    "             project_id: str,\n",
+    "def pipeline(project_id: str,\n",
     "             bigquery_destination: str,\n",
     "             bq_dataset_path: str,\n",
     "             instances_format: str,\n",
@@ -568,8 +568,6 @@
    "outputs": [],
    "source": [
     "pipeline_params = {\n",
-    "    'bq_table': f'{PROJECT_ID}.test_dataset.dry-beans-inferencing',\n",
-    "    'data_path': f'gs://{PROJECT_ID}-bucket/data.csv',\n",
     "    'project_id': PROJECT_ID,\n",
     "    'bigquery_destination': f'bq://{PROJECT_ID}.test_dataset.dry-beans-inferencing-results',\n",
     "    'bq_dataset_path': f'bq://{PROJECT_ID}.test_dataset.dry-beans-inferencing',\n",