You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/user_guide/operators/anomaly_detection_operator/advanced_use_cases.rst
+50-63Lines changed: 50 additions & 63 deletions
Original file line number
Diff line number
Diff line change
@@ -4,99 +4,86 @@ Advanced Use Cases
4
4
5
5
**Documentation: Anomaly Detection Science and Model Parameterization**
6
6
7
-
**The Science of Anomaly Detection**
7
+
The Science of Anomaly Detection
8
+
--------------------------------
8
9
9
-
Forecasting is a complex yet essential discipline that involves predicting future values or events based on historical data and various mathematical and statistical techniques. To achieve accurate forecasts, it is crucial to understand some fundamental concepts:
10
+
Anomaly Detection comes in many forms. We will go through some of these and give guidance as to whether this Operator is going to be helpful for each use case.
10
11
11
-
**Seasonality**
12
+
* Constructive v Destructive v Pre-Processing: This Operator focuses on the Constructive and Pre-Processing use cases. Destructive can work, but more specific parameters may be required.
13
+
* Supervised v Semi-Supervised v Unsupervised: All 3 of these approaches are supported by AutoMLX. AutoTS supports only Unsupervised at this time.
14
+
* Time Series. This Operator is focused on just time-series data.
12
15
13
-
Seasonality refers to patterns in data that repeat at regular intervals, typically within a year. For example, retail sales often exhibit seasonality with spikes during holidays or specific seasons. Seasonal components can be daily, weekly, monthly, or yearly, and understanding them is vital for capturing and predicting such patterns accurately.
14
16
15
-
**Stationarity**
17
+
Data Parameterization
18
+
---------------------
16
19
17
-
Stationarity is a critical property of time series data. A time series is considered stationary when its statistical properties, such as mean, variance, and autocorrelation, remain constant over time. Stationary data simplifies forecasting since it allows models to assume that future patterns will resemble past patterns.
20
+
**Read Data from the Database**
18
21
19
-
**Cold Start**
22
+
.. code-block:: yaml
23
+
24
+
kind: operator
25
+
type: anomaly
26
+
version: v1
27
+
spec:
28
+
input_data:
29
+
connect_args:
30
+
user: XXX
31
+
password: YYY
32
+
dsn: "localhost/orclpdb"
33
+
sql: 'SELECT Store_ID, Sales, Date FROM live_data'
34
+
datetime_column:
35
+
name: ds
36
+
target_column: y
20
37
21
-
The "cold start" problem arises when you have limited historical data for a new product, service, or entity. Traditional forecasting models may struggle to make accurate predictions in these cases due to insufficient historical context.
22
38
23
-
**Passing Parameters to Models**
39
+
**Read Part of a Dataset**
24
40
25
-
To enhance the accuracy and adaptability of forecasting models, our system allows you to pass parameters directly. Here's how to do it:
41
+
42
+
.. code-block:: yaml
43
+
44
+
kind: operator
45
+
type: anomaly
46
+
version: v1
47
+
spec:
48
+
input_data:
49
+
url: oci://bucket@namespace/data
50
+
format: hdf
51
+
limit: 1000# Only the first 1000 rows
52
+
columns: ["y", "ds"] # Ignore other columns
53
+
datetime_column:
54
+
name: ds
55
+
target_column: y
26
56
27
57
28
58
**Specify Model Type**
29
59
30
-
Sometimes users will know which models they want to use. When users know this in advance, they can specify using the ``model_kwargs`` dictionary. In the following example, we will instruct the model to *only* use the ``DecisionTreeRegressor`` model.
60
+
Sometimes users will know which models they want to use. When users know this in advance, they can specify using the ``model_kwargs`` dictionary. In the following example, we will instruct the model to *only* use the ``IsolationForestOD`` model.
31
61
32
62
.. code-block:: yaml
33
63
34
64
kind: operator
35
-
type: forecast
65
+
type: anomaly
36
66
version: v1
37
67
spec:
38
68
model: automlx
39
69
model_kwargs:
40
70
model_list:
41
-
- NaiveForecaster
71
+
- IsolationForestOD
42
72
search_space:
43
-
NaiveForecaster:
44
-
sp: [1,100]
73
+
IsolationForestOD:
74
+
n_estimators:
75
+
range': [10, 50]
76
+
type': 'discrete'
45
77
46
78
47
-
When using autots, there are model_list *families*. These families are named after the shared characteristics of the models included. For example, we can use the autots "superfast" model_list and set it in the following way:
79
+
AutoTS offers the same extensibility:
48
80
49
81
.. code-block:: yaml
50
82
51
83
kind: operator
52
-
type: forecast
84
+
type: anomaly
53
85
version: v1
54
86
spec:
55
87
model: autots
56
88
model_kwargs:
57
-
model_list: superfast
58
-
59
-
60
-
Note: this is only supported for the ``autots`` model.
61
-
62
-
63
-
**Specify Other Model Details**
64
-
65
-
In addition to ``model_list``, there are many other parameters that can be specified. Users may specify, for example, the search space they want to search for their given model type. In automlx, specifying a hyperparameter range is as simple as:
66
-
67
-
.. code-block:: yaml
68
-
69
-
kind: operator
70
-
type: forecast
71
-
version: v1
72
-
spec:
73
-
model: automlx
74
-
model_kwargs:
75
-
search_space:
76
-
LogisticRegression:
77
-
C:
78
-
range: [0.03125, 512]
79
-
type': continuous
80
-
solver:
81
-
range: ['newton-cg', 'lbfgs', 'liblinear', 'sag']
82
-
type': categorical
83
-
class_weight:
84
-
range: [None, 'balanced']
85
-
type: categorical
86
-
87
-
88
-
**When Models Perform Poorly and the "Auto" Method**
89
-
90
-
Forecasting models are not one-size-fits-all, and some models may perform poorly under certain conditions. Common scenarios where models might struggle include:
91
-
92
-
- **Sparse Data:** When there's limited historical data available, traditional models may have difficulty making accurate predictions, especially for cold start problems.
93
-
94
-
- **High Seasonality:** Extremely seasonal data with complex patterns can challenge traditional models, as they might not capture all nuances.
95
-
96
-
- **Non-Linear Relationships:** In cases where the relationships between input variables and forecasts are nonlinear, linear models may underperform.
97
-
98
-
- **Changing Dynamics:** If the underlying data-generating process changes over time, static models may fail to adapt.
99
-
100
-
Our system offers an "auto" method that strives to anticipate and address these challenges. It dynamically selects the most suitable forecasting model and parameterizes it based on the characteristics of your data. It can automatically detect seasonality, stationarity, and cold start issues, then choose the best-fitting model and adjust its parameters accordingly.
101
-
102
-
By using the "auto" method, you can rely on the system's intelligence to adapt to your data's unique characteristics and make more accurate forecasts, even in challenging scenarios. This approach simplifies the forecasting process and often leads to better results than manual model selection and parameter tuning.
0 commit comments