diff --git a/docs/assets/images/concepts/mlops/data-transformations-hopsworks.jpg b/docs/assets/images/concepts/mlops/data-transformations-hopsworks.jpg new file mode 100644 index 000000000..0b83b5525 Binary files /dev/null and b/docs/assets/images/concepts/mlops/data-transformations-hopsworks.jpg differ diff --git a/docs/assets/images/concepts/mlops/supported-feature-engineering-tools.jpg b/docs/assets/images/concepts/mlops/supported-feature-engineering-tools.jpg new file mode 100644 index 000000000..fadcd62a0 Binary files /dev/null and b/docs/assets/images/concepts/mlops/supported-feature-engineering-tools.jpg differ diff --git a/docs/assets/images/concepts/mlops/taxonomy-transformations.jpg b/docs/assets/images/concepts/mlops/taxonomy-transformations.jpg new file mode 100644 index 000000000..cff412cb5 Binary files /dev/null and b/docs/assets/images/concepts/mlops/taxonomy-transformations.jpg differ diff --git a/docs/assets/images/concepts/mlops/transformation-features.jpg b/docs/assets/images/concepts/mlops/transformation-features.jpg new file mode 100644 index 000000000..77688e43c Binary files /dev/null and b/docs/assets/images/concepts/mlops/transformation-features.jpg differ diff --git a/docs/assets/images/concepts/mlops/transformation-in-modular-AI-pipeline.jpg b/docs/assets/images/concepts/mlops/transformation-in-modular-AI-pipeline.jpg new file mode 100644 index 000000000..5d21a32ab Binary files /dev/null and b/docs/assets/images/concepts/mlops/transformation-in-modular-AI-pipeline.jpg differ diff --git a/docs/concepts/mlops/data_transformations.md b/docs/concepts/mlops/data_transformations.md new file mode 100644 index 000000000..170743a46 --- /dev/null +++ b/docs/concepts/mlops/data_transformations.md @@ -0,0 +1,49 @@ +# Data Transformations + +[Data transformations](https://www.hopsworks.ai/dictionary/data-transformation) are integral to all AI applications. Data transformations produce new features that can enhance the performance of an AI application. However, [not all transformations in an AI application are equivalent](https://www.hopsworks.ai/post/a-taxonomy-for-data-transformations-in-ai-systems). + +Transformations like binning and aggregations typically create reusable features, while transformations like one-hot encoding, scaling and normalization often produce model-specific features. Additionally, in real-time AI systems, some features can only be computed during inference when the request is received, as they need request-time parameters to be computed. + +![Types of features](../../assets/images/concepts/mlops/transformation-features.jpg) + +This classification of features can be used to create a taxonomy for data transformation that would apply to any scalable and modular AI system that aims to reuse features. The taxonomy helps identify which classes of data transformation can cause [online-offline](https://www.hopsworks.ai/dictionary/online-offline-feature-skew) skews in AI systems, allowing for their prevention. Hopsworks provides support for a feature view abstraction as well as model-dependent transformations and on-demand transformations to prevent online-offline skew. + +## Data Transformation Taxonomy for AI Systems + +Transformation functions in an AI system can be classified into three types based on the nature of the input features they generate: [model-independent](https://www.hopsworks.ai/dictionary/model-independent-transformations), [model-dependent](https://www.hopsworks.ai/dictionary/model-dependent-transformations), and [on-demand](https://www.hopsworks.ai/dictionary/on-demand-transformation) transformations. + +![Types of transformations](../../assets/images/concepts/mlops/taxonomy-transformations.jpg) + +**Model-independent transformations** create reusable features that can be utilized across one or more machine-learning models. These transformations include techniques such as grouped aggregations (e.g., minimum, maximum, or average of a variable), windowed aggregations (e.g., the number of clicks per day), and binning to generate categorical variables. Since the data produced by model-independent transformations are reusable, these features can be stored in a feature store. + +**Model-dependent transformations** generate features specific to one model. These include transformations that are unique to a particular model or are parameterized by the training dataset, making them model-specific. For instance, text tokenization is a transformation required by all large language models (LLMs) but each LLM has their own (unique) tokenizer. Other transformations, such as encoding categorical variables in a numerical representation or scaling/normalizing/standardizing numerical variables to enhance the performance of gradient-based models, are parameterized by the training dataset. Consequently, the features produced are applicable only to the model trained using that specific training dataset. Since these features are not reusable, there is no need to store them in a feature store. Also, storing encoded features in a feature store leads to write amplification, as every time feature values are written to a feature group, all existing rows in the feature group have to be re-encoded (and creation of a training dataset using a subset or rows in the feature group becomes impossible as they cannot be re-encoded). + +**On-demand transformations** are exclusive to [real-time AI systems](https://www.hopsworks.ai/dictionary/real-time-machine-learning), where predictions must be generated in real time based on incoming prediction requests. On-demand transformations compute on-demand features, which usually require at least one input parameter that is only available in a prediction request for their computation. These transformations can also combine request-time parameters with precomputed features from feature stores. Some examples include generating *zip_codes* from latitude and longitude received in the prediction request or calculating the *time_since_last_transaction* from a transaction request. The on-demand features produced can also be computed and [backfilled](https://www.hopsworks.ai/dictionary/backfill-features) into a feature store when the necessary historical data required for their computation becomes available. Backfilling on-demand features into the feature store eliminates the need to recompute them when creating training data. On-demand transformations are typically also model-independent transformations (model-dependent transformations can be applied after the on-demand transformation). + + + +Each of these transformations is employed within specific areas in a modular AI system and can be illustrated using the figure below. +![Types of transformations in modular AI Pipeline](../../assets/images/concepts/mlops/transformation-in-modular-AI-pipeline.jpg) + +Model-independent transformations are utilized exclusively in areas where new and historical data arrives, typically within feature pipelines. Model-dependent transformations are necessary during the creation of training data, in training programs and must also be consistently applied in inference programs prior to making predictions. On-demand transformations are primarily employed in online inference programs, though they can also be integrated into feature engineering programs to backfill data into the feature store. + +The presence of model-dependent and on-demand transformations across different modules in a modular AI system introduces the potential for online-offline skew. Hopsworks provides support for model-dependent transformations and on-demand transformations to easily create modular skew-free AI pipelines. + +## Hopsworks and the Data Transformation Taxonomy + +![Data transformations Hopsworks](../../assets/images/concepts/mlops/data-transformations-hopsworks.jpg) + +In Hopsworks, an AI system is typically decomposed into different [AI pipelines](https://www.hopsworks.ai/dictionary/ai-pipelines) and usually falls into either a [feature pipeline](https://www.hopsworks.ai/dictionary/feature-pipeline), a [training pipeline](https://www.hopsworks.ai/dictionary/training-pipeline), or an [inference pipeline](https://www.hopsworks.ai/dictionary/inference-pipeline). + +Hopsworks stores reusable feature data, created by model-independent transformations within the feature pipeline, into [feature groups](../fs/feature_group/fg_overview.md) (tables containing feature data in both offline and online stores). Model-independent transformations in Hopsworks can be performed using a wide range of commonly used data engineering tools and the generated features can be seamlessly inserted into feature groups. The figure below illustrates the different software tools supported by Hopsworks for creating reusable features through model-independent transformations. + + +![Supported feature engineering tools](../../assets/images/concepts/mlops/supported-feature-engineering-tools.jpg) + +Additionally, Hopsworks provides a simple Python API to [create custom transformation functions](../../user_guides/fs/transformation_functions.md) as either Python or Pandas User-Defined Functions (UDFs). Pandas UDFs enable the vectorized execution of transformation functions, offering significantly higher throughput compared to Python UDFs for large volumes of data. They can also be scaled out across workers in a Spark program, allowing for scalability from gigabytes (GBs) to terabytes (TBs) or more. However, Python UDFs can be much faster for small volumes of data, such as in the case of online inference. + +Transformation functions defined in Hopsworks can then be attached to feature groups to [create on-demand transformation](../../user_guides/fs/feature_group/on_demand_transformations.md). On-demand transformations in feature groups are executed automatically whenever data is inserted into them to compute and backfill the on-demand features into the feature group. Backfilling on-demand features removes the need to recompute them while creating training and batch data. + +Hopsworks also provides a powerful abstraction known as [feature views](../fs/feature_view/fv_overview.md), which enables feature reuse and prevents skew between training and inference pipelines. A feature view is a meta-data-only selection of features, created from potentially different feature groups. It includes the input and output schema required for a model. This means that a feature view describes not only the input features but also the output targets, along with any helper columns necessary for training or inference of the model. This allows feature views to create consistent snapshots of data for both training and inference of a model. Additionally feature views, also compute and save statistics for the training datasets they create. + +Hopsworks supports attaching transformations functions to feature views to [create model-dependent transformations](../../user_guides/fs/feature_view/model-dependent-transformations.md) that have no online-offline skew. These transformations get access to the same training dataset statistics during both training and inference ensuring their consistency. Additionally, feature views through lineage get access to the on-demand transformation used to create on-demand features if any are selected during the creation of the feature view. This allows for the computation of on-demand features in real-time during online-inference. \ No newline at end of file diff --git a/docs/user_guides/fs/feature_group/on_demand_transformations.md b/docs/user_guides/fs/feature_group/on_demand_transformations.md new file mode 100644 index 000000000..6f4c671b2 --- /dev/null +++ b/docs/user_guides/fs/feature_group/on_demand_transformations.md @@ -0,0 +1,184 @@ +# On-Demand Transformation Functions + +[On-demand transformations](https://www.hopsworks.ai/dictionary/on-demand-transformation) produce on-demand features, which usually require parameters accessible during inference for their calculation. Hopsworks facilitates the creation of on-demand transformations without introducing [online-offline skew](https://www.hopsworks.ai/dictionary/online-offline-feature-skew), ensuring consistency while allowing their dynamic computation during online inference. + +## On Demand Transformation Function Creation + + +An on-demand transformation function can be created by attaching a [transformation function](../transformation_functions.md) to a feature group. Each on-demand transformation function creates one on-demand feature having the same name as the transformation function. For instance, in the example below, the on-demand transformation function `transaction_age` will generate one on-demand feature called `transaction_age`. Hence, only one-to-one or many-to-one transformation functions can be used to create an on-demand transformation functions. + +!!! warning "On-demand transformation" + All on-demand transformation functions attached to a feature group must have unique names and, in contrast to model-dependent transformations, they do not have access to training dataset statistics. + +Each on-demand transformation function can map specific features to its arguments by explicitly providing their names as arguments to the transformation function. If no feature names are provided, the transformation function will default to using features that match the name of the transformation function's argument. + + + +=== "Python" +!!! example "Creating on-demand transformation functions." + ```python + # Define transformation function + @hopsworks.udf(return_type=int, drop=["current_date"]) + def transaction_age(transaction_date, current_date): + return (current_date - transaction_date).dt.days + + # Attach transformation function to feature group to create on-demand transformation function. + fg = feature_store.create_feature_group(name="fg_transactions", + version=1, + description="Transaction Features", + online_enabled=True, + primary_key=['id'], + event_time='event_time' + transformation_functions=[transaction_age] + ) + ``` + + +### Specifying input features + +The features to be used by the on-demand transformation function can be specified by providing the feature names as input to the transformation functions. + +=== "Python" +!!! example "Creating on-demand transformations by specifying features to be passed to transformation function." + ```python + fg = feature_store.create_feature_group(name="fg_transactions", + version=1, + description="Transaction Features", + online_enabled=True, + primary_key=['id'], + event_time='event_time' + transformation_functions=[age_transaction('transaction_time', 'current_time')] + ) + ``` + +## Usage + +On-demand transformation functions attached to a feature group are automatically executed in the feature pipeline when you [insert data](../create/#batch-write-api) into a feature group and [by the Python client while retrieving feature vectors](../feature_view/feature-vectors.md#retrieval) for online inference using feature views that contain on-demand features. + +The on-demand features computed by on-demand transformation functions are positioned after all other features in a feature group and are ordered alphabetically by their names. + +### Inserting data + +All on-demand transformation functions attached to a feature group are executed whenever new data is inserted. This process computes on-demand features from historical data. The DataFrame used for insertion must include all features required for executing all on-demand transformation functions in the feature group. + +Inserting on-demand features as historical features saves time and computational resources by removing the need to compute all on-demand features while generating training or batch data. + +### Accessing on-demand features in feature views + +A feature view can include on-demand features from feature groups by selecting them in the [query](../feature_view/query.md) used to create the feature view. These on-demand features are equivalent to regular features, and [model-dependent transformations](../feature_view/model-dependent-transformations.md) can be applied to them if required. + +=== "Python" +!!! example "Creating feature view with on-demand features" + ```python + + # Selecting on-demand features in query + query = fg.select(["id", "feature1", "feature2", "on_demand_feature3", "on_demand_feature4"]) + + # Creating a feature view using a query that contains on-demand transformations and model-dependent transformations + feature_view = fs.create_feature_view( + name='transactions_view', + query=query, + transformation_functions=[ + min_max_scaler("feature1"), + min_max_scaler("on_demand_feature3"), + ] + ) + ``` + +### Computing on-demand features + +On-demand features in the feature view are computed in real-time during online inference using the same on-demand transformation functions used to create them. Hopsworks, by default, automatically computes all on-demand features when retrieving feature view input features (feature vectors) with the functions `get_feature_vector` and `get_feature_vectors`. Additionally, on-demand features can be computed using the `compute_on_demand_features` function or by manually executing the same on-demand transformation function. + +The values for the input parameters required to compute on-demand features can be provided using the `request_parameters` argument. If values are not provided through the `request_parameters` argument, the transformation function will verify if the feature vector contains the necessary input parameters and will use those values instead. However, if the required input parameters are also not present in the feature vector, an error will be thrown. + +!!! note + By default the functions `get_feature_vector` and `get_feature_vectors` will apply model-dependent transformation present in the feature view after computing on-demand features. + +#### Retrieving a feature vector + +The `get_feature_vector` function retrieves a single feature vector based on the feature view's serving key(s). The on-demand features in the feature vector can be computed using real-time data by passing a dictionary that associates the name of each input parameter needed for the on-demand transformation function with its respective new value to the `request_parameter` argument. + +=== "Python" +!!! example "Computing on-demand features while retrieving a feature vector" + ```python + + feature_vector = feature_view.get_feature_vector(entry={"id":1}, request_parameter={"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}) + ``` + +#### Retrieving feature vectors + +The `get_feature_vectors` function retrieves multiple feature vectors using a list of feature view serving keys. The `request_parameter` in this case, can be a list of dictionaries that specifies the input parameters for the computation of on-demand features for each serving key or can be a dictionary if the on-demand transformations require the same parameters for all serving keys. + +=== "Python" +!!! example "Computing on-demand features while retrieving a feature vectors" + ```python + + # Specify unique request parameters for each serving key. + feature_vector = feature_view.get_feature_vectors(entry=[{"id":1}, {"id":2}], request_parameter=[{"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}, + {"transaction_time":datetime(2022, 11, 20, 12, 50, 00), "current_time":datetime.now()}]) + + # Specify common request parameters for all serving key. + feature_vector = feature_view.get_feature_vectors(entry=[{"id":1}, {"id":2}], request_parameter={"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}) + ``` + +The `get_feature_vector` and `get_feature_vectors` can also return untransformed features by setting the parameter `transform` to `False`. + +=== "Python" +!!! example "Returning untransformed feature vectors" + ```python + + untransformed_feature_vector = feature_view.get_feature_vector(entry={"id":1}, transform=False) + + untransformed_feature_vectors = feature_view.get_feature_vectors(entry=[{"id":1}, {"id":2}], transform=False) + ``` + +#### Compute all on-demand features + +The `compute_on_demand_features` function computes all on-demand features attached to a feature view and adds them to the feature vectors provided as input to the function. This function does not apply model-dependent transformations to any of the features. The `transform` function can be used to apply model-dependent transformations to the returned values if required. + +The `request_parameter` in this case, can be a list of dictionaries that specifies the input parameters for the computation of on-demand features for each feature vector given as input to the function or can be a dictionary if the on-demand transformations require the same parameters for all input feature vectors. + +=== "Python" +!!! example "Computing all on-demand features and manually applying model dependent transformations." + ```python + + # Specify request parameters for each serving key. + untransformed_feature_vector = feature_view.get_feature_vector(entry={"id":1}, transform=False) + + # re-compute and add on-demand features to the feature vector + feature_vector_with_on_demand_features = fv.compute_on_demand_features(untransformed_feature_vector, + request_parameter={"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}) + + # Applying model dependent transformations + encoded_feature_vector = fv.transform(feature_vector_with_on_demand_features) + + # Specify request parameters for each serving key. + untransformed_feature_vectors = feature_view.get_feature_vectors(entry=[{"id":1}, {"id":2}], transform=False) + + # re-compute and add on-demand features to the feature vectors - Specify unique request parameter for each feature vector + feature_vectors_with_on_demand_features = fv.compute_on_demand_features(untransformed_feature_vectors, + request_parameter=[{"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}, + {"transaction_time":datetime(2022, 11, 20, 12, 50, 00), "current_time":datetime.now()}]) + + # re-compute and add on-demand feature to the feature vectors - Specify common request parameter for all feature vectors + feature_vectors_with_on_demand_features = fv.compute_on_demand_features(untransformed_feature_vectors, request_parameter={"transaction_time":datetime(2022, 12, 28, 23, 55, 59), "current_time":datetime.now()}) + + # Applying model dependent transformations + encoded_feature_vector = fv.transform(feature_vectors_with_on_demand_features) + + ``` + +#### Compute one on-demand feature + +On-demand transformation functions can also be accessed and executed as normal functions by using the dictionary `on_demand_transformations` that maps the on-demand features to their corresponding on-demand transformation function. + +=== "Python" +!!! example "Executing each on-demand transformation function" + ```python + + # Specify request parameters for each serving key. + feature_vector = feature_view.get_feature_vector(entry={"id":1}, transform=False, return_type="pandas") + + # Applying model dependent transformations + feature_vector["on_demand_feature1"] = fv.on_demand_transformations["on_demand_feature1"](feature_vector["transaction_time"], datetime.now()) + ``` \ No newline at end of file diff --git a/docs/user_guides/fs/feature_view/index.md b/docs/user_guides/fs/feature_view/index.md index 46d231483..e8830e210 100644 --- a/docs/user_guides/fs/feature_view/index.md +++ b/docs/user_guides/fs/feature_view/index.md @@ -9,6 +9,6 @@ This section serves to provide guides and examples for the common usage of abstr - [Feature Server](feature-server.md) - [Query](query.md) - [Helper columns](helper-columns.md) -- [Model-Dependent Transformation Functions](transformation-function.md) +- [Model-Dependent Transformation Functions](model-dependent-transformations.md) - [Spines](spine-query.md) - [Feature Monitoring](feature_monitoring.md) \ No newline at end of file diff --git a/docs/user_guides/fs/feature_view/model-dependent-transformations.md b/docs/user_guides/fs/feature_view/model-dependent-transformations.md new file mode 100644 index 000000000..394caa719 --- /dev/null +++ b/docs/user_guides/fs/feature_view/model-dependent-transformations.md @@ -0,0 +1,131 @@ +# Model Dependent Transformation Functions + + +[Model-dependent transformations](https://www.hopsworks.ai/dictionary/model-dependent-transformations) transform feature data for a specific model. Feature encoding is one example of such a transformations. Feature encoding is parameterized by statistics from the training dataset, and, as such, many model-dependent transformations require the training dataset statistics as a parameter. Hopsworks enhances the robustness of AI pipelines by preventing [training-inference skew](https://www.hopsworks.ai/dictionary/training-inference-skew) by ensuring that the same model-dependent transformations and statistical parameters are used during both training dataset generation and online inference. + +Additionally, Hopsworks offers built-in model-dependent transformation functions, such as `min_max_scaler`, `standard_scaler`, `robust_scaler`, `label_encoder`, and `one_hot_encoder`, which can be easily imported and declaratively applied to features in a feature view. + +## Model Dependent Transformation Function Creation + +Hopsworks allows you to create a model-dependent transformation function by attaching a [transformation function](../transformation_functions.md) to a feature view. The attached transformation function can be a simple function that takes one feature as input and outputs the transformed feature data. For example, in the case of min-max scaling a numerical feature, you will have a number as input parameter to the transformation function and a number as output. However, in the case of one-hot encoding a categorical variable, you will have a string as input and an array of 1s and 0s and output. You can also have transformation functions that take multiple features as input and produce one or more values as output. That is, transformation functions can be one-to-one, one-to-many, many-to-one, or many-to-many. + +Each model-dependent transformation function can map specific features to its arguments by explicitly providing their names as arguments to the transformation function. If no feature names are provided, the transformation function will default to using features from the feature view that match the name of the transformation function's argument. + +The output columns generated by a model-dependent transformation function follows a naming convention structured as `functionName_features_outputColumnNumber` if the transformation function outputs multiple columns and `functionName_features` if the transformation function outputs one column. For instance, for the function named `add_one_multiple` that outputs multiple columns in the example given below, produces output columns that would be labeled as  `add_one_multiple_feature1_feature2_feature3_0`,  `add_one_multiple_feature1_feature2_feature3_1` and  `add_one_multiple_feature1_feature2_feature3_2`. The function named `add_two` that outputs a single column in the example given below, produces a single output column names as `add_two_feature`. + + +=== "Python" + + !!! example "Creating model-dependent transformation functions" + ```python + # Defining a many to many transformation function. + @udf(return_type=[int, int, int], drop=["feature1", "feature3"]) + def add_one_multiple(feature1, feature2, feature3): + return pd.DataFrame({"add_one_feature1":feature1 + 1, "add_one_feature2":feature2 + 1, "add_one_feature3":feature3 + 1}) + + # Defining a one to one transformation function. + @udf(return_type=int) + def add_two(feature): + return feature + 2 + + # Creating model-dependent transformations by attaching transformation functions to feature views. + feature_view = fs.create_feature_view( + name='transactions_view', + query=query, + labels=["fraud_label"], + transformation_functions=[ + add_two, + add_one_multiple + ] + ) + ``` + +### Specifying input features + +The features to be used by a model-dependent transformation function can be specified by providing the feature names (from the feature view / feature group) as input to the transformation functions. + + +=== "Python" + + !!! example "Specifying input features to be passed to a model-dependent transformation function" + ```python + feature_view = fs.create_feature_view( + name='transactions_view', + query=query, + labels=["fraud_label"], + transformation_functions=[ + add_two("feature_1"), + add_two("feature_2"), + add_one_multiple("feature_5", "feature_6", "feature_7") + ] + ) + ``` + +### Using built-in transformations + +Built-in transformation functions are attached in the same way. The only difference is that they can either be retrieved from the Hopsworks or imported from the `hopsworks` module. + +=== "Python" + + !!! example "Creating model-dependent transformation using built-in transformation functions retrieved from Hopsworks" + ```python + min_max_scaler = fs.get_transformation_function(name="min_max_scaler") + standard_scaler = fs.get_transformation_function(name="standard_scaler") + robust_scaler = fs.get_transformation_function(name="robust_scaler") + label_encoder = fs.get_transformation_function(name="label_encoder") + + feature_view = fs.create_feature_view( + name='transactions_view', + query=query, + labels=["fraud_label"], + transformation_functions = [ + label_encoder("category"), + robust_scaler("amount"), + min_max_scaler("loc_delta"), + standard_scaler("age_at_transaction") + ] + ) + ``` + +To attach built-in transformation functions from the `hopsworks` module they can be directly imported into the code from `hopsworks.builtin_transformations`. + +=== "Python" + + !!! example "Creating model-dependent transformation using built-in transformation functions imported from hopsworks" + ```python + from hopsworks.builtin_transformations import min_max_scaler, label_encoder, robust_scaler, standard_scaler + + feature_view = fs.create_feature_view( + name='transactions_view', + query=query, + labels=["fraud_label"], + transformation_functions = [ + label_encoder("category": ), + robust_scaler("amount"), + min_max_scaler("loc_delta"), + standard_scaler("age_at_transaction") + ] + ) + ``` + + +## Using Model Dependent Transformations + +Model-dependent transformations attached to a feature view are automatically applied when you [create training data](./training-data.md#creation), [read training data](./training-data.md#read-training-data), [read batch inference data](./batch-data.md#creation-with-transformation), or [get feature vectors](./feature-vectors.md#retrieval-with-transformation). The generated data includes untransformed features, on-demand features, if any, and the transformed features. The transformed features are organized by their output column names in alphabetical order and are positioned after the untransformed and on-demand features. + +Model-dependent transformation functions can also be manually applied to a feature vector using the `transform` function. + +=== "Python" + + !!! example "Manually applying model-dependent transformations during online inference" + ```python + # Initialize the feature view with the correct training dataset version used for model-dependent transformations + fv.init_serving(training_dataset_version) + + # Get untransformed feature Vector + feature_vector = fv.get_feature_vector(entry={"index":10}, transformed=False, return_type="pandas") + + # Apply Model Dependent transformations + encode_feature_vector = fv.transform(feature_vector) + ``` + diff --git a/docs/user_guides/fs/feature_view/overview.md b/docs/user_guides/fs/feature_view/overview.md index e003e2214..e5ab9e3ce 100644 --- a/docs/user_guides/fs/feature_view/overview.md +++ b/docs/user_guides/fs/feature_view/overview.md @@ -4,8 +4,8 @@ A feature view is a set of features that come from one or more feature groups. I If you want to understand more about the concept of feature view, you can refer to [here](../../../concepts/fs/feature_view/fv_overview.md). -## Creation -[Query](./query.md) and [transformation function](./transformation-function.md) are the building blocks of a feature view. You can define your set of features by building a `query`. You can also define which columns in your feature view are the `labels`, which is useful for supervised machine learning tasks. Furthermore, in python client, each feature can be attached to its own transformation function. This way, when a feature is read (for training or scoring), the transformation is executed on-demand - just before the feature data is returned. For example, when a client reads a numerical feature, the feature value could be normalized by a StandardScalar transformation function before it is returned to the client. +## Feature View Creation +[Query](./query.md) and [transformation function](./model-dependent-transformations.md) are the building blocks of a feature view. You can define your set of features by building a `query`. You can also define which columns in your feature view are the `labels`, which is useful for supervised machine learning tasks. Furthermore, in python client, each feature can be attached to its own transformation function. This way, when a feature is read (for training or scoring), the transformation is executed on-demand - just before the feature data is returned. For example, when a client reads a numerical feature, the feature value could be normalized by a StandardScalar transformation function before it is returned to the client. === "Python" @@ -44,7 +44,7 @@ If you want to understand more about the concept of feature view, you can refer .build(); ``` -You can refer to [query](./query.md) and [transformation function](./transformation-function.md) for creating `query` and `transformation_function`. To see a full example of how to create a feature view, you can read [this notebook](https://github.com/logicalclocks/hopsworks-tutorials/blob/master/fraud_batch/2_feature_view_creation.ipynb). +You can refer to [query](./query.md) and [transformation function](./model-dependent-transformations.md) for creating `query` and `transformation_function`. To see a full example of how to create a feature view, you can read [this notebook](https://github.com/logicalclocks/hopsworks-tutorials/blob/master/fraud_batch/2_feature_view_creation.ipynb). ## Retrieval Once you have created a feature view, you can retrieve it by its name and version. diff --git a/docs/user_guides/fs/feature_view/transformation-function.md b/docs/user_guides/fs/feature_view/transformation-function.md deleted file mode 100644 index ea97dcba4..000000000 --- a/docs/user_guides/fs/feature_view/transformation-function.md +++ /dev/null @@ -1,220 +0,0 @@ -# Model Dependent Transformation Functions - -Hopsworks provides functionality to attach transformation functions to [feature views](./overview.md). - -These transformation functions are primarily [model-dependent transformations](https://www.hopsworks.ai/dictionary/model-dependent-transformations). Model-dependent transformations generate feature data tailored to a specific model, often requiring the computation of training dataset statistics. Hopsworks enables you to define custom model-dependent transformation functions that can take multiple features and their associated statistics as input and produce multiple transformed features as output. Hopsworks also automatically executes the defined transformation function as a [`@pandas_udf`](https://spark.apache.org/docs/3.1.2/api/python/reference/api/pyspark.sql.functions.pandas_udf.html) in a PySpark application and as Pandas functions in Python clients. - -Custom transformation functions created in Hopsworks can be directly attached to feature views or stored in the feature store for later retrieval and attachment. These custom functions can be part of a library [installed](../../../user_guides/projects/python/python_install.md) in Hopsworks or added when starting a [Jupyter notebook](../../../user_guides/projects/jupyter/python_notebook.md) or [Hopsworks job](../../../user_guides/projects/jobs/spark_job.md). - -Hopsworks also includes built-in transformation functions such as `min_max_scaler`, `standard_scaler`, `robust_scaler`, `label_encoder`, and `one_hot_encoder` that can be easily imported and used. - -!!! warning "Pyspark decorators" - - Don't decorate transformation functions with Pyspark `@udf` or `@pandas_udf`, and also make sure not to use any Pyspark dependencies. That is because, the transformation functions may be executed by Python clients. Hopsworks will automatically run transformations as pandas udfs for you only if it is used inside Pyspark application. - -!!! warning "Java/Scala support" - - Creating and attaching Transformation functions to feature views are not supported for HSFS Java or Scala client. If feature view with transformation function was created using python client, you cannot get training data or get feature vectors from HSFS Java or Scala client. - - -## Creation of Custom Transformation Functions - -User-defined, custom transformation functions can be created in Hopsworks using the [`@udf`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/udf/) decorator. These functions should be designed as Pandas functions, meaning they must take input features as a [Pandas Series](https://pandas.pydata.org/docs/reference/api/pandas.Series.html) and return either a Pandas Series or a [Pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html). - -The `@udf` decorator in Hopsworks creates a metadata class called `HopsworksUdf`. This class manages the necessary operations to supply feature statistics to custom transformation functions and execute them as `@pandas_udf` in PySpark applications or as pure Pandas functions in Python clients. The decorator requires the `return_type` of the transformation function, which indicates the type of features returned. This can be a single Python type if the transformation function returns a single transformed feature as a Pandas Series, or a list of Python types if it returns multiple transformed features as a Pandas DataFrame. The supported types include `str`, `int`, `float`, `bool`, `datetime.datetime`, `datetime.date`, and `datetime.time`. - -Hopsworks supports four types of transformation functions: - -1. One to One: Transforms one feature into one transformed feature. -2. One to Many: Transforms one feature into multiple transformed features. -3. Many to One: Transforms multiple features into one transformed feature. -4. Many to Many: Transforms multiple features into multiple transformed features. - -To create a One to One transformation function, the hopsworks `@udf` decorator must be provided with the return type as a Python type and the transformation function should take one argument as input and return a Pandas Series. - -=== "Python" - - !!! example "Creation of a Custom One to One Transformation Function in Hopsworks." - ```python - from hopsworks import udf - - @udf(int) - def add_one(feature): - return feature + 1 - ``` - -Creation of a Many to One transformation function is similar to that of One to One transformation function, the only difference being that the transformation function accepts multiple features as input. - -=== "Python" - !!! example "Creation of a Many to One Custom Transformation Function in Hopsworks." - ```python - from hopsworks import udf - - @udf(int) - def add_features(feature1, feature2, feature3): - return feature + feature2 + feature3 - ``` - -To create a One to Many transformation function, the hopsworks `@udf` decorator must be provided with the return type as a list of Python types and the transformation function should take one argument as input and return multiple features as a Pandas DataFrame. The return types provided to the decorator must match the types of each column in the returned Pandas DataFrame. - -=== "Python" - !!! example "Creation of a One to Many Custom Transformation Function in Hopsworks." - ```python - from hopsworks import udf - import pandas as pd - - @udf([int, int]) - def add_one_and_two(feature1): - return pd.DataFrame({"add_one":feature1 + 1, "add_two":feature1 + 2}) - ``` - -Creation of a Many to Many transformation function is similar to that of One to May transformation function, the only difference being that the transformation function accepts multiple features as input. - -=== "Python" - !!! example "Creation of a Many to Many Custom Transformation Function in Hopsworks." - ```python - from hopsworks import udf - import pandas as pd - - @udf([int, int, int]) - def add_one_multiple(feature1, feature2, feature2): - return pd.DataFrame({"add_one_feature1":feature1 + 1, "add_one_feature2":feature2 + 1, "add_one_feature3":feature3 + 1}) - ``` -To access statistics pertaining to an argument provided as input to the transformation function, it is necessary to define a keyword argument named `statistics` in the transformation function. This statistics argument should be provided with an instance of class `TransformationStatistics` as default value. The `TransformationStatistics` instance must be initialized with the names of the arguments for which statistical information is required. - -The `TransformationStatistics` instance contains separate objects with the same name as the arguments used to initialize it. These objects encapsulate statistics related to the argument as instances of the `FeatureTransformationStatistics` class. Upon instantiation, instances of `FeatureTransformationStatistics` are initialized with `None` values. These placeholders are subsequently populated with the required statistics when the training dataset is created. - -=== "Python" - !!! example "Creation of a Custom Transformation Function in Hopsworks that accesses Feature Statistics" - ```python - from hopsworks import udf - from hsfs.transformation_statistics import TransformationStatistics - - stats = TransformationStatistics("argument1", "argument2", "argument3") - - @udf(int) - def add_features(argument1, argument2, argument3, statistics=stats): - return argument + argument2 + argument3 + statistics.argument1.mean + statistics.argument2.mean + statistics.argument3.mean - ``` - -The output column generated by the transformation function follows a naming convention structured as `functionName_features_outputColumnNumber`. For instance, for the function named `add_one_multiple`, the output columns would be labeled as `add_one_multiple_feature1_feature2_feature3_0`, `add_one_multiple_feature1_feature2_feature3_1`, and `add_one_multiple_feature1_feature2_feature3_2`. - -## Apply transformation functions to features - -Transformation functions can be attached to a feature view as a list. Each transformation function can specify which features are to be use by explicitly providing their names as arguments. If no feature names are provided explicitly, the transformation function will default to using features from the feature view that matches the name of the transformation function's argument. Then the transformation functions are applied when you [read training data](./training-data.md#read-training-data), [read batch data](./batch-data.md#creation-with-transformation), or [get feature vectors](./feature-vectors.md#retrieval-with-transformation). The generated data includes both transformed and untransformed features in a DataFrame. The transformed features are organized by their output column names in alphabetical order and are positioned after the untransformed features. By default all features provided as input to a transformation function are dropped when training data, batch data or feature vectors as created. - -=== "Python" - - !!! example "Attaching transformation functions to the feature view" - ```python - feature_view = fs.create_feature_view( - name='transactions_view', - query=query, - labels=["fraud_label"], - transformation_functions=[ - add_one, - add_features, - add_one_and_two, - add_one_multiple - ] - ) - ``` - -To explicitly pass the features to a transformation function the feature name to be used can be passed as arguments to the transformation function. - - -=== "Python" - - !!! example "Attaching transformation functions to the feature view by explicitly specifying features to be passed to transformation function" - ```python - feature_view = fs.create_feature_view( - name='transactions_view', - query=query, - labels=["fraud_label"], - transformation_functions=[ - add_one("feature_1"), - add_one("feature_2"), - add_features("feature_1", "feature_2", "feature_3"), - add_one_and_two("feature_4"), - add_one_multiple("feature_5", "feature_6", "feature_7") - ] - ) - ``` - -Built-in transformation functions are attached in the same way. The only difference is that they can either be retrieved from the Hopsworks or imported from the hsfs module - -=== "Python" - - !!! example "Attaching built-in transformation functions to the feature view by retrieving from Hopsworks" - ```python - min_max_scaler = fs.get_transformation_function(name="min_max_scaler") - standard_scaler = fs.get_transformation_function(name="standard_scaler") - robust_scaler = fs.get_transformation_function(name="robust_scaler") - label_encoder = fs.get_transformation_function(name="label_encoder") - - feature_view = fs.create_feature_view( - name='transactions_view', - query=query, - labels=["fraud_label"], - transformation_functions = [ - label_encoder("category"), - robust_scaler("amount"), - min_max_scaler("loc_delta"), - standard_scaler("age_at_transaction") - ] - ) - ``` - -To attach built in transformation functions from the hsfs module they can be directly imported into the code from `hsfs.builtin_transformations`. - -=== "Python" - - !!! example "Attaching built-in transformation functions to the feature view by importing from hsfs" - ```python - from hsfs.builtin_transformations import min_max_scaler, label_encoder, robust_scaler, standard_scaler - - feature_view = fs.create_feature_view( - name='transactions_view', - query=query, - labels=["fraud_label"], - transformation_functions = [ - label_encoder("category": ), - robust_scaler("amount"), - min_max_scaler("loc_delta"), - standard_scaler("age_at_transaction") - ] - ) - ``` - -## Saving Transformation Functions to Feature Store -To save a transformation function to the feature store, use the `create_transformation_function` which would create a `TransformationFunction` object. The `TransformationFunction` object can then be saved by calling the save function. - -=== "Python" - - !!! example "Register transformation function `add_one` in the Hopsworks feature store." - ```python - plus_one_meta = fs.create_transformation_function( - transformation_function=add_one, - version=1) - plus_one_meta.save() - ``` - -## Retrieval from Feature Store -To retrieve all transformation functions from the feature store, use `get_transformation_functions` which will return the list of available `TransformationFunction` objects. A specific transformation function can be retrieved with the `get_transformation_function` method where you can provide its name and version of the transformation function. If only the function name is provided then it will default to version 1. - -=== "Python" - - !!! example "Retrieving transformation functions from the feature store" - ```python - # get all transformation functions - fs.get_transformation_functions() - - # get transformation function by name. This will default to version 1 - plus_one_fn = fs.get_transformation_function(name="plus_one") - - # get built-in transformation function min max scaler - min_max_scaler_fn = fs.get_transformation_function(name="min_max_scaler") - - # get transformation function by name and version. - plus_one_fn = fs.get_transformation_function(name="plus_one", version=2) - ``` \ No newline at end of file diff --git a/docs/user_guides/fs/index.md b/docs/user_guides/fs/index.md index a60aecacf..9e64e72c5 100644 --- a/docs/user_guides/fs/index.md +++ b/docs/user_guides/fs/index.md @@ -7,4 +7,5 @@ This section serves to provide guides and examples for the common usage of abstr - [Feature Views](feature_view/index.md) - [Vector Similarity Search](vector_similarity_search.md) - [Compute Engines](compute_engines.md) -- [Integrations](../integrations/index.md) \ No newline at end of file +- [Integrations](../integrations/index.md) +- [Transformations](transformation_functions.md) \ No newline at end of file diff --git a/docs/user_guides/fs/transformation_functions.md b/docs/user_guides/fs/transformation_functions.md new file mode 100644 index 000000000..4ea2001b2 --- /dev/null +++ b/docs/user_guides/fs/transformation_functions.md @@ -0,0 +1,178 @@ + +# Transformation Functions + +In AI systems, [transformation functions](https://www.hopsworks.ai/dictionary/transformation) transform data to create features, the inputs to machine learning models (in both training and inference). The [taxonomy of data transformations](../../concepts/mlops/data_transformations.md) introduces three types of data transformation prevalent in all AI systems. Hopsworks offers simple Python APIs to define custom transformation functions. These can be used along with [feature groups](./feature_group/index.md) and [feature views](./feature_view/overview.md) to create [on-demand transformations](./feature_group/on_demand_transformations.md) and [model-dependent transformations](./feature_view/model-dependent-transformations.md), producing modular AI pipelines that are skew-free. + +## Custom Transformation Function Creation + +User-defined transformation functions can be created in Hopsworks using the [`@udf`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/udf/) decorator. These functions should be designed as Pandas functions, meaning they must take input features as a [Pandas Series](https://pandas.pydata.org/docs/reference/api/pandas.Series.html) and return either a Pandas Series or a [Pandas DataFrame](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html). Hopsworks automatically executes the defined transformation function as a [`pandas_udf`](https://spark.apache.org/docs/3.1.2/api/python/reference/api/pyspark.sql.functions.pandas_udf.html) in a PySpark application and as Pandas functions in Python clients. + +!!! warning "Java/Scala support" + + Hopsworks supports transformations functions in Python (Pandas UDFs, Python UDFs). Transformations functions can also be executed in Python-based DataFrame frameworks (PySpark, Pandas). There is currently no support for transformation functions in SQL or Java-based feature pipelines. + +Transformation functions created in Hopsworks can be directly attached to feature views or feature groups or stored in the feature store for later retrieval. These functions can be part of a library [installed](../../user_guides/projects/python/python_install.md) in Hopsworks or be defined in a [Jupyter notebook](../../user_guides/projects/jupyter/python_notebook.md) running a Python kernel or added when starting a Jupyter notebook or [Hopsworks job](../../user_guides/projects/jobs/spark_job.md). + +!!! warning "PySpark Kernels" + + Definition transformation function within a Jupyter notebook is only supported in Python Kernel. In a PySpark Kernel transformation function have to defined as modules or added when starting a Jupyter notebook. + + +The `@udf` decorator in Hopsworks creates a metadata class called [`HopsworksUdf`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/hopsworks_udf/). This class manages the necessary operations to execute the transformation function. The decorator has two arguments `return_type` and `drop`. The `return_type` is a mandatory argument and denotes the data types of the features returned by the transformation function. It can be a single Python type if the transformation function returns a single transformed feature or a list of Python types if it returns multiple transformed features. The supported types include `str`, `int`, `float`, `bool`, `datetime.datetime`, `datetime.date`, and `datetime.time`. The `drop` argument is optional and specifies the input arguments to remove from the final output after all transformation functions are applied. By default, all input arguments are retained in the final transformed output. The supported python types that be used with the `return_type` argument are provided as a table below + +| Supported Python Types | +|--------------------------| +| str | +| int | +| float | +| bool | +| datetime.datetime | +| datetime.date | +| datetime.time | + + +Hopsworks supports four types of transformation functions: + +1. One-to-one: Transforms one feature into one transformed feature. +2. One-to-many: Transforms one feature into multiple transformed features. +3. Many-to-one: Transforms multiple features into one transformed feature. +4. Many-to-many: Transforms multiple features into multiple transformed features. + + +### One-to-one transformations + +To create a one-to-one transformation function, the Hopsworks `@udf` decorator must be provided with the `return_type` as a single Python type. The transformation function should take one argument as input and return a Pandas Series. + +=== "Python" + + !!! example "Creation of a one-to-one transformation function in Hopsworks." + ```python + from hopsworks import udf + + @udf(return_type=int) + def add_one(feature): + return feature + 1 + ``` + +### Many-to-one transformations + +The creation of many-to-one transformation functions is similar to that of a one-to-one transformation function, the only difference being that the transformation function accepts multiple features as input. + +=== "Python" + !!! example "Creation of a many-to-one transformation function in Hopsworks." + ```python + from hopsworks import udf + + @udf(return_type=int) + def add_features(feature1, feature2, feature3): + return feature + feature2 + feature3 + ``` + +### One-to-many transformations + +To create a one-to-many transformation function, the Hopsworks `@udf` decorator must be provided with the `return_type` as a list of Python types, and the transformation function should take one argument as input and return multiple features as a Pandas DataFrame. The return types provided to the decorator must match the types of each column in the returned Pandas DataFrame. + +=== "Python" + !!! example "Creation of a one-to-many transformation function in Hopsworks." + ```python + from hopsworks import udf + import pandas as pd + + @udf(return_type=[int, int]) + def add_one_and_two(feature1): + return pd.DataFrame({"add_one":feature1 + 1, "add_two":feature1 + 2}) + ``` + +### Many-to-many transformations + +The creation of a many-to-many transformation function is similar to that of a one-to-many transformation function, the only difference being that the transformation function accepts multiple features as input. + +=== "Python" + !!! example "Creation of a many-to-many transformation function in Hopsworks." + ```python + from hopsworks import udf + import pandas as pd + + @udf(return_type=[int, int, int]) + def add_one_multiple(feature1, feature2, feature3): + return pd.DataFrame({"add_one_feature1":feature1 + 1, "add_one_feature2":feature2 + 1, "add_one_feature3":feature3 + 1}) + ``` + +### Dropping input features + +The `drop` parameter of the `@udf` decorator is used to drop specific columns in the input DataFrame after transformation. If any argument of the transformation function is passed to the `drop` parameter, then the column mapped to the argument is dropped after the transformation functions are applied. In the example below, the columns mapped to the arguments `feature1` and `feature2` are dropped after the application of all transformation functions. + + +=== "Python" + !!! example "Specify arguments to drop after transformation" + ```python + from hopsworks import udf + import pandas as pd + + @udf(return_type=[int, int, int], drop=["feature1", "feature3"]) + def add_one_multiple(feature1, feature2, feature3): + return pd.DataFrame({"add_one_feature1":feature1 + 1, "add_one_feature2":feature2 + 1, "add_one_feature3":feature3 + 1}) + ``` + +### Training dataset statistics + +A keyword argument `statistics` can be defined in the transformation function if it requires training dataset statistics for any of its arguments. The `statistics` argument must be assigned an instance of the class [`TransformationStatistics`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/transformation_statistics/) as the default value. The `TransformationStatistics` instance must be initialized using the names of the arguments requiring statistics. + +!!! warning "Transformation Statistics" + + The statistics provided to the transformation function is the statistics computed using [the train set](https://www.hopsworks.ai/dictionary/train-training-set). Training dataset statistics are not available for on-demand transformations. + +The `TransformationStatistics` instance contains separate objects with the same name as the arguments used to initialize it. These objects encapsulate statistics related to the argument as instances of the class [`FeatureTransformationStatistics`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_transformation_statistics/). Upon instantiation, instances of `FeatureTransformationStatistics` contain `None` values and are updated with the required statistics after the creation of a training dataset. + + +=== "Python" + !!! example "Creation of a transformation function in Hopsworks that uses training dataset statistics" + ```python + from hopsworks import udf + from hopsworks.transformation_statistics import TransformationStatistics + + stats = TransformationStatistics("argument1", "argument2", "argument3") + + @udf(int) + def add_features(argument1, argument2, argument3, statistics=stats): + return argument + argument2 + argument3 + statistics.argument1.mean + statistics.argument2.mean + statistics.argument3.mean + ``` + + +## Saving to the Feature Store + +To save a transformation function to the feature store, use the function `create_transformation_function`. It creates a [`TransformationFunction`](http://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/transformation_functions_api/) object which can then be saved by calling the save function. The save function will throw an error if another transformation function with the same name and version is already saved in the feature store. + +=== "Python" + + !!! example "Register transformation function `add_one` in the Hopsworks feature store" + ```python + plus_one_meta = fs.create_transformation_function( + transformation_function=add_one, + version=1) + plus_one_meta.save() + ``` + +## Retrieval from the Feature Store + +To retrieve all transformation functions from the feature store, use the function `get_transformation_functions`, which returns the list of `TransformationFunction` objects. + +A specific transformation function can be retrieved using its `name` and `version` with the function `get_transformation_function`. If only the `name` is provided, then the version will default to 1. + +=== "Python" + + !!! example "Retrieving transformation functions from the feature store" + ```python + # get all transformation functions + fs.get_transformation_functions() + + # get transformation function by name. This will default to version 1 + plus_one_fn = fs.get_transformation_function(name="plus_one") + + # get transformation function by name and version. + plus_one_fn = fs.get_transformation_function(name="plus_one", version=2) + ``` + +## Using transformation functions + +Transformation functions can be used by attaching it to a feature view to [create model-dependent transformations](./feature_view/model-dependent-transformations.md) or attached to feature groups to [create on-demand transformations](./feature_group/on_demand_transformations.md) diff --git a/mkdocs.yml b/mkdocs.yml index 4301dda42..614c39a9c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -50,6 +50,7 @@ nav: - Model Serving: concepts/mlops/serving.md - Vector Database: concepts/mlops/opensearch.md - BI Tools: concepts/mlops/bi_tools.md + - Data Transformations: concepts/mlops/data_transformations.md - Development: - Outside Hopsworks: concepts/dev/outside.md # api-keys - Inside Hopsworks: concepts/dev/inside.md @@ -85,6 +86,7 @@ nav: - Getting started: user_guides/fs/feature_group/feature_monitoring.md - Advanced guide: user_guides/fs/feature_monitoring/feature_monitoring_advanced.md - Notification: user_guides/fs/feature_group/notification.md + - On-Demand Transformations: user_guides/fs/feature_group/on_demand_transformations.md - Feature View: - user_guides/fs/feature_view/index.md - Overview: user_guides/fs/feature_view/overview.md @@ -94,12 +96,13 @@ nav: - Feature server: user_guides/fs/feature_view/feature-server.md - Query: user_guides/fs/feature_view/query.md - Helper Columns: user_guides/fs/feature_view/helper-columns.md - - Model-Dependent Transformation Functions: user_guides/fs/feature_view/transformation-function.md + - Model-Dependent Transformation Functions: user_guides/fs/feature_view/model-dependent-transformations.md - Spines: user_guides/fs/feature_view/spine-query.md - Feature Monitoring: - Getting started: user_guides/fs/feature_view/feature_monitoring.md - Advanced guide: user_guides/fs/feature_monitoring/feature_monitoring_advanced.md - Vector Similarity Search: user_guides/fs/vector_similarity_search.md + - Transformation Functions: user_guides/fs/transformation_functions.md - Compute Engines: user_guides/fs/compute_engines.md - Client Integrations: - user_guides/integrations/index.md