From 186ef7bc6efa8dbe57f2b5a0ba6046cb96276eb4 Mon Sep 17 00:00:00 2001 From: saimedhi Date: Mon, 30 Jun 2025 13:56:49 -0700 Subject: [PATCH 1/9] Update opensearch flow documentation with Neural sparse search configuring details Signed-off-by: saimedhi --- _vector-search/ai-search/building-flows.md | 48 ++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 8a840ffd4a..072f920b31 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -441,3 +441,51 @@ Override the query so that it contains a `knn` query, including the embedding ou ``` {% include copy.html %} + + +--- + +## Neural sparse search + +### ML resources +Create and deploy a [Neural Sparse Encoding model](https://github.com/opensearch-project/dashboards-flow-framework/blob/main/documentation/models.md#neural-sparse-encoding). + +### Index + +Ensure the index mappings include `rank_features` field, as follows: + +``` +"": { + "type": "rank_features" +} +``` + +{% include copy.html %} + +### Ingest pipeline + +Single ML inference processor. Map your input text to the `text_doc` model input field. Optionally map the output `response` to a new document field. Transform the response if needed using JSONPath expression. + + +### Search pipeline + +Single ML inference **search request** processor. Map the query field containing the input text to the `text_doc` model input field. Optionally map the output `response` to a new field. Transform the response if needed using JSONPath expression. Override the query to a neural sparse query. For example: + +``` +{ + "_source": { + "excludes": [ + "" + ] + }, + "query": { + "neural_sparse": { + "": { + "query_tokens": ${response}, + } + } + } +} +``` + +{% include copy.html %} \ No newline at end of file From 176e6e43f541b6be70d8332f093af5541e97e6a6 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:43:41 -0700 Subject: [PATCH 2/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 072f920b31..8ace5b77ba 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -448,6 +448,7 @@ Override the query so that it contains a `knn` query, including the embedding ou ## Neural sparse search ### ML resources + Create and deploy a [Neural Sparse Encoding model](https://github.com/opensearch-project/dashboards-flow-framework/blob/main/documentation/models.md#neural-sparse-encoding). ### Index From 244df6f42e3c7f8429e83a2d08943f533332a606 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:44:04 -0700 Subject: [PATCH 3/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 8ace5b77ba..856deb02e0 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -453,7 +453,7 @@ Create and deploy a [Neural Sparse Encoding model](https://github.com/opensearch ### Index -Ensure the index mappings include `rank_features` field, as follows: +Ensure that the index mappings include a `rank_features` field: ``` "": { From ffe59d3284fe8663a71e4d176c4f59e3a4a132cf Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:44:17 -0700 Subject: [PATCH 4/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 856deb02e0..2cea083604 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -449,7 +449,7 @@ Override the query so that it contains a `knn` query, including the embedding ou ### ML resources -Create and deploy a [Neural Sparse Encoding model](https://github.com/opensearch-project/dashboards-flow-framework/blob/main/documentation/models.md#neural-sparse-encoding). +Create and deploy a [neural sparse encoding model](https://github.com/opensearch-project/dashboards-flow-framework/blob/main/documentation/models.md#neural-sparse-encoding). ### Index From 516744ba6a9832270f41bd649e0351162b4939b1 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:44:44 -0700 Subject: [PATCH 5/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 1 - 1 file changed, 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 2cea083604..8608d3a991 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -460,7 +460,6 @@ Ensure that the index mappings include a `rank_features` field: "type": "rank_features" } ``` - {% include copy.html %} ### Ingest pipeline From a42bdad678178f9ac52b6c63dba6d0e3b9dc2db2 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:45:10 -0700 Subject: [PATCH 6/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index 8608d3a991..edf7479780 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -464,7 +464,7 @@ Ensure that the index mappings include a `rank_features` field: ### Ingest pipeline -Single ML inference processor. Map your input text to the `text_doc` model input field. Optionally map the output `response` to a new document field. Transform the response if needed using JSONPath expression. +Configure a single ML inference processor. Map your input text to the `text_doc` model input field. Optionally, map the output `response` to a new document field. Transform the response if needed using a JSONPath expression. ### Search pipeline From f76dfce2a9871417f6daf1b7e50bc0b07adf0895 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:46:02 -0700 Subject: [PATCH 7/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index edf7479780..eee0ef0d67 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -469,7 +469,7 @@ Configure a single ML inference processor. Map your input text to the `text_doc` ### Search pipeline -Single ML inference **search request** processor. Map the query field containing the input text to the `text_doc` model input field. Optionally map the output `response` to a new field. Transform the response if needed using JSONPath expression. Override the query to a neural sparse query. For example: +Configure a single ML inference search request processor. Map the query field containing the input text to the `text_doc` model input field. Optionally, map the output `response` to a new field. Transform the response if needed using a JSONPath expression. Include a neural sparse query: ``` { From 716b2a5b2d85776d181e5e0ca2a609d55db8eac1 Mon Sep 17 00:00:00 2001 From: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> Date: Mon, 7 Jul 2025 13:46:09 -0700 Subject: [PATCH 8/9] Update _vector-search/ai-search/building-flows.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 1 - 1 file changed, 1 deletion(-) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index eee0ef0d67..d53510ff8c 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -487,5 +487,4 @@ Configure a single ML inference search request processor. Map the query field co } } ``` - {% include copy.html %} \ No newline at end of file From a118af5a21cd6fee1862d6b4912624cf91fc111b Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Mon, 7 Jul 2025 17:07:53 -0400 Subject: [PATCH 9/9] Update _vector-search/ai-search/building-flows.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _vector-search/ai-search/building-flows.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/_vector-search/ai-search/building-flows.md b/_vector-search/ai-search/building-flows.md index d53510ff8c..245765b2e1 100644 --- a/_vector-search/ai-search/building-flows.md +++ b/_vector-search/ai-search/building-flows.md @@ -447,6 +447,8 @@ Override the query so that it contains a `knn` query, including the embedding ou ## Neural sparse search +This example demonstrates how to configure neural sparse search. + ### ML resources Create and deploy a [neural sparse encoding model](https://github.com/opensearch-project/dashboards-flow-framework/blob/main/documentation/models.md#neural-sparse-encoding).