Skip to content

.Net New Feature: VectorStore - Support for .Any() Method in Filters #13152

@pawap90

Description

@pawap90

name: VectorStore - Support for .Any() Method in Filters
about: Add support for Enumerable.Any() in VectorStore LINQ filters to enable dynamic array matching against list/array columns


Feature Request

Currently, VectorStoreCollection LINQ-based filters do not support the .Any() method in filter expressions when working with array/list properties. This limitation prevents filtering on collections where dynamic arrays need to be matched against columns with list/array data types.

Problem

When attempting to use .Any() method in filter expressions for PostgreSQL vector store operations, the following exception is thrown:

System.NotSupportedException : Unsupported method call: Enumerable.Any
   at Microsoft.SemanticKernel.Connectors.SqlFilterTranslator.TranslateMethodCall(MethodCallExpression methodCall, Boolean isSearchCondition)
   at Microsoft.SemanticKernel.Connectors.SqlFilterTranslator.Translate(Expression node, Boolean isSearchCondition)
   at Microsoft.SemanticKernel.Connectors.SqlFilterTranslator.Translate(Boolean appendWhere)
   at Microsoft.SemanticKernel.Connectors.PgVector.PostgresSqlBuilder.BuildSelectWhereCommand[TRecord](NpgsqlCommand command, String schema, String tableName, CollectionModel model, Expression`1 filter, Int32 top, FilteredRecordRetrievalOptions`1 options)

Example Scenario

Consider the following data model:

protected class VectorStoreRecord
{
    [VectorStoreKey]
    public required int Id { get; set; }

    [VectorStoreData(StorageName = "tags")]
    public List<string> Tags { get; set; } = [];

    [VectorStoreData]
    public required string Text { get; set; }
}

Sample data:

new VectorStoreRecord
{
    Id = 1,
    Tags = [ "manual", "pdf", "introduction" ],
    Text = "Chapter 1: Introduction..."
},
new VectorStoreRecord
{
    Id = 2,
    Tags = [ "manual", "pdf", "installation" ],
    Text = "Chapter 2: Installation..."
}

Here's where the exception is thrown:

private readonly VectorStoreCollection<int, VectorStoreRecord> collection;

/* ... */

string[] tagsToFilter = ["pdf", "introduction", "faq"];
await foreach (var result in collection.GetAsync(
    filter: r => r.Tags.Any(tag => tagsToFilter.Contains(tag)), // Throws NotSupportedException
    top: 10
))

Note: This issue was specifically tested and reproduced using the PostgreSQL vector store implementation.

Test repo

You can find a minimal reproduction of this issue in the following repository:
SemanticKernel.PostgresVectorStore.FilterTests

Questions

  1. Are there plans to expand LINQ method support in PostgreSQL vector store filters?
  2. Would the team be open to a community contribution implementing this feature?

Thank you for considering this enhancement!

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETIssue or Pull requests regarding .NET codetriage

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions