Skip to content

Python: Introducing the Faiss Connector #10810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 11, 2025

Conversation

eavanvalkenburg
Copy link
Member

@eavanvalkenburg eavanvalkenburg commented Mar 5, 2025

Motivation and Context

This PR adds a FaissVectorStore and FaissCollection
Faiss is a really fast similarity search engine, which runs in memory.
You can also use GPU's with it, but you will have to build the package yourself.

Description

Closes #4130

TODO:

  • other index kinds
  • deal with distance functions
  • validate GPU setup
  • maybe subclass InMemoryCollection
  • tests

Contribution Checklist

@eavanvalkenburg eavanvalkenburg requested a review from a team as a code owner March 5, 2025 16:19
@eavanvalkenburg eavanvalkenburg marked this pull request as draft March 5, 2025 16:19
@markwallace-microsoft markwallace-microsoft added python Pull requests for the Python Semantic Kernel memory labels Mar 5, 2025
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Mar 5, 2025

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
semantic_kernel/connectors/memory
   faiss.py119794%91, 99, 103, 131–132, 159, 217
semantic_kernel/connectors/memory/in_memory
   in_memory_collection.py1391589%70, 122, 124, 144, 183, 198, 210, 228–231, 235, 237, 239–240
TOTAL19762258487% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
3226 5 💤 0 ❌ 0 🔥 1m 42s ⏱️

@eavanvalkenburg eavanvalkenburg force-pushed the faiss branch 3 times, most recently from c6c678d to 22306a1 Compare March 10, 2025 08:52
@eavanvalkenburg eavanvalkenburg marked this pull request as ready for review March 10, 2025 11:04
@eavanvalkenburg eavanvalkenburg requested a review from Copilot March 10, 2025 11:30
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR introduces a new Faiss connector for semantic-kernel, including implementations for FaissCollection and FaissStore along with comprehensive unit tests and sample usage updates.

  • Adds FaissCollection and FaissStore classes to integrate Faiss as an in-memory similarity search engine.
  • Updates tests, configuration (pyproject.toml), and sample usage to support the new Faiss connector.

Reviewed Changes

File Description
python/tests/unit/connectors/memory/faiss/test_faiss.py Adds unit tests for Faiss functionality.
python/semantic_kernel/connectors/memory/faiss.py Implements the FaissCollection and FaissStore classes.
python/tests/conftest.py Updates import for Kernel initialization.
python/pyproject.toml Adds dependency for faiss-cpu.
python/samples/concepts/memory/complex_memory.py Integrates FaissCollection into sample usage.
python/semantic_kernel/connectors/memory/in_memory/in_memory_collection.py Refines generic types for in-memory collections.
python/semantic_kernel/data/vector_storage/vector_store.py Minor type annotations update.

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Mar 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 11, 2025
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Mar 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 11, 2025
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Mar 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 11, 2025
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Mar 11, 2025
Merged via the queue into microsoft:main with commit 45f4234 Mar 11, 2025
28 checks passed
@eavanvalkenburg eavanvalkenburg deleted the faiss branch March 11, 2025 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
memory python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python: Add Faiss Connector with new Vector Search Design
4 participants