Skip to content

feat: Include vectorized text in search queries #953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 21, 2024

Conversation

cecheta
Copy link
Collaborator

@cecheta cecheta commented May 20, 2024

Required by #750

Purpose

  • If advanced image processing is enabled, then the question will be vectorised with the same model as used for vectorising images
  • A second vector query is included when retrieving documents from AI search
  • Note that this is only for non-integrated vectorisation
  • Any retrieved images are not yet passed to the LLM, however the caption of the image is

Does this introduce a breaking change?

  • Yes
  • No

How to Test

  • Enable advanced image processing
  • Deploy
  • Upload images
  • Ask questions based on images

Copy link

github-actions bot commented May 20, 2024

Coverage

Coverage Report •
FileStmtsMissCoverMissing
code/backend/batch/utilities/helpers
   azure_computer_vision_client.py530100% 
   azure_search_helper.py570100% 
code/backend/batch/utilities/helpers/embedders
   push_embedder.py730100% 
code/backend/batch/utilities/search
   azure_search_handler.py62296%24, 30
   integrated_vectorization_search_handler.py68395%35, 135, 144
   search.py14192%16
   search_handler_base.py512354%16–18, 21–23, 27, 31, 35, 39, 43, 47, 51, 54–55, 57–59, 61–62, 65–66, 68
TOTAL246067272% 

Tests Skipped Failures Errors Time
211 0 💤 0 ❌ 0 🔥 11.437s ⏱️

adamdougal
adamdougal previously approved these changes May 20, 2024
Copy link
Collaborator

@adamdougal adamdougal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cecheta
Copy link
Collaborator Author

cecheta commented May 20, 2024

lgtm

I jumped the gun 😬

adamdougal
adamdougal previously approved these changes May 20, 2024
ross-p-smith
ross-p-smith previously approved these changes May 20, 2024
@cecheta cecheta dismissed stale reviews from ross-p-smith and adamdougal via a31c563 May 21, 2024 10:27
@cecheta cecheta force-pushed the cecheta/748/question-embeddings branch from b25fd64 to a31c563 Compare May 21, 2024 10:27
@cecheta cecheta added this pull request to the merge queue May 21, 2024
Merged via the queue into main with commit 8642df3 May 21, 2024
11 checks passed
@cecheta cecheta deleted the cecheta/748/question-embeddings branch May 21, 2024 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants