Wip better search #24

garland3 · 2025-09-17T12:24:12Z

No description provided.

backend/utils/crud.py

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

Copilot

Pull Request Overview

This PR implements an enhanced search functionality for the image gallery, replacing simple filename search with comprehensive filtering capabilities. The changes enable users to search across multiple image attributes including metadata, content type, and upload information.

Replaces basic filename search with a multi-field search system supporting wildcards
Adds server-side search filtering with database query optimization
Introduces a unified metadata field handling approach using both metadata and metadata_ properties

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
frontend/src/components/ImageMetadata.js	Updates metadata handling to support both `metadata` and `metadata_` fields consistently
frontend/src/components/ImageGallery.js	Implements advanced search UI with field selection, wildcard support, and dynamic metadata key options
frontend/src/Project.js	Passes search parameters to the backend API for server-side filtering
frontend/src/App.css	Adds styling for new search interface components and custom metadata table
backend/utils/crud.py	Implements database-level search filtering with support for JSON metadata queries
backend/routers/images.py	Adds search parameter handling and cache key updates for the images API

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-17T12:33:36Z

backend/utils/crud.py

+        elif search_field == 'metadata':
+            # Search across all metadata values using JSON text search
+            # This uses PostgreSQL's jsonb operators
+            query = query.where(text("metadata_::text ILIKE :search_value")).params(search_value=search_value_lower)


Using raw SQL with text() and user input creates potential SQL injection vulnerability. Consider using SQLAlchemy's cast() function or proper JSON operators instead of string interpolation in raw SQL.

Copilot · 2025-09-17T12:33:36Z

backend/utils/crud.py

+        else:
+            # Search specific metadata key using JSON path
+            # This searches for the specific key in the metadata JSON
+            query = query.where(text("metadata_ ->> :key ILIKE :search_value")).params(key=search_field, search_value=search_value_lower)


Similar SQL injection risk as above. The search_field parameter is passed directly to raw SQL. Use SQLAlchemy's proper JSON path operators or validate/sanitize the key parameter.

Suggested change

query = query.where(text("metadata_ ->> :key ILIKE :search_value")).params(key=search_field, search_value=search_value_lower)

import re

# Only allow alphanumeric and underscore keys (adjust as needed)

if re.match(r'^[a-zA-Z0-9_]+$', search_field):

# Safe to interpolate the key name directly

query = query.where(text(f"metadata_ ->> '{search_field}' ILIKE :search_value")).params(search_value=search_value_lower)

else:

# Invalid key, skip filtering

pass

Copilot · 2025-09-17T12:33:36Z

frontend/src/components/ImageGallery.js

+  useEffect(() => {
+    if (refreshProjectImages) {
+      refreshProjectImages({
+        searchField: searchValue ? searchField : null,
+        searchValue: searchValue || null
+      });
+    }
+  }, [searchField, searchValue]);


This useEffect will trigger API calls on every keystroke when typing in the search field. Consider debouncing the search to avoid excessive API requests.

Copilot · 2025-09-17T12:33:37Z

frontend/src/components/ImageGallery.js

+          const filename = (image.filename || '').toLowerCase();
+          // Support wildcard search with *
+          if (searchLower.includes('*')) {
+            const regex = new RegExp(searchLower.replace(/\*/g, '.*'), 'i');


Creating regex from user input without escaping special characters can cause ReDoS (Regular Expression Denial of Service) attacks. Consider escaping regex special characters or using a safer wildcard implementation.

Suggested change

const regex = new RegExp(searchLower.replace(/\*/g, '.*'), 'i');

// Escape regex special characters except for '*', then replace '*' with '.*'

const escapeRegExp = (str) => str.replace(/[-[\]/{}()+?.\\^$|]/g, '\\$&');

const safePattern = escapeRegExp(searchLower).replace(/\\\*/g, '.*');

const regex = new RegExp('^' + safePattern + '$', 'i');

backend/utils/crud.py

 import uuid
-from sqlalchemy import select, update, delete, and_
+import re
+from sqlalchemy import select, update, delete, and_, text, or_, cast, String


The best way to fix the problem is to remove the unnecessary text import from the import statement on line 3 in backend/utils/crud.py. Keep all other imported symbols intact, as they may be used elsewhere in the code. The change should only modify the import line, specifically deleting text from the list of symbols imported from sqlalchemy. No other changes are required to existing functionality—just a precise edit to the import.

backend/utils/crud.py

Copilot

Pull Request Overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-17T12:46:19Z

backend/utils/crud.py

+                query = query.where(models.DataInstance.metadata_[search_field].astext.ilike(search_value_lower))
+            else:
+                # Invalid key format, skip filtering for security
+                logger.warning(f"Invalid metadata key format rejected: {search_field}")


The logger is not imported or defined in this file. This will cause a NameError when an invalid metadata key format is encountered.

Copilot · 2025-09-17T12:46:19Z

frontend/src/components/ImageGallery.js

+          // Support wildcard search with * (with ReDoS protection)
+          if (searchLower.includes('*')) {
+            // Escape all regex special characters except *
+            const escapeRegExp = (str) => str.replace(/[-[\]/{}()+?.\\^$|]/g, '\\$&');


[nitpick] The escapeRegExp function is defined inline within the filter logic. Consider extracting this utility function outside the component or to a utils module for better maintainability and potential reuse.

Copilot · 2025-09-17T12:46:19Z

backend/utils/crud.py

+        else:
+            # Search specific metadata key using JSON path with input validation
+            # Only allow alphanumeric characters, underscores, and hyphens for security
+            if re.match(r'^[a-zA-Z0-9_-]+$', search_field):


[nitpick] The regex pattern for validating metadata keys is a magic string that could benefit from being defined as a constant at the module level with a descriptive name like VALID_METADATA_KEY_PATTERN.

garland3 added 2 commits September 16, 2025 00:25

wip: better search

bf7beb1

fixed some problems on search

a2ee926

garland3 requested review from Copilot and removed request for Copilot September 17, 2025 12:24

github-advanced-security bot found potential problems Sep 17, 2025

View reviewed changes

backend/utils/crud.py Fixed Show fixed Hide fixed

Potential fix for code scanning alert no. 1481: Unused import

fdd6e8e

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

Copilot AI review requested due to automatic review settings September 17, 2025 12:31

Copilot AI reviewed Sep 17, 2025

View reviewed changes

bot fixes

80b888d

github-advanced-security bot found potential problems Sep 17, 2025

View reviewed changes

tests

d0092e0

garland3 requested a review from Copilot September 17, 2025 12:44

Copilot AI reviewed Sep 17, 2025

View reviewed changes

tests

a39adda

garland3 merged commit 9493e71 into main Sep 17, 2025
9 checks passed

garland3 deleted the wip-better-search branch September 17, 2025 12:54

@@ -1,6 +1,6 @@
             import uuid
             import re
-            from sqlalchemy import select, update, delete, and_, text, or_, cast, String
+            from sqlalchemy import select, update, delete, and_, or_, cast, String
             from sqlalchemy.ext.asyncio import AsyncSession
             from sqlalchemy.orm import selectinload
             from core import models, schemas

-            query = query.where(text("metadata_ ->> :key ILIKE :search_value")).params(key=search_field, search_value=search_value_lower)
+            import re
+            # Only allow alphanumeric and underscore keys (adjust as needed)
+            if re.match(r'^[a-zA-Z0-9_]+$', search_field):
+                # Safe to interpolate the key name directly
+                query = query.where(text(f"metadata_ ->> '{search_field}' ILIKE :search_value")).params(search_value=search_value_lower)
+            else:
+                # Invalid key, skip filtering
+                pass

-            const regex = new RegExp(searchLower.replace(/\*/g, '.*'), 'i');
+            // Escape regex special characters except for '*', then replace '*' with '.*'
+            const escapeRegExp = (str) => str.replace(/[-[\]/{}()+?.\\^$|]/g, '\\$&');
+            const safePattern = escapeRegExp(searchLower).replace(/\\\*/g, '.*');
+            const regex = new RegExp('^' + safePattern + '$', 'i');

Wip better search #24

Wip better search #24

Uh oh!

Conversation

garland3 commented Sep 17, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Check notice

Copilot Autofix

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant