feat: Durable web search golem:web-search API across multiple providers in Rust #60
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
google.mp4 |
|
Brave brave.mp4 |
|
serper serper.mp4 |
|
tavily tavily.mp4 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
This pull request introduces a robust and extensible durable web search API for the Golem platform, implemented in Rust. This
web-searchcomponent provides a unified interface for performing web searches across multiple providers—including Brave, Google, Serper, and Tavily—enabling users to leverage the strengths of each search engine through a single, consistent API.Key features include support for durable, replayable search sessions, advanced filtering (such as domain inclusion/exclusion and safe search levels), multilingual and regional queries, and the ability to retrieve both standard and enriched content (like images and HTML snippets). The API is designed for reliability and flexibility, making it suitable for a wide range of use cases, from simple information retrieval to advanced, policy-driven or content-sensitive searches.
Tests Added
A comprehensive suite of tests has been included to validate the new web search API, including:
Basic Search Test:
Verifies simple search queries return expected results.
Session Pagination Test:
Ensures multiple pages of results can be fetched in a session.
Error Handling Test:
Confirms graceful handling of invalid queries and provider errors.
Domain Filtering Test:
Tests inclusion and exclusion of specific domains in search results.
Image and HTML Content Test:
Checks correct handling of image results and HTML snippets.
Multilingual and Regional Search Test:
Validates search with different languages and regions.
Advanced Search Test:
Demonstrates advanced search with high safe search and content chunks.
This test ensures the API can perform advanced queries with strict safe search settings and return results in content chunks, supporting use cases like child safety and trusted content discovery.
/closes Implement Durable Web Search
golem:web-searchAPI Across Different Providers in Rust #34/claim Implement Durable Web Search
golem:web-searchAPI Across Different Providers in Rust #34