-
Notifications
You must be signed in to change notification settings - Fork 225
DEV: (RediSearch) doc expiration capabilities in Redis 8 #1792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+90
−0
Merged
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
79 changes: 79 additions & 0 deletions
79
content/develop/ai/search-and-query/advanced-concepts/expiration.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
--- | ||
categories: | ||
- docs | ||
- develop | ||
- stack | ||
- oss | ||
- rs | ||
- rc | ||
- oss | ||
- kubernetes | ||
- clients | ||
description: How the Redis Query Engine handles expiring keys and hash fields | ||
linkTitle: Key and field expiration | ||
title: Key and field expiration behavior | ||
weight: 8 | ||
--- | ||
|
||
The Redis Query Engine behavior with expiring keys and hash fields has been enhanced starting with Redis 8 to provide more consistent and predictable results. | ||
|
||
## Key expiration | ||
|
||
### Expiration times | ||
|
||
**Before Redis 8**: Expiration times were not taken into account when computing the result set. | ||
|
||
**Redis 8 and later**: The query engine prefers to return documents that are valid (not expired) at the time when the query or cursor read started. | ||
|
||
### Active expiration | ||
|
||
Active expiration can affect the number of results returned. For example, if a user requests 100 documents but 10 are actively expired during query execution, only 90 documents will be returned. | ||
|
||
**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts. | ||
|
||
### Passive expiration | ||
|
||
**Before Redis 8**: `nil` could have been returned as a document name in the result set for a query when a key was passively expired. | ||
dwdougherty marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
**Redis 8 and later**: Only valid document names will be returned. Passively expired keys are filtered out from the result set. | ||
|
||
## Field expiration | ||
|
||
Field expiration was introduced in Redis 7.4 and provides fine-grained control over hash field lifecycles. | ||
|
||
### Expiration times | ||
|
||
**Before Redis 8**: Field expiration times were not taken into account when computing the result set. | ||
|
||
**Redis 8 and later**: The query engine prefers to return documents that are valid (fields not expired) at the time when the query or cursor read started. | ||
|
||
### Active expiration | ||
|
||
Similar to key expiration, active field expiration can affect the number of results returned. If fields that match query criteria are actively expired during execution, fewer results will be returned than requested. | ||
|
||
**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts. | ||
|
||
### Passive expiration | ||
|
||
**Before Redis 8**: The query engine would possibly return documents without fields that were passively expired, even if the expired field caused the document to match the query. This behavior depended to some degree on the `SORTABLE` keyword usage. | ||
dwdougherty marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
**Redis 8 and later**: Documents will return with all fields that existed when the query or cursor read started. Passively expired fields are handled consistently. | ||
|
||
## Related commands | ||
|
||
The following commands are directly related to key and field expiration: | ||
|
||
- [`EXPIRE`]({{< relref "/commands/expire" >}}) - Set expiration time for keys | ||
- [`HEXPIRE`]({{< relref "/commands/hexpire" >}}) - Set expiration time for hash fields | ||
- [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) - Search queries affected by expiration behavior | ||
- [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) - Aggregation queries affected by expiration behavior | ||
|
||
## Best practices | ||
|
||
1. **Consistent timing**: Be aware that query results are based on the state at query start time, not when individual documents are processed. | ||
|
||
2. **Result count expectations**: When using expiring keys or fields, the actual number of results may be less than requested due to active expiration during query execution. | ||
dwdougherty marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
3. **Field-level expiration**: Use hash field expiration (available since Redis 7.4) for fine-grained control over document field lifecycles without affecting the entire document. | ||
|
||
4. **Query planning**: Consider expiration patterns when designing queries, especially for time-sensitive applications where expired data should not appear in results. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.