Skip to content

DEV: (RediSearch) doc expiration capabilities in Redis 8 #1792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions content/commands/expire.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,10 @@ still take the full state of the expires existing in the dataset, so when a
replica is elected to master it will be able to expire the keys independently,
fully acting as a master.

### Redis Query Engine and expiration

Starting with Redis 8, the Redis Query Engine has enhanced behavior when handling expiring keys. For detailed information about how [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) and [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) commands interact with expiring keys, see [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}}).

## Return information

{{< multitabs id="expire-return-info"
Expand Down
1 change: 1 addition & 0 deletions content/commands/ft.aggregate.md
Original file line number Diff line number Diff line change
Expand Up @@ -496,5 +496,6 @@ Next, count GitHub events by user (actor), to produce the most active users.
## Related topics

- [Aggregations]({{< relref "/develop/ai/search-and-query/advanced-concepts/aggregations" >}})
- [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})
- [RediSearch]({{< relref "/develop/ai/search-and-query" >}})

1 change: 1 addition & 0 deletions content/commands/ft.search.md
Original file line number Diff line number Diff line change
Expand Up @@ -860,5 +860,6 @@ Query with `CONTAINS` operator:

- [Extensions]({{< relref "/develop/ai/search-and-query/administration/extensions" >}})
- [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}})
- [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})
- [Query syntax]({{< relref "/develop/ai/search-and-query/query/" >}})
- [RediSearch]({{< relref "/develop/ai/search-and-query/" >}})
4 changes: 4 additions & 0 deletions content/commands/hexpire.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ You can call `HEXPIRE` using as argument a field that already has an
existing TTL set.
In this case, the time to live is _updated_ to the new value.

## Redis Query Engine and field expiration

Starting with Redis 8, the Redis Query Engine has enhanced behavior when handling expiring hash fields. For detailed information about how [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) and [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) commands interact with expiring hash fields, see [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}}).

## Example

```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Redis Open Source supports the following Redis Query Engine features. This artic
* Numeric filters and ranges
* Geo-filtering using Redis [geo commands]({{< relref "/commands/" >}}?group=geo)
* [Vector search]({{< relref "/develop/ai/search-and-query/vectors" >}})
* [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})


## Full-text search features
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
categories:
- docs
- develop
- stack
- oss
- rs
- rc
- oss
- kubernetes
- clients
description: How the Redis Query Engine handles expiring keys and hash fields
linkTitle: Key and field expiration
title: Key and field expiration behavior
weight: 8
---

The Redis Query Engine behavior with expiring keys and hash fields has been enhanced starting with Redis 8 to provide more consistent and predictable results.

## Key expiration

### Expiration times

**Before Redis 8**: Expiration times were not taken into account when computing the result set.

**Redis 8 and later**: The query engine returns only documents that are valid (not expired) at the time when the query or cursor read started.

### Active expiration

Active expiration can affect the number of results returned. For example, if a user requests 100 documents but 10 are actively expired during query execution, only 90 documents will be returned.

**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.

### Passive expiration

**Before Redis 8**: A query could return `nil` as a document name in the result set for a key that was passively expired.

**Redis 8 and later**: Only valid document names will be returned. Passively expired keys are filtered out from the result set.

## Field expiration

Field expiration was introduced in Redis 7.4 and provides fine-grained control over hash field lifecycles.

### Expiration times

**Before Redis 8**: Field expiration times were not taken into account when computing the result set.

**Redis 8 and later**: The query engine returns only documents that are valid (fields not expired) at the time when the query or cursor read started.

### Active expiration

Similar to key expiration, active field expiration can affect the number of results returned. If fields that match query criteria are actively expired during execution, fewer results will be returned than requested.

**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.

### Passive expiration

**Before Redis 8**: The query engine could return documents without fields that were passively expired, even if the expired field caused the document to match the query. This behavior depended to some degree on the `SORTABLE` keyword usage.

**Redis 8 and later**: Documents will return with all fields that existed when the query or cursor read started. Passively expired fields are handled consistently.

## Best practices

* **Consistent timing**: Be aware that query results are based on the state at query start time, not when individual documents are processed.

* **Result count expectations**: Be aware that when you use expiring keys or fields, the actual number of results may be less than you expected due to active expiration during the execution of the query.

* **Field-level expiration**: Use hash field expiration (available since Redis 7.4) for fine-grained control over document field lifecycles without affecting the entire document.

* **Query planning**: Consider expiration patterns when designing queries, especially for time-sensitive applications where expired data should not appear in results.

## Related commands

The following commands are directly related to key and field expiration:

- [`EXPIRE`]({{< relref "/commands/expire" >}}) - Set expiration time for keys
- [`HEXPIRE`]({{< relref "/commands/hexpire" >}}) - Set expiration time for hash fields
- [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) - Search queries affected by expiration behavior
- [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) - Aggregation queries affected by expiration behavior