Skip to content

Commit d3e7f9f

Browse files
authored
DEV: add RQE dev to prod page (#1062)
* DEV: add RQE dev to prod page * Fix capitalization * Apply review comments.
1 parent b54dace commit d3e7f9f

File tree

1 file changed

+105
-0
lines changed

1 file changed

+105
-0
lines changed
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
---
2+
Title: Move from Development to Production with Redis Query Engine
3+
alwaysopen: false
4+
categories:
5+
- docs
6+
- develop
7+
- stack
8+
- oss
9+
- kubernetes
10+
- clients
11+
linkTitle: RQE DEV to PROD
12+
weight: 2
13+
---
14+
15+
Transitioning a Redis Community Edition with Redis Query Engine (RQE) environment from development to production requires thoughtful consideration of configuration, performance tuning, and resource allocation. This guide outlines key practices to ensure your Redis deployment operates optimally under production workloads.
16+
17+
## Configuration parameter considerations
18+
19+
RQE offers several configurable parameters that influence query results and performance. While a full list of these parameters and their functions can be found [here]({{< relref "/develop/interact/search-and-query/advanced-concepts/dialects" >}}), this section highlights the most commonly adjusted parameters for production environments.
20+
21+
### 1. `TIMEOUT`
22+
23+
- Purpose: limits the duration a query is allowed to execute.
24+
- Default: 500 milliseconds.
25+
- Behavior:
26+
- Ensures that queries do not monopolize the main Redis thread.
27+
- If a query exceeds the `TIMEOUT` value, its outcome is determined by the `ON_TIMEOUT` setting:
28+
- `FAIL`: the query will return an error.
29+
- `PARTIAL`: this setting will return the top results accumulated by the query until it timed out.
30+
- Recommendations:
31+
- Caution: be mindful when increasing `TIMEOUT`,<!-- especially in environments without QPF,--> as long-running queries can degrade overall system performance.
32+
<!-- - With Query Performance Factor (QPF): if QPF (multi-threaded query execution) is enabled, the risks of increasing this value are mitigated, as queries execute against index memory using multiple threads.-->
33+
34+
### 2. `MINPREFIX`
35+
36+
- Purpose: sets the minimum number of characters required for wildcard searches.
37+
- Default: 2 characters.
38+
- Behavior:
39+
- Queries like `he*` are valid, while `h*` would not meet the threshold.
40+
- Recommendations:
41+
- Lowering this value to 1 can significantly increase result sets, which may lead to degraded performance.
42+
- Keep the default unless there is a strong use case for single-character wildcards.
43+
44+
### 3. `MAXPREFIXEXPANSIONS`
45+
46+
- Purpose: Defines the maximum number of expansions for a wildcard query term.
47+
- Default: 200 expansions.
48+
- Behavior:
49+
- Expansions: when a wildcard query term is processed, Redis generates a list of all possible matches from the index that satisfy the wildcard. For example, the query he* might expand to terms like hello, hero, and heat. Each of these matches is an "expansion."
50+
- This parameter limits how many of these expansions Redis will generate and process. If the number of possible matches exceeds the limit, the query may return incomplete results or fail, depending on the query context.
51+
- Recommendations:
52+
- Avoid increasing this parameter excessively, as it can lead to performance bottlenecks during query execution.
53+
- If wildcard searches are common, consider optimizing your index to reduce the reliance on large wildcard expansions.
54+
55+
### 4. `DEFAULT_DIALECT`
56+
57+
- Purpose: specifies the default query dialect used by [`FT.SEARCH`]({{< baseurl >}}/commands/ft.search) and [`FT.AGGREGATE`]({{< baseurl >}}/commands/ft.aggregate) commands.
58+
- Default: [Dialect 1]({{< relref "/develop/interact/search-and-query/advanced-concepts/dialects" >}}).
59+
- Recommendations:
60+
- Update the default to [**Dialect 4**]({{< relref "/develop/interact/search-and-query/advanced-concepts/dialects#dialect-4" >}}) for better performance and access to advanced features.
61+
- Individual commands can override this parameter if necessary, but setting a higher default ensures consistent performance across queries.
62+
63+
## Testing
64+
65+
### 1. Correctness
66+
- Run a few test queries and check the results are what you expect.
67+
- Use the following tools to validate and debug:
68+
- Redis CLI: use the [`MONITOR`]({{< baseurl >}}/commands/monitor) command or [profiling features]({{< relref "/develop/tools/insight#profiler" >}}) in Redis Insight to analyze commands.
69+
- [`FT.PROFILE`]({{< baseurl >}}/commands/ft.profile): Provides detailed insights into individual query execution paths, helping identify bottlenecks and inefficiencies.
70+
71+
### 2. Performance
72+
- Test query performance in a controlled test environment that mirrors production as closely as possible.
73+
- Use tools like `memtier_benchmark` or custom test applications to simulate load.
74+
- Network Considerations:
75+
- Minimize latency during testing by locating test clients in the same network as the Redis instance.
76+
- For Redis Cloud, ensure test machines are in a **VPC-peered environment** with the target Redis database.
77+
78+
## Sizing requirements
79+
80+
Redis Search has resource requirements distinct from general caching use cases. Proper sizing ensures that the system can handle production workloads efficiently.
81+
82+
### Key considerations:
83+
1. CPU:
84+
- Adequate CPU resources are critical<!--, especially when using QPF-->.
85+
- Ensure CPUs are not over-subscribed with search threads and shard processes.
86+
2. RAM:
87+
- Plan for sufficient memory to store the dataset and indexes, plus overhead for operations.
88+
3. Network:
89+
- High throughput and low latency are essential, particularly for applications with demanding query patterns.
90+
91+
### Tools:
92+
- Use the [Redis Search Sizing Calculator](https://redis.io/redisearch-sizing-calculator/) to estimate resource requirements based on your dataset and workload.
93+
94+
## Demand spikes
95+
96+
Production environments must be sized for peak load scenarios to ensure performance remains acceptable under maximum stress.
97+
98+
### Recommendations:
99+
1. Plan for Spikes:
100+
- If query workloads are expected to vary significantly, ensure the infrastructure can handle peak loads.
101+
- Monitor real-world usage patterns and adjust capacity as needed.
102+
2. Autoscaling:
103+
- Consider using autoscaling strategies in cloud environments to dynamically adjust resources based on load.
104+
105+
By following these best practices, you can ensure a smooth and efficient transition from development to production with Redis Community Edition and RQE. Proper configuration, rigorous testing, and careful resource planning are critical to delivering a reliable and high-performance Redis deployment.

0 commit comments

Comments
 (0)