Skip to content

Commit 362e090

Browse files
Swiddiskolchfa-awsnatebower
authored
Document performance limitation for SQL (#10075)
* Document performance limitation for SQL Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Fix styling Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Add link to data sources section Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Update for PR feedback Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Fix style-job Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * a few -> around 5 million Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Update restriction details Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Add information on reducing search space Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> * Update _search-plugins/sql/limitation.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Nathan Bower <nbower@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@gmail.com> Signed-off-by: Nathan Bower <nbower@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
1 parent 83f276e commit 362e090

File tree

1 file changed

+37
-3
lines changed

1 file changed

+37
-3
lines changed

_search-plugins/sql/limitation.md

Lines changed: 37 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,44 @@ FROM (
2929

3030
But, if the outer query has `GROUP BY` or `ORDER BY`, then it's not supported.
3131

32-
## JOIN does not support aggregations on the joined result
32+
## JOIN queries
3333

34-
The `join` query does not support aggregations on the joined result.
35-
For example, e.g. `SELECT depo.name, avg(empo.age) FROM empo JOIN depo WHERE empo.id == depo.id GROUP BY depo.name` is not supported.
34+
Because OpenSearch doesn't natively support relational operations, `JOIN` queries are supported on a best-effort basis.
35+
36+
### JOIN does not support aggregations on the joined result
37+
38+
The `JOIN` query does not support aggregations on the joined result.
39+
40+
For example, `SELECT depo.name, avg(empo.age) FROM empo JOIN depo WHERE empo.id = depo.id GROUP BY depo.name` is not supported.
41+
42+
### Performance
43+
44+
`JOIN` queries are prone to expensive index scanning operations.
45+
46+
`JOIN` queries may experience performance issues when working with result sets larger than 5 million matching records.
47+
To improve `JOIN` performance, reduce the number of records being joined by filtering your data first. For example, limit the join to a specific range of key values:
48+
49+
```sql
50+
SELECT l.key, l.spanId, r.spanId
51+
FROM logs_left AS l
52+
JOIN logs_right AS r
53+
ON l.key = r.key
54+
WHERE l.key >= 17491637400000
55+
AND l.key < 17491637500000
56+
AND r.key >= 17491637400000
57+
AND r.key < 17491637500000
58+
LIMIT 10
59+
```
60+
{% include copy.html %}
61+
62+
By default, JOIN queries will automatically terminate after 60 seconds to prevent excessive resource consumption. You can adjust this timeout period using a hint in your query. For example, to set a 5-minute (300-second) timeout, use the following code:
63+
64+
```sql
65+
SELECT /*! JOIN_TIME_OUT(300) */ left.a, right.b FROM left JOIN right ON left.id = right.id;
66+
```
67+
{% include copy.html %}
68+
69+
These performance restrictions don't apply when [querying external data sources]({{site.url}}{{site.baseurl}}/dashboards/management/query-data-source/).
3670

3771
## Pagination only supports basic queries
3872

0 commit comments

Comments
 (0)