Skip to content

Commit 5ac90fa

Browse files
authored
DEV: document Unicode limitations (#1234)
* DEV: document Unicode limitations * Apply code review comments
1 parent fc38085 commit 5ac90fa

File tree

2 files changed

+56
-2
lines changed

2 files changed

+56
-2
lines changed

content/develop/interact/search-and-query/basic-constructs/field-and-type-options.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,4 +232,31 @@ You can search for documents with specific text values using the `<term>` or the
232232
- Search for a term only in the `title` attribute
233233
```
234234
FT.SEARCH books-idx "@title:dogs"
235-
```
235+
```
236+
237+
## Unicode considerations
238+
239+
Redis Query Engine only supports Unicode characters in the [basic multilingual plane](https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane); U+0000 to U+FFFF. Unicode characters beyond U+FFFF, such as Emojis, are not supported and would not be retrieved by queries including such characters in the following use cases:
240+
241+
* Querying TEXT fields with Prefix/Suffix/Infix
242+
* Querying TEXT fields with fuzzy
243+
244+
Examples:
245+
246+
```
247+
redis> FT.CREATE idx SCHEMA tag TAG text TEXT
248+
OK
249+
redis> HSET doc:1 tag '😀😁🙂' text '😀😁🙂'
250+
(integer) 2
251+
redis> HSET doc:2 tag '😀😁🙂abc' text '😀😁🙂abc'
252+
(integer) 2
253+
redis> FT.SEARCH idx '@text:(*😀😁🙂)' NOCONTENT
254+
1) (integer) 0
255+
redis> FT.SEARCH idx '@text:(*😀😁🙂*)' NOCONTENT
256+
1) (integer) 0
257+
redis> FT.SEARCH idx '@text:(😀😁🙂*)' NOCONTENT
258+
1) (integer) 0
259+
260+
redis> FT.SEARCH idx '@text:(%😀😁🙃%)' NOCONTENT
261+
1) (integer) 0
262+
```

content/develop/interact/search-and-query/query/full-text.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,4 +117,31 @@ If you want to increase the maximum word distance to two, you can use the follow
117117

118118
{{< clients-example query_ft ft5 >}}
119119
FT.SEARCH idx:bicycle "%%optamised%%"
120-
{{< /clients-example >}}
120+
{{< /clients-example >}}
121+
122+
## Unicode considerations
123+
124+
Redis Query Engine only supports Unicode characters in the [basic multilingual plane](https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane); U+0000 to U+FFFF. Unicode characters beyond U+FFFF, such as Emojis, are not supported and would not be retrieved by queries including such characters in the following use cases:
125+
126+
* Querying TEXT fields with Prefix/Suffix/Infix
127+
* Querying TEXT fields with fuzzy
128+
129+
Examples:
130+
131+
```
132+
redis> FT.CREATE idx SCHEMA tag TAG text TEXT
133+
OK
134+
redis> HSET doc:1 tag '😀😁🙂' text '😀😁🙂'
135+
(integer) 2
136+
redis> HSET doc:2 tag '😀😁🙂abc' text '😀😁🙂abc'
137+
(integer) 2
138+
redis> FT.SEARCH idx '@text:(*😀😁🙂)' NOCONTENT
139+
1) (integer) 0
140+
redis> FT.SEARCH idx '@text:(*😀😁🙂*)' NOCONTENT
141+
1) (integer) 0
142+
redis> FT.SEARCH idx '@text:(😀😁🙂*)' NOCONTENT
143+
1) (integer) 0
144+
145+
redis> FT.SEARCH idx '@text:(%😀😁🙃%)' NOCONTENT
146+
1) (integer) 0
147+
```

0 commit comments

Comments
 (0)