Skip to content

Commit be8c2c8

Browse files
authored
Merge pull request #65 from cipherstash/docs-v2
docs: seperate json docs into reference guide
2 parents 22f6ae5 + 0c7ca4d commit be8c2c8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+935
-748
lines changed

README.md

Lines changed: 31 additions & 166 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
# Encrypt Query Language (EQL)
22

3-
[![Why we built EQL](https://img.shields.io/badge/concept-Why%20EQL-8A2BE2)](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/concepts/WHY.md)
4-
[![Getting started](https://img.shields.io/badge/guide-Getting%20started-008000)](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/tutorials/GETTINGSTARTED.md)
5-
[![CipherStash Proxy](https://img.shields.io/badge/guide-CipherStash%20Proxy-A48CF3)](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/tutorials/PROXY.md)
6-
[![CipherStash Migrator](https://img.shields.io/badge/guide-CipherStash%20Migrator-A48CF3)](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/MIGRATOR.md)
3+
[![Test EQL](https://github.com/cipherstash/encrypt-query-language/actions/workflows/test-eql.yml/badge.svg?branch=main)](https://github.com/cipherstash/encrypt-query-language/actions/workflows/test-eql.yml)
4+
[![Release EQL](https://github.com/cipherstash/encrypt-query-language/actions/workflows/release-eql.yml/badge.svg?branch=main)](https://github.com/cipherstash/encrypt-query-language/actions/workflows/release-eql.yml)
75

86
Encrypt Query Language (EQL) is a set of abstractions for transmitting, storing, and interacting with encrypted data and indexes in PostgreSQL.
97

@@ -12,12 +10,13 @@ Store encrypted data alongside your existing data.
1210
- Encrypted data is stored using a `jsonb` column type
1311
- Query encrypted data with specialized SQL functions
1412
- Index encrypted columns to enable searchable encryption
15-
- Integrate with [CipherStash Proxy](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/tutorials/PROXY.md) for transparent encryption/decryption
13+
- Integrate with [CipherStash Proxy](/docs/tutorials/PROXY.md) for transparent encryption/decryption
1614

1715
## Table of Contents
1816

1917
- [Installation](#installation)
2018
- [CipherStash Proxy](#cipherstash-proxy)
19+
- [Documentation](#documentation)
2120
- [Getting started](#getting-started)
2221
- [Enable encrypted columns](#enable-encrypted-columns)
2322
- [Configuring the column](#configuring-the-column)
@@ -33,11 +32,6 @@ Store encrypted data alongside your existing data.
3332
- [Full-text search (`cs_match_v1`)](#full-text-search-cs_match_v1)
3433
- [Range queries (`cs_ore_64_8_v1`)](#range-queries-cs_ore_64_8_v1)
3534
- [JSON and JSONB support](#json-and-jsonb-support)
36-
- [Configuring the index](#configuring-the-index)
37-
- [Inserting JSON data](#inserting-json-data)
38-
- [Reading JSON data](#reading-json-data)
39-
- [Advanced JSON queries](#advanced-json-queries)
40-
- [EQL payload data format](#eql-payload-data-format)
4135
- [Frequently Asked Questions](#frequently-asked-questions)
4236
- [How do I integrate CipherStash EQL with my application?](#how-do-i-integrate-cipherstash-eql-with-my-application)
4337
- [Can I use EQL without the CipherStash Proxy?](#can-i-use-eql-without-the-cipherstash-proxy)
@@ -49,7 +43,7 @@ Store encrypted data alongside your existing data.
4943

5044
## Installation
5145

52-
The simplest way to get up and running with EQL is to execute the install SQL file directly in your database.
46+
The simplest way to get up and running with EQL is to execute the install SQL file directly in your PostgreSQL database.
5347

5448
1. Download the latest EQL install script:
5549

@@ -65,12 +59,16 @@ The simplest way to get up and running with EQL is to execute the install SQL fi
6559

6660
### CipherStash Proxy
6761

68-
EQL relies on [CipherStash Proxy](https://github.com/cipherstash/encrypt-query-language/blob/main/PROXY.md) for low-latency encryption & decryption.
62+
EQL relies on [CipherStash Proxy](docs/tutorials/PROXY.md) for low-latency encryption & decryption.
6963
We plan to support direct language integration in the future.
7064

65+
## Documentation
66+
67+
You can read more about the EQL concepts and reference guides in the [documentation directory](docs/README.md).
68+
7169
## Getting started
7270

73-
Once the custom types and functions are installed, you can start using EQL in your queries.
71+
Once the custom types and functions are installed in your PostgreSQL database, you can start using EQL in your queries.
7472

7573
### Enable encrypted columns
7674

@@ -119,9 +117,9 @@ SELECT cs_refresh_encrypt_config();
119117
120118
## Storing data
121119

122-
Encrypted data is stored as `jsonb` values in the database, regardless of the original data type.
120+
Encrypted data is stored as `jsonb` values in the PostgreSQL database, regardless of the original data type.
123121

124-
You can read more about the data format [here][#data-format].
122+
You can read more about the data format [here](docs/reference/PAYLOAD.md).
125123

126124
### Inserting Data
127125

@@ -135,7 +133,7 @@ INSERT INTO users (encrypted_email) VALUES (
135133
);
136134
```
137135

138-
Data is stored in the database as:
136+
Data is stored in the PostgreSQL database as:
139137

140138
```json
141139
{
@@ -201,7 +199,7 @@ SELECT cs_add_index_v1(
201199
);
202200
```
203201

204-
You can read more about the index configuration options [here][https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/INDEX.md].
202+
You can read more about the index configuration options [here](docs/reference/INDEX.md).
205203

206204
**Example (Unique index):**
207205

@@ -339,155 +337,15 @@ SELECT field, COUNT(*) FROM users GROUP BY field;
339337

340338
## JSON and JSONB support
341339

342-
EQL supports encrypting, decrypting, and searching JSON and JSONB objects.
343-
344-
### Configuring the index
345-
346-
Similar to how you configure indexes for text data, you can configure indexes for JSON and JSONB data.
347-
The only difference is that you need to specify the `cast_as` parameter as `json` or `jsonb`.
348-
349-
```sql
350-
SELECT cs_add_index_v1(
351-
'users',
352-
'encrypted_json',
353-
'ste_vec',
354-
'jsonb',
355-
'{"prefix": "users/encrypted_json"}' -- The prefix is in the form of "table/column"
356-
);
357-
```
358-
359-
You can read more about the index configuration options [here](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/INDEX.md).
360-
361-
### Inserting JSON data
362-
363-
When inserting JSON data, this works the same as inserting text data.
364-
You need to wrap the JSON data in the appropriate EQL payload.
365-
CipherStash Proxy will **encrypt** the data automatically.
366-
367-
**Example:**
368-
369-
Assuming you want to store the following JSON data:
370-
371-
```json
372-
{
373-
"name": "John Doe",
374-
"metadata": {
375-
"age": 42
376-
}
377-
}
378-
```
379-
380-
The EQL payload would be:
381-
382-
```sql
383-
INSERT INTO users (encrypted_json) VALUES (
384-
'{"v":1,"k":"pt","p":"{\"name\":\"John Doe\",\"metadata\":{\"age\":42}}","i":{"t":"users","c":"encrypted_json"}}'
385-
);
386-
```
387-
388-
Data is stored in the database as:
389-
390-
```json
391-
{
392-
"i": {
393-
"c": "encrypted_json",
394-
"t": "users"
395-
},
396-
"k": "sv",
397-
"v": 1,
398-
"sv": [
399-
...ciphertext...
400-
]
401-
}
402-
```
403-
404-
### Reading JSON data
405-
406-
When querying data, select the encrypted column. CipherStash Proxy will **decrypt** the data automatically.
407-
408-
**Example:**
409-
410-
```sql
411-
SELECT encrypted_json FROM users;
412-
```
413-
414-
Data is returned as:
415-
416-
```json
417-
{
418-
"k": "pt",
419-
"p": "{\"metadata\":{\"age\":42},\"name\":\"John Doe\"}",
420-
"i": {
421-
"t": "users",
422-
"c": "encrypted_json"
423-
},
424-
"v": 1,
425-
"q": null
426-
}
427-
```
428-
429-
### Advanced JSON queries
430-
431-
We support a wide range of JSON/JSONB functions and operators.
432-
You can read more about the JSONB support in the [JSONB reference guide](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/JSON.md).
433-
434-
## EQL payload data format
435-
436-
Encrypted data is stored as `jsonb` with a specific schema:
437-
438-
- **Plaintext payload (client side):**
439-
440-
```json
441-
{
442-
"v": 1,
443-
"k": "pt",
444-
"p": "plaintext value",
445-
"e": {
446-
"t": "table_name",
447-
"c": "column_name"
448-
}
449-
}
450-
```
451-
452-
- **Encrypted payload (database side):**
453-
454-
```json
455-
{
456-
"v": 1,
457-
"k": "ct",
458-
"c": "ciphertext value",
459-
"e": {
460-
"t": "table_name",
461-
"c": "column_name"
462-
}
463-
}
464-
```
465-
466-
The format is defined as a [JSON Schema](./cs_encrypted_v1.schema.json).
467-
468-
It should never be necessary to directly interact with the stored `jsonb`.
469-
CipherStash Proxy handles the encoding, and EQL provides the functions.
470-
471-
| Field | Name | Description |
472-
| ----- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
473-
| s | Schema version | JSON Schema version of this json document. |
474-
| v | Version | The configuration version that generated this stored value. |
475-
| k | Kind | The kind of the data (plaintext/pt, ciphertext/ct, encrypting/et). |
476-
| i.t | Table identifier | Name of the table containing encrypted column. |
477-
| i.c | Column identifier | Name of the encrypted column. |
478-
| p | Plaintext | Plaintext value sent by database client. Required if kind is plaintext/pt or encrypting/et. |
479-
| q | For query | Specifies that the plaintext should be encrypted for a specific query operation. If `null`, source encryption and encryption for all indexes will be performed. Valid values are `"match"`, `"ore"`, `"unique"`, `"ste_vec"`, and `"ejson_path"`. |
480-
| c | Ciphertext | Ciphertext value. Encrypted by Proxy. Required if kind is plaintext/pt or encrypting/et. |
481-
| m | Match index | Ciphertext index value. Encrypted by Proxy. |
482-
| o | ORE index | Ciphertext index value. Encrypted by Proxy. |
483-
| u | Unique index | Ciphertext index value. Encrypted by Proxy. |
484-
| sv | STE vector index | Ciphertext index value. Encrypted by Proxy. |
340+
EQL supports encrypting entire JSON and JSONB data sets.
341+
This warrants a separate section in the documentation.
342+
You can read more about the JSONB support in the [JSONB reference guide](docs/reference/JSON.md).
485343

486344
## Frequently Asked Questions
487345

488346
### How do I integrate CipherStash EQL with my application?
489347

490-
Use CipherStash Proxy to intercept database queries and handle encryption and decryption automatically.
348+
Use CipherStash Proxy to intercept PostgreSQL queries and handle encryption and decryption automatically.
491349
The proxy interacts with the database using the EQL functions and types defined in this documentation.
492350

493351
Use the [helper packages](#helper-packages) to integate EQL functions into your application.
@@ -501,16 +359,23 @@ No, CipherStash Proxy is required to handle the encryption and decryption operat
501359
Data is encrypted using CipherStash's cryptographic schemes and stored in the `cs_encrypted_v1` column as a JSONB payload.
502360
Encryption and decryption are handled by CipherStash Proxy.
503361

504-
## Helper packages
362+
## Helper packages and examples
505363

506364
We've created a few langague specific packages to help you interact with the payloads:
507365

366+
508367
| Language | ORM | Example | Package |
509368
| ---------- | ----------- | ----------------------------------------------------------------- | ---------------------------------------------------------------- |
510-
| Go | Xorm | [Go/Xorm examples](./languages/go/xorm/README.md) | [goeql](https://github.com/cipherstash/goeql) |
511-
| Typescript | Drizzle | [Drizzle examples](./languages/javascript/apps/drizzle/README.md) | [cipherstash/eql](./languages/javascript/packages/eql/README.md) |
512-
| Typescript | Prisma | [Prisma examples](./languages/javascript/apps/prisma/README.md) | [cipherstash/eql](./languages/javascript/packages/eql/README.md) |
513-
| Python | SQL Alchemy | [Python examples](./languages/python/jupyter_notebook/README.md) | |
369+
| Go | Xorm | [Go/Xorm examples](./examples/go/xorm/README.md) | [goeql](https://github.com/cipherstash/goeql) |
370+
| Typescript | Drizzle | [Drizzle examples](./examples/javascript/apps/drizzle/README.md) | [cipherstash/eql](./examples/javascript/packages/eql/README.md) |
371+
| Typescript | Prisma | [Drizzle examples](./examples/javascript/apps/prisma/README.md) | [cipherstash/eql](./examples/javascript/packages/eql/README.md) |
372+
| Python | SQL Alchemy | [Python examples](./examples/python/jupyter_notebook/README.md) | |
373+
374+
### Language specific packages
375+
376+
- [Go](https://github.com/cipherstash/goeql)
377+
- [Javascript](examples/javascript/packages/eql/README.md)
378+
- Python (coming soon)
514379

515380
## Releasing
516381

docs/README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# EQL Documentation
2+
3+
This directory contains the documentation for the Encrypt Query Language (EQL).
4+
5+
## Concepts
6+
7+
The following concepts are available:
8+
9+
- [Why we built EQL](concepts/WHY.md)
10+
11+
## Reference
12+
13+
The following reference guides are available:
14+
15+
- [EQL index configuration](reference/INDEX.md)
16+
- [JSONB and JSON support](reference/JSON.md)
17+
- [Migrating plaintext data](reference/MIGRATOR.md)
18+
- [EQL payload data format](reference/PAYLOAD.md)
19+
20+
## Tutorials
21+
22+
The following tutorials are available:
23+
24+
- [Getting started](tutorials/GETTINGSTARTED.md)
25+
- [Using CipherStash Proxy](tutorials/PROXY.md)

0 commit comments

Comments
 (0)