Merge branch 'main' into cip-874-jupyter-notebook-updates

Author: yujiyokoo

.github/workflows/release-eql.yml

@@ -36,10 +36,19 @@ jobs:
       - name: Build EQL release
         run: |
           just build
-          mv release/cipherstash-encrypt-dsl.sql release/cipherstash-eql.sql
+
+      - name: Upload EQL artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: eql-release
+          path: |
+            release/cipherstash-encrypt.sql
+            release/cipherstash-encrypt-uninstall.sql
 
       - name: Publish EQL release artifacts
         uses: softprops/action-gh-release@v2
         if: startsWith(github.ref, 'refs/tags/')
         with:
-          files: release/cipherstash-eql.sql
+          files: |
+            release/cipherstash-encrypt.sql
+            release/cipherstash-encrypt-uninstall.sql

.github/workflows/test-eql.yml

@@ -0,0 +1,51 @@
+name: "Test EQL"
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - ".github/workflows/test-eql.yml"
+      - "sql/*.sql"
+
+  pull_request:
+    branches:
+      - main
+    paths:
+      - ".github/workflows/test-eql.yml"
+      - "sql/*.sql"
+
+  workflow_dispatch:
+
+defaults:
+  run:
+    shell: bash -l {0}
+
+jobs:
+  test:
+    name: "Test EQL SQL components"
+    runs-on: ubuntu-24.04
+
+    strategy:
+      fail-fast: false
+      matrix:
+        postgres-version: [17, 16, 15, 14]
+
+    env:
+      CS_DATABASE__PASSWORD:
+      CS_DATABASE__PORT: 5432
+      CS_DATABASE__NAME: test
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: extractions/setup-just@v1
+
+      - uses: ankane/setup-postgres@v1
+        with:
+          postgres-version: ${{ matrix.postgres-version }}
+          database: ${{ env.CS_DATABASE__NAME }}
+
+      - name: Test EQL
+        run: |
+          just build test
+

.gitignore

@@ -183,3 +183,5 @@ cipherstash-proxy.toml
 
 # build artifacts
 release/
+
+.mise.*

MIGRATOR.md

@@ -0,0 +1,74 @@
+# CipherStash Migrator
+
+The CipherStash Migrator is a tool that can be used to migrate plaintext data in a database to its encrypted equivalent.
+It works inside the CipherStash Proxy Docker container and can handle different data types such as text, JSONB, integers, booleans, floats, and dates.
+By specifying the relevant columns in your table, the migrator will seamlessly encrypt the existing data and store it in designated encrypted columns.
+
+## Prerequisites
+
+- [CipherStash Proxy](PROXY.md)
+- [Have set up EQL in your database](GETTINGSTARTED.md)
+  - Ensure that the columns where data will be migrated already exist.
+
+Here’s a draft for the technical usage documentation for the CipherStash Migrator tool:
+
+## Usage
+
+The CipherStash Migrator allows you to specify key-value pairs where the key is the plaintext column, and the value is the corresponding encrypted column.
+Multiple key-value pairs can be specified, and the tool will perform a migration for each specified column.
+
+### Running the migrator
+
+You will need to SSH into the CipherStash Proxy Docker container to run the migrator.
+
+```bash
+docker exec -it eql-cipherstash-proxy bash
+```
+
+Once inside the container, you have access to the migrator tool.
+
+```bash
+cipherstash-migrator --version
+```
+
+#### Flags
+
+| Flag | Description | Required |
+| --- | --- | --- |
+| `--columns` | Specifies the plaintext columns and their corresponding encrypted columns. The format is `plaintext_column=encrypted_column`. | Yes |
+| `--table` | Specifies the table where the data will be migrated. | Yes |
+| `--database-name` | Specifies the database name. | Yes |
+| `--username` | Specifies the database username. | Yes |
+| `--password` | Specifies the database password. | Yes |
+
+#### Supported data types
+
+- Text
+- JSONB
+- Integer
+- Boolean
+- Float
+- Date
+
+### Example
+
+The following is an example of how to run the migrator with a single column:
+
+```bash
+cipherstash-migrator --columns example_column=example_column_encrypted --table examples --database-name postgres --username postgres --password postgres
+```
+
+If you require additional data types, please [raise an issue](https://github.com/cipherstash/encrypt-query-language/issues)
+
+### Running migrations with multiple columns
+
+To run a migration on multiple columns at once, specify multiple key-value pairs in the `--columns` option:
+
+```bash
+cipherstash-migrator --columns test_text=encrypted_text test_jsonb=encrypted_jsonb test_int=encrypted_int test_boolean=encrypted_boolean --table examples --database-name migrator_test --username postgres --password postgres
+```
+
+## Notes
+
+- Ensure that the corresponding encrypted columns already exist in the table before running the migration.
+- Data migration operations should be tested in a development environment before being executed in production.

PROXY.md

@@ -51,10 +51,11 @@ Populate the following fields with your values:
 
 ## Running the Proxy
 
-To run the proxy, you can use the `start.sh` script in this directory. This script will start the proxy using the configuration in the `cipherstash-proxy.toml` file.
+To run the proxy, you can use `docker compose` to start the proxy using the configuration in the `cipherstash-proxy.toml` file.
+Run the following command from the `cipherstash-proxy` directory:
 
 ```bash
-./start.sh
+docker compose up
 ```
 
 ## Using the Proxy