Skip to content

Commit fbb518c

Browse files
EdouardDemEdouardDem
committed
feat: add readme files
1 parent 857b11e commit fbb518c

File tree

18 files changed

+1112
-880
lines changed

18 files changed

+1112
-880
lines changed

‎README.md

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Directus Sync
2+
3+
The `directus-sync` command-line interface (CLI) provides a set of tools for managing and synchronizing the schema and
4+
collections within Directus across different environments.
5+
6+
# Usage
7+
8+
The CLI is available using the `npx` command.
9+
10+
```shell
11+
npx directus-sync <command> [options]
12+
```
13+
14+
Here's how to use each command in the CLI:
15+
16+
## Global Options
17+
18+
These options can be used with any command to configure the operation of `directus-sync`:
19+
20+
- `-d, --debug`
21+
Display additional logging information. Useful for debugging or verifying what `directus-sync` is doing under the
22+
hood.
23+
24+
- `-u, --directus-url <directusUrl>`
25+
Specify the Directus instance URL. Alternatively, set the `DIRECTUS_URL` environment variable.
26+
27+
- `-t, --directus-token <directusToken>`
28+
Provide the Directus access token. Alternatively, set the `DIRECTUS_TOKEN` environment variable.
29+
30+
- `--no-split`
31+
Indicates whether the schema snapshot should be split into multiple files. By default, snapshots are split.
32+
33+
- `--dump-path <dumpPath>`
34+
Set the base path for the dump. This must be an absolute path. The default
35+
is `"./directus-config"`.
36+
37+
- `--collections-path <collectionPath>`
38+
Specify the path for the collections dump, relative to the dump path. The default is `"collections"`.
39+
40+
- `--snapshot-path <snapshotPath>`
41+
Specify the path for the schema snapshot dump, relative to the dump path. The default is `"snapshot"`.
42+
43+
- `-h, --help`
44+
Display help information for the `directus-sync` commands.
45+
46+
## Commands
47+
48+
### Pull
49+
50+
```shell
51+
npx directus-sync pull
52+
```
53+
54+
Retrieves the current schema and collections from Directus and stores them locally. This command does not modify the
55+
database.
56+
57+
### Diff
58+
59+
```shell
60+
npx directus-sync diff
61+
```
62+
63+
Analyzes and describes the difference (diff) between your local schema and collections and the state of the Directus
64+
instance. This command is non-destructive and does not apply any changes to the database.
65+
66+
### Push
67+
68+
```shell
69+
npx directus-sync push
70+
```
71+
72+
Applies the changes from your local environment to the Directus instance. This command pushes your local schema and
73+
collection configurations to Directus, updating the instance to reflect your local state.
74+
75+
### Untrack
76+
77+
```shell
78+
npx directus-sync untrack --collection <collection> --id <id>
79+
```
80+
81+
Removes tracking from an element within Directus. You must specify the collection and the ID of the element you wish to
82+
stop tracking.
83+
84+
### Tracked Elements
85+
86+
`directus-sync` tracks the following Directus collections:
87+
88+
- dashboards
89+
- flows
90+
- operations
91+
- panels
92+
- permissions
93+
- roles
94+
- settings
95+
- webhooks
96+
97+
For these collections, data changes are committed to the code, allowing for replication on other Directus instances. A
98+
mapping table links Directus instance IDs with SyncIDs, managed by the `directus-extension-sync`.
99+
100+
## Dependency: `directus-extension-sync`
101+
102+
To utilize the `directus-sync` tool, it is imperative to install the `directus-extension-sync` extension on your
103+
Directus instance. This extension acts as a bridge between `directus-sync` and Directus, managing the crucial mapping
104+
table that correlates the SyncIDs with Directus's internal IDs.
105+
106+
### Installation
107+
108+
The `directus-extension-sync` must be added to each Directus instance involved in the synchronization process, whether
109+
as a source or a destination. Follow the installation instructions provided in
110+
the `directus-extension-sync` [repository](https://www.npmjs.com/package/directus-extension-sync)
111+
to add this extension to your Directus setup.
112+
113+
## How It Works
114+
115+
`directus-sync` operates on a tagging system similar to Terraform, where each trackable element within Directus is
116+
assigned a unique synchronization identifier (SyncID). This system is key to enabling version control for the
117+
configurations and schema within Directus. Here is a step-by-step explanation of how `directus-sync` functions:
118+
119+
### Tagging and Tracking
120+
121+
Upon execution of the `pull` command, `directus-sync` will:
122+
123+
1. Scan the specified Directus collections, which include dashboards, flows, operations, panels, permissions, roles,
124+
settings, and webhooks.
125+
2. Assign a SyncID to each element within these collections if it doesn't already have one.
126+
3. Commit the data of these collections into code, allowing for versioning and tracking of configuration changes.
127+
128+
This SyncID tagging facilitates the replication of configurations across different instances of Directus while
129+
maintaining the integrity and links between different entities.
130+
131+
### Mapping Table
132+
133+
Since it's not possible to add tags directly to entities within Directus, `directus-sync` uses a mapping table that
134+
correlates the SyncIDs with the internal IDs used by Directus. This mapping is essential for the synchronization
135+
process, as it ensures that each element can be accurately identified and updated across different environments.
136+
137+
### Synchronization Process
138+
139+
The synchronization process is split into two main commands:
140+
141+
- `diff`: This command performs a comparison between the local JSON files generated by `pull` and the current state of a
142+
Directus instance. It outlines the elements that need to be created, updated, or deleted to achieve synchronization.
143+
144+
- `push`: This command executes the actual synchronization plan, applying the necessary changes to the Directus
145+
instance. It handles dependencies and circular dependencies carefully by potentially running the synchronization
146+
process multiple times until the Directus instance is fully in sync with the JSON definitions.
147+
148+
### Schema Management
149+
150+
The Directus schema, which defines the data modeling and user interface, is managed by the Directus API. However, for
151+
better code repository management, `directus-sync` stores the schema elements in separate files organized within a clear
152+
directory structure. This separation allows developers to easily track changes to the schema and apply version control
153+
principles to the database structure.
154+
155+
### Non-Tracked Elements and Ignored Fields
156+
157+
Elements that are not meant to be tracked, such as user activities and logs, are not affected by the synchronization
158+
process. Certain fields are specifically ignored during synchronization because they are not relevant for version
159+
control purposes, such as creation dates and the identity of the user who created an entity.
160+
161+
### Strengths of `directus-sync`
162+
163+
The strength of `directus-sync` lies in its ability to maintain consistent and reproducible configurations across
164+
multiple environments. It ensures that only the necessary changes are made, avoiding unnecessary recreation of
165+
configurations and maintaining the relationships between tracked and non-tracked entities. This selective updating is
166+
what makes `directus-sync` a robust tool for managing Directus instances in a team or multi-environment setup.
167+
168+
By following these mechanisms, `directus-sync` streamlines the development workflow, allowing for local changes to be
169+
efficiently deployed to various environments, all while keeping the Directus instances synchronized and
170+
version-controlled.

‎api/README.md

Lines changed: 37 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,43 @@
1-
# Directus Sync
1+
# `directus-extension-sync`
22

3-
Allow versioning of Directus configurations and manage synchronizations between environments.
3+
## Overview
4+
5+
The `directus-extension-sync` is an extension that manages the mapping between
6+
synchronization identifiers (SyncIDs) and Directus's internal entity IDs. This extension is a critical dependency for
7+
the `directus-sync` CLI tool, enabling it to perform version control and synchronization tasks across different Directus
8+
instances.
9+
10+
## Features
11+
12+
- **ID Mapping**: Maintains a mapping table linking SyncIDs with Directus's internal IDs.
13+
- **Initialization**: Automatically creates the mapping table upon first use.
14+
- **CRUD Operations**: Provides endpoints to create, read, update, and delete mappings.
15+
16+
## Installation
17+
18+
In your Directus installation root, run:
19+
20+
```bash
21+
npm install directus-extension-sync
22+
```
23+
24+
Then, restart Directus.
25+
26+
## Usage
27+
28+
The extension provides a set of RESTful endpoints that are used internally by the `directus-sync` tool to manage
29+
SyncIDs. These endpoints include:
30+
31+
- `GET /directus-extension-sync/table/:table/sync_id/:sync_id`: Retrieve a mapping by SyncID.
32+
- `GET /directus-extension-sync/table/:table/local_id/:local_id`: Retrieve a mapping by local ID.
33+
- `GET /directus-extension-sync/table/:table`: Retrieve all mappings for a table.
34+
- `POST /directus-extension-sync/table/:table`: Create a new mapping entry.
35+
- `DELETE /directus-extension-sync/table/:table/sync_id/:sync_id`: Remove a mapping by SyncID.
36+
- `DELETE /directus-extension-sync/table/:table/local_id/:local_id`: Remove a mapping by local ID.
437

538
## Development
639

7-
Link the package to a development project:
40+
Link the package to a development Directus instance:
841

942
```bash
1043
npm run link /path/to/directus/extensions
@@ -15,3 +48,4 @@ The run in development mode:
1548
```bash
1649
npm run dev
1750
```
51+

‎cli/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@
33
node_modules
44
dist
55
directus-config
6+
README.md

‎cli/README.md

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎cli/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
"format": "prettier --write \"**/*.{js,jsx,ts,tsx,json,md,gql,mjs}\"",
1212
"start": "ts-node-esm src/index.ts",
1313
"test": "jest",
14-
"prepublishOnly": "npm install && npm run format && npm test && npm run build"
14+
"prepublishOnly": "cp ../README.md ./README.md && npm install && npm run format && npm test && npm run build"
1515
},
1616
"keywords": [],
1717
"author": "Edouard Demotes-Mainard <edouard@tractr.net>",

‎cli/src/index.ts

Lines changed: 63 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -1,70 +1,82 @@
11
import 'reflect-metadata';
2-
import {program} from 'commander';
3-
import {disposeContext, initContext, logEndAndClose, logErrorAndStop, runDiff, runPull, runPush,} from './lib';
2+
import { program } from 'commander';
3+
import {
4+
disposeContext,
5+
initContext,
6+
logEndAndClose,
7+
logErrorAndStop,
8+
runDiff,
9+
runPull,
10+
runPush,
11+
} from './lib';
412
import Path from 'path';
5-
import {runUntrack} from "./lib/commands/untrack";
13+
import { runUntrack } from './lib/commands/untrack';
614

715
const defaultDumpPath = Path.join(process.cwd(), 'directus-config');
816
const defaultSnapshotPath = 'snapshot';
917
const defaultCollectionsPath = 'collections';
1018

1119
program
12-
.option('-d, --debug', 'display more logging', false)
13-
.option(
14-
'-u, --directus-url <directusUrl>',
15-
'Directus URL. Can also be set via DIRECTUS_URL env var',
16-
)
17-
.option(
18-
'-t, --directus-token <directusToken>',
19-
'Directus access token. Can also be set via DIRECTUS_TOKEN env var',
20-
)
21-
.option(
22-
'--no-split',
23-
'should the schema snapshot be split into multiple files',
24-
true,
25-
)
26-
.option(
27-
'--dump-path <dumpPath>',
28-
'the base path for the dump, must be an absolute path',
29-
defaultDumpPath,
30-
)
31-
.option(
32-
'--collections-path <collectionPath>',
33-
'the path for the collections dump, relative to the dump path',
34-
defaultCollectionsPath,
35-
)
36-
.option(
37-
'--snapshot-path <snapshotPath>',
38-
'the path for the schema snapshot dump, relative to the dump path',
39-
defaultSnapshotPath,
40-
);
20+
.option('-d, --debug', 'display more logging', false)
21+
.option(
22+
'-u, --directus-url <directusUrl>',
23+
'Directus URL. Can also be set via DIRECTUS_URL env var',
24+
)
25+
.option(
26+
'-t, --directus-token <directusToken>',
27+
'Directus access token. Can also be set via DIRECTUS_TOKEN env var',
28+
)
29+
.option(
30+
'--no-split',
31+
'should the schema snapshot be split into multiple files',
32+
true,
33+
)
34+
.option(
35+
'--dump-path <dumpPath>',
36+
'the base path for the dump, must be an absolute path',
37+
defaultDumpPath,
38+
)
39+
.option(
40+
'--collections-path <collectionPath>',
41+
'the path for the collections dump, relative to the dump path',
42+
defaultCollectionsPath,
43+
)
44+
.option(
45+
'--snapshot-path <snapshotPath>',
46+
'the path for the schema snapshot dump, relative to the dump path',
47+
defaultSnapshotPath,
48+
);
4149

42-
registerCommand('pull', 'get the schema and collections and store them locally', runPull);
4350
registerCommand(
44-
'diff',
45-
'describe the schema and collections diff. Does not modify the database.',
46-
runDiff
51+
'pull',
52+
'get the schema and collections and store them locally',
53+
runPull,
54+
);
55+
registerCommand(
56+
'diff',
57+
'describe the schema and collections diff. Does not modify the database.',
58+
runDiff,
4759
);
4860
registerCommand('push', 'push the schema and collections', runPush);
4961
registerCommand('untrack', 'stop tracking of an element', runUntrack)
50-
.option('-c, --collection <collection>', 'the collection of the element')
51-
.option('-i, --id <id>', 'the id of the element to untrack');
62+
.option('-c, --collection <collection>', 'the collection of the element')
63+
.option('-i, --id <id>', 'the id of the element to untrack');
5264

5365
program.parse(process.argv);
5466

5567
function registerCommand(
56-
name: string,
57-
description: string,
58-
action: (options?: any) => Promise<void>
68+
name: string,
69+
description: string,
70+
action: (options?: any) => Promise<void>,
5971
) {
60-
return program
61-
.command(name)
62-
.description(description)
63-
.action((options) => {
64-
return initContext(program.opts())
65-
.then(() => action(options))
66-
.catch(logErrorAndStop)
67-
.then(disposeContext)
68-
.then(logEndAndClose);
69-
});
72+
return program
73+
.command(name)
74+
.description(description)
75+
.action((options) => {
76+
return initContext(program.opts())
77+
.then(() => action(options))
78+
.catch(logErrorAndStop)
79+
.then(disposeContext)
80+
.then(logEndAndClose);
81+
});
7082
}

0 commit comments

Comments
 (0)