@@ -6,6 +6,24 @@ A [SQLite](https://sqlite.org) adapter plugin for dbt (data build tool)
6
6
This is an experimental plugin. Please read these docs carefully and
7
7
use at your own risk. Issues and PRs welcome!
8
8
9
+ ## The Use Case
10
+
11
+ SQLite is an embedded SQL database. It can be a good choice if
12
+ your project meets some or all of these criteria:
13
+
14
+ - the database file lives on fast, local storage (not on a network drive)
15
+ - the amount of data is relatively small (GBs, not TBs)
16
+ - you're a data team of one with no need to share access to the database
17
+ - you mainly care about the final results of your pipelines, which are just
18
+ a few models that you export into other systems for multi-user access
19
+ or into BI/viz tools for analysis
20
+ - your project is a proof of concept, to eventually be moved into
21
+ another database or data warehouse platform
22
+
23
+ SQLite can be surprisingly fast, despite the query optimizer not being as
24
+ sophisticated as other databases and data warehouse platforms. Tip: realize
25
+ your models as tables and create helpful indexes in post-hooks to speed up joins.
26
+
9
27
## How to Use This
10
28
11
29
Use the right version:
@@ -46,7 +64,7 @@ dbt_sqlite:
46
64
# connect schemas to paths: at least one of these must be 'main'
47
65
schemas_and_paths : ' main=/my_project/data/etl.db;dataset=/my_project/data/dataset_v1.db'
48
66
49
- # directory where all *.db files are attached as schema, using base filename
67
+ # directory where all *.db files are attached as schema, using base filename
50
68
# as schema name, and where new schema are created. this can overlap with the dirs of
51
69
# files in schemas_and_paths as long as there's no conflicts.
52
70
schema_directory : ' /my_project/data'
0 commit comments