|
| 1 | +The `fs-bq-import-collection` script is for use with the official Firebase Extension [**Export Collections to BiqQuery**](https://github.com/firebase/extensions/tree/master/firestore-bigquery-export). |
| 2 | + |
1 | 3 | ### Overview
|
2 | 4 |
|
3 |
| -The import script (`fs-bq-import-collection`) can read all existing documents in a Cloud Firestore collection and insert them into the raw changelog table created by the Export Collections to BigQuery extension. The script adds a special changelog for each document with the operation of `IMPORT` and the timestamp of epoch. This ensures that any operation on an imported document supersedes the import record. |
| 5 | +The import script (`fs-bq-import-collection`) can read all existing documents in a Cloud Firestore collection and insert them into the raw changelog table created by the Export Collections to BigQuery extension. The import script adds a special changelog for each document with the operation of `IMPORT` and the timestamp of epoch. This ensures that any operation on an imported document supersedes the import record. |
4 | 6 |
|
5 |
| -You may pause and resume the script from the last batch at any point. |
| 7 | +You may pause and resume the import script from the last batch at any point. |
6 | 8 |
|
7 | 9 | #### Important notes
|
8 | 10 |
|
9 |
| -- Run the script over the entire collection **_after_** installing the Export Collections to BigQuery extension; otherwise the writes to your database during the import might not be exported to the dataset. |
| 11 | +- You must run the import script over the entire collection **_after_** installing the Export Collections to BigQuery extension; otherwise the writes to your database during the import might not be exported to the dataset. |
| 12 | + |
10 | 13 | - The import script can take up to _O(collection size)_ time to finish. If your collection is large, you might want to consider [loading data from a Cloud Firestore export into BigQuery](https://cloud.google.com/bigquery/docs/loading-data-cloud-firestore).
|
11 |
| -- You will see redundant rows in your raw changelog table: |
12 | 14 |
|
13 |
| - - If document changes occur in the time between installing the extension and running this import script. |
| 15 | +- You will see redundant rows in your raw changelog table if either of the following happen: |
| 16 | + |
| 17 | + - If document changes occur in the time between installing the extension and running the import script. |
14 | 18 | - If you run the import script multiple times over the same collection.
|
15 | 19 |
|
16 |
| -### Install and run the script |
| 20 | +### Run the script |
17 | 21 |
|
18 |
| -This import script uses several values from your installation of the extension: |
| 22 | +The import script uses several values from your installation of the extension: |
19 | 23 |
|
20 | 24 | - `${PROJECT_ID}`: the project ID for the Firebase project in which you installed the extension
|
21 | 25 | - `${COLLECTION_PATH}`: the collection path that you specified during extension installation
|
22 | 26 | - `${DATASET_ID}`: the ID that you specified for your dataset during extension installation
|
23 | 27 |
|
24 |
| -1. Run `npx @firebaseextensions/fs-bq-import-collection`. |
| 28 | +Run the import script using [`npx` (the Node Package Runner)](https://www.npmjs.com/package/npx) via `npm` (the Node Package Manager). |
| 29 | + |
| 30 | +1. Make sure that you've installed the required tools to run the import script: |
| 31 | + |
| 32 | + - To access the `npm` command tools, you need to install [Node.js](https://www.nodejs.org/). |
| 33 | + - If you use `npm` v5.1 or earlier, you need to explicitly install `npx`. Run `npm install --global npx`. |
| 34 | + |
| 35 | +1. Run the import script via `npx` by running the following command: |
| 36 | + |
| 37 | + ``` |
| 38 | + npx @firebaseextensions/fs-bq-import-collection |
| 39 | + ``` |
25 | 40 |
|
26 | 41 | 1. When prompted, enter the Cloud Firestore collection path that you specified during extension installation, `${COLLECTION_PATH}`.
|
27 | 42 |
|
|
0 commit comments