Skip to content

Commit c044297

Browse files
Merge pull request #71 from firebase/rachelsaunders-bq-importscript
docs(firestore-bigquery-export): update import script instructions for npm info
2 parents 2e8a805 + 4777f39 commit c044297

File tree

1 file changed

+23
-8
lines changed

1 file changed

+23
-8
lines changed

firestore-bigquery-export/guides/IMPORT_EXISTING_DOCUMENTS.md

Lines changed: 23 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,42 @@
1+
The `fs-bq-import-collection` script is for use with the official Firebase Extension [**Export Collections to BiqQuery**](https://github.com/firebase/extensions/tree/master/firestore-bigquery-export).
2+
13
### Overview
24

3-
The import script (`fs-bq-import-collection`) can read all existing documents in a Cloud Firestore collection and insert them into the raw changelog table created by the Export Collections to BigQuery extension. The script adds a special changelog for each document with the operation of `IMPORT` and the timestamp of epoch. This ensures that any operation on an imported document supersedes the import record.
5+
The import script (`fs-bq-import-collection`) can read all existing documents in a Cloud Firestore collection and insert them into the raw changelog table created by the Export Collections to BigQuery extension. The import script adds a special changelog for each document with the operation of `IMPORT` and the timestamp of epoch. This ensures that any operation on an imported document supersedes the import record.
46

5-
You may pause and resume the script from the last batch at any point.
7+
You may pause and resume the import script from the last batch at any point.
68

79
#### Important notes
810

9-
- Run the script over the entire collection **_after_** installing the Export Collections to BigQuery extension; otherwise the writes to your database during the import might not be exported to the dataset.
11+
- You must run the import script over the entire collection **_after_** installing the Export Collections to BigQuery extension; otherwise the writes to your database during the import might not be exported to the dataset.
12+
1013
- The import script can take up to _O(collection size)_ time to finish. If your collection is large, you might want to consider [loading data from a Cloud Firestore export into BigQuery](https://cloud.google.com/bigquery/docs/loading-data-cloud-firestore).
11-
- You will see redundant rows in your raw changelog table:
1214

13-
- If document changes occur in the time between installing the extension and running this import script.
15+
- You will see redundant rows in your raw changelog table if either of the following happen:
16+
17+
- If document changes occur in the time between installing the extension and running the import script.
1418
- If you run the import script multiple times over the same collection.
1519

16-
### Install and run the script
20+
### Run the script
1721

18-
This import script uses several values from your installation of the extension:
22+
The import script uses several values from your installation of the extension:
1923

2024
- `${PROJECT_ID}`: the project ID for the Firebase project in which you installed the extension
2125
- `${COLLECTION_PATH}`: the collection path that you specified during extension installation
2226
- `${DATASET_ID}`: the ID that you specified for your dataset during extension installation
2327

24-
1. Run `npx @firebaseextensions/fs-bq-import-collection`.
28+
Run the import script using [`npx` (the Node Package Runner)](https://www.npmjs.com/package/npx) via `npm` (the Node Package Manager).
29+
30+
1. Make sure that you've installed the required tools to run the import script:
31+
32+
- To access the `npm` command tools, you need to install [Node.js](https://www.nodejs.org/).
33+
- If you use `npm` v5.1 or earlier, you need to explicitly install `npx`. Run `npm install --global npx`.
34+
35+
1. Run the import script via `npx` by running the following command:
36+
37+
```
38+
npx @firebaseextensions/fs-bq-import-collection
39+
```
2540
2641
1. When prompted, enter the Cloud Firestore collection path that you specified during extension installation, `${COLLECTION_PATH}`.
2742

0 commit comments

Comments
 (0)