docs(firestore-bigquery-export): improve cross-project IAM docs (#2261)

Author: cabljac

firestore-bigquery-export/CHANGELOG.md

@@ -4,6 +4,8 @@ feat - add basic materialized views support
 
 fix - do not add/update clustering if an invalid clustering field is present.
 
+docs - improve cross-project IAM documentation
+
 ## Version 0.1.56
 
 feat - improve sync strategy by immediately writing to BQ, and using cloud tasks only as a last resort

firestore-bigquery-export/POSTINSTALL.md

@@ -4,30 +4,30 @@ You can test out this extension right away!
 
 1.  Go to your [Cloud Firestore dashboard](https://console.firebase.google.com/project/${param:BIGQUERY_PROJECT_ID}/firestore/data) in the Firebase console.
 
-1.  If it doesn't already exist, create the collection you specified during installation: `${param:COLLECTION_PATH}`
+2.  If it doesn't already exist, create the collection you specified during installation: `${param:COLLECTION_PATH}`
 
-1.  Create a document in the collection called `bigquery-mirror-test` that contains any fields with any values that you'd like.
+3.  Create a document in the collection called `bigquery-mirror-test` that contains any fields with any values that you'd like.
 
-1.  Go to the [BigQuery web UI](https://console.cloud.google.com/bigquery?project=${param:BIGQUERY_PROJECT_ID}&p=${param:BIGQUERY_PROJECT_ID}&d=${param:DATASET_ID}) in the Google Cloud Platform console.
+4.  Go to the [BigQuery web UI](https://console.cloud.google.com/bigquery?project=${param:BIGQUERY_PROJECT_ID}&p=${param:BIGQUERY_PROJECT_ID}&d=${param:DATASET_ID}) in the Google Cloud Platform console.
 
-1.  Query your **raw changelog table**, which should contain a single log of creating the `bigquery-mirror-test` document.
+5.  Query your **raw changelog table**, which should contain a single log of creating the `bigquery-mirror-test` document.
 
     ```
     SELECT *
     FROM `${param:BIGQUERY_PROJECT_ID}.${param:DATASET_ID}.${param:TABLE_ID}_raw_changelog`
     ```
 
-1.  Query your **latest view**, which should return the latest change event for the only document present -- `bigquery-mirror-test`.
+6.  Query your **latest view**, which should return the latest change event for the only document present -- `bigquery-mirror-test`.
 
     ```
     SELECT *
     FROM `${param:BIGQUERY_PROJECT_ID}.${param:DATASET_ID}.${param:TABLE_ID}_raw_latest`
     ```
 
-1.  Delete the `bigquery-mirror-test` document from [Cloud Firestore](https://console.firebase.google.com/project/${param:BIGQUERY_PROJECT_ID}/firestore/data).
+7.  Delete the `bigquery-mirror-test` document from [Cloud Firestore](https://console.firebase.google.com/project/${param:BIGQUERY_PROJECT_ID}/firestore/data).
     The `bigquery-mirror-test` document will disappear from the **latest view** and a `DELETE` event will be added to the **raw changelog table**.
 
-1.  You can check the changelogs of a single document with this query:
+8.  You can check the changelogs of a single document with this query:
 
     ```
     SELECT *
@@ -58,33 +58,39 @@ Enabling wildcard references will provide an additional STRING based column. The
 
 By default, the extension exports data to BigQuery in the same project as your Firebase project. However, you can configure it to export to a BigQuery instance in a different Google Cloud project. To do this:
 
-1. During installation, set the `BIGQUERY_PROJECT_ID` parameter to your target BigQuery project ID.
+1. During installation, set the `BIGQUERY_PROJECT_ID` parameter as your target BigQuery project ID.
 
-2. After installation, you'll need to grant the extension's service account the necessary BigQuery permissions on the target project. You can use our provided scripts:
+2. Identify the service account on the source project associated with the extension. By default, it will be constructed as `ext-<extension-instance-id>@project-id.iam.gserviceaccount.com`. However, if the extension instance ID is too long, it may be truncated and 4 random characters appended to abide by service account length limits.
+
+3. To find the exact service account, navigate to IAM & Admin -> IAM in the Google Cloud Platform Console. Look for the service account listed with "Name" as "Firebase Extensions <your extension instance ID> service account". The value in the "Principal" column will be the service account that needs permissions granted in the target project.
+
+4. Grant the extension's service account the necessary BigQuery permissions on the target project. You can use our provided scripts:
 
 **For Linux/Mac (Bash):**
 ```bash
 curl -O https://raw.githubusercontent.com/firebase/extensions/master/firestore-bigquery-export/scripts/grant-crossproject-access.sh
 chmod +x grant-crossproject-access.sh
-./grant-crossproject-access.sh -f SOURCE_FIREBASE_PROJECT -b TARGET_BIGQUERY_PROJECT [-i EXTENSION_INSTANCE_ID]
+./grant-crossproject-access.sh -f SOURCE_FIREBASE_PROJECT -b TARGET_BIGQUERY_PROJECT [-i EXTENSION_INSTANCE_ID] [-s SERVICE_ACCOUNT]
 ```
 
 **For Windows (PowerShell):**
 ```powershell
 Invoke-WebRequest -Uri "https://raw.githubusercontent.com/firebase/extensions/master/firestore-bigquery-export/scripts/grant-crossproject-access.ps1" -OutFile "grant-crossproject-access.ps1"
-.\grant-crossproject-access.ps1 -FirebaseProject SOURCE_FIREBASE_PROJECT -BigQueryProject TARGET_BIGQUERY_PROJECT [-ExtensionInstanceId EXTENSION_INSTANCE_ID]
+.\grant-crossproject-access.ps1 -FirebaseProject SOURCE_FIREBASE_PROJECT -BigQueryProject TARGET_BIGQUERY_PROJECT [-ExtensionInstanceId EXTENSION_INSTANCE_ID] [-ServiceAccount SERVICE_ACCOUNT]
 ```
 
 **Parameters:**
 For Bash script:
 - `-f`: Your Firebase (source) project ID
 - `-b`: Your target BigQuery project ID
 - `-i`: (Optional) Extension instance ID if different from default "firestore-bigquery-export"
+- `-s`: (Optional) Service account email. If not provided, it will be constructed using the extension instance ID
 
 For PowerShell script:
 - `-FirebaseProject`: Your Firebase (source) project ID
 - `-BigQueryProject`: Your target BigQuery project ID
 - `-ExtensionInstanceId`: (Optional) Extension instance ID if different from default "firestore-bigquery-export"
+- `-ServiceAccount`: (Optional) Service account email. If not provided, it will be constructed using the extension instance ID
 
 **Prerequisites:**
 - You must have the [gcloud CLI](https://cloud.google.com/sdk/docs/install) installed and configured

firestore-bigquery-export/scripts/grant-crossproject-access.ps1

@@ -1,11 +1,12 @@
 # Help message
 function Show-Help {
-    Write-Host "Usage: .\grant-crossproject-access.ps1 -FirebaseProject <PROJECT_ID> -BigQueryProject <PROJECT_ID> [-ExtensionInstanceId <INSTANCE_ID>]"
+    Write-Host "Usage: .\grant-crossproject-access.ps1 -FirebaseProject <PROJECT_ID> -BigQueryProject <PROJECT_ID> [-ExtensionInstanceId <INSTANCE_ID>] [-ServiceAccount <SERVICE_ACCOUNT>]"
     Write-Host
     Write-Host "Parameters:"
     Write-Host "  -FirebaseProject        Firebase (source) project ID"
     Write-Host "  -BigQueryProject        BigQuery project ID where dataset will be created"
     Write-Host "  -ExtensionInstanceId    Extension instance ID (default: firestore-bigquery-export)"
+    Write-Host "  -ServiceAccount         Service account email (optional, will be constructed if not provided)"
     exit 1
 }
 
@@ -18,11 +19,16 @@ param(
     [string]$BigQueryProject,
 
     [Parameter(Mandatory=$false)]
-    [string]$ExtensionInstanceId = "firestore-bigquery-export"
+    [string]$ExtensionInstanceId = "firestore-bigquery-export",
+
+    [Parameter(Mandatory=$false)]
+    [string]$ServiceAccount = ""
 )
 
-# Construct service account email
-$ServiceAccount = "ext-${ExtensionInstanceId}@${FirebaseProject}.iam.gserviceaccount.com"
+# Construct service account email if not provided
+if (-not $ServiceAccount) {
+    $ServiceAccount = "ext-${ExtensionInstanceId}@${FirebaseProject}.iam.gserviceaccount.com"
+}
 
 Write-Host "Using service account: $ServiceAccount"
 Write-Host "Adding BigQuery permissions to $ServiceAccount on project: $BigQueryProject"

firestore-bigquery-export/scripts/grant-crossproject-access.sh

@@ -2,12 +2,13 @@
 
 # Help message
 function show_help {
-    echo "Usage: $0 -f FIREBASE_PROJECT -b BIGQUERY_PROJECT -i EXTENSION_INSTANCE_ID"
+    echo "Usage: $0 -f FIREBASE_PROJECT -b BIGQUERY_PROJECT -i EXTENSION_INSTANCE_ID [-s SERVICE_ACCOUNT]"
     echo
     echo "Options:"
     echo "  -f    Firebase (source) project ID"
     echo "  -b    BigQuery project ID where dataset will be created"
     echo "  -i    Extension instance ID (default: firestore-bigquery-export)"
+    echo "  -s    Service account email (optional, will be constructed if not provided)"
     echo "  -h    Show this help message"
     exit 1
 }
@@ -16,11 +17,12 @@ function show_help {
 EXT_INSTANCE_ID="firestore-bigquery-export"
 
 # Parse command line arguments
-while getopts "f:b:i:h" opt; do
+while getopts "f:b:i:s:h" opt; do
     case $opt in
         f) FIREBASE_PROJECT="$OPTARG";;
         b) BIGQUERY_PROJECT="$OPTARG";;
         i) EXT_INSTANCE_ID="$OPTARG";;
+        s) SERVICE_ACCOUNT="$OPTARG";;
         h) show_help;;
         ?) show_help;;
     esac
@@ -32,8 +34,10 @@ if [ -z "$FIREBASE_PROJECT" ] || [ -z "$BIGQUERY_PROJECT" ]; then
     show_help
 fi
 
-# Construct service account email
-SERVICE_ACCOUNT="ext-${EXT_INSTANCE_ID}@${FIREBASE_PROJECT}.iam.gserviceaccount.com"
+# Construct service account email if not provided
+if [ -z "$SERVICE_ACCOUNT" ]; then
+    SERVICE_ACCOUNT="ext-${EXT_INSTANCE_ID}@${FIREBASE_PROJECT}.iam.gserviceaccount.com"
+fi
 
 echo "Using service account: $SERVICE_ACCOUNT"
 echo "Adding BigQuery permissions to $SERVICE_ACCOUNT on project: $BIGQUERY_PROJECT"