You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/README.md
+5-15Lines changed: 5 additions & 15 deletions
Original file line number
Diff line number
Diff line change
@@ -5,18 +5,7 @@
5
5
6
6
Using Airflow to implement our ETL pipelines.
7
7
8
-
## Table of Contents
9
-
10
-
-[Prerequisites](#prerequisites)
11
-
-[Installation](#installation)
12
-
-[Configuration](#configuration)
13
-
-[BigQuery (Optional)](#bigquery-optional)
14
-
-[Running the Project](#running-the-project)
15
-
-[Local Environment with Docker](#local-environment-with-docker)
16
-
-[Production](#production)
17
-
-[Contact](#contact)
18
-
19
-
---
8
+
[TOC]
20
9
21
10
## Prerequisites
22
11
@@ -26,6 +15,7 @@ Using Airflow to implement our ETL pipelines.
26
15
-[uv]
27
16
28
17
## Installation
18
+
29
19
We use [uv] to manage dependencies and virtual environment.
30
20
31
21
Below are the steps to create a virtual environment using [uv]:
@@ -69,8 +59,8 @@ deactivate
69
59
### BigQuery (Optional)
70
60
71
61
- Set up the Authentication for GCP: <https://googleapis.dev/python/google-api-core/latest/auth.html>
72
-
- After running `gcloud auth application-default login`, you will get a credentials.json file located at `$HOME/.config/gcloud/application_default_credentials.json`.
73
-
- Run `export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"` if you have it.
62
+
- After running `gcloud auth application-default login`, you will get a credentials.json file located at `$HOME/.config/gcloud/application_default_credentials.json`.
63
+
- Run `export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"` if you have it.
74
64
-`service-account.json`: Please contact @david30907d via email or Discord. You do not need this json file if you are running the sandbox staging instance for development.
75
65
76
66
## Running the Project
@@ -111,6 +101,7 @@ make down-dev
111
101
> The difference between production and dev/test compose files is that the dev/test compose file uses a locally built image, while the production compose file uses the image from Docker Hub.
112
102
113
103
#### Use images from Artifacts
104
+
114
105
If you are an authorized maintainer, you can pull the image from the [GCP Artifact Registry].
115
106
116
107
Docker client must be configured to use the [GCP Artifact Registry].
@@ -132,7 +123,6 @@ Available tags:
132
123
-`staging`: when pushing to the staging environment
133
124
-`latest`: when pushing to the production environment
134
125
135
-
136
126
## Contact
137
127
138
128
[PyCon TW Volunteer Data Team - Discord](https://discord.com/channels/752904426057892052/900721883383758879)
0 commit comments