Automated Bank Loan and Investment Status Pipeline — Google Cloud Platform
This repository contains a simplified, demonstration version of a real-world GCP project. It illustrates the structure, configurations, and code required to automate a bank loan and investment status process using Google Cloud Platform services.
| File/Folder | Purpose |
|---|---|
scripts/ |
Python scripts containing SQL queries for extracting data and loading it into BigQuery target tables. Note: The production environment contains 12+ scripts. This demo includes only a subset for clarity. |
accp.yaml |
Pipeline configuration file defining build instructions for the container image, including image name, build context, and other ACCP pipeline parameters. Referenced during pipeline initialization. |
Dockerfile |
Image build specification defining the base image, system dependencies, and Python packages required to execute the scripts. Referenced in accp.yaml. |
requirements.txt |
List of Python dependencies to be installed in the image environment. Referenced in the Dockerfile. |
positions_dag.py |
Apache Airflow DAG for orchestrating and scheduling the end-to-end process. References the container image built from the above files and defines execution logic and scheduling parameters. |
- Code and Configuration — Python scripts, configuration files, and dependencies are stored in this repository.
- Image Build — The ACCP pipeline uses
accp.yaml,Dockerfileandrequirement.txtto build a container image with all required dependencies. - Data Processing — The container runs Python scripts to fetch, transform, and load data into BigQuery.
- Orchestration — Airflow triggers the container execution according to the schedule defined in
positions_dag.py.
