Report Service

The Reports building block facilitates the creation and engagement with dashboards.

Supported Operating Systems

• Ubuntu (Recommended: Version 22.04)

Setup Options

Note: This guide focuses on setting up services using Docker. For a streamlined and efficient setup, it is recommended to use Dockerized Services & Dependencies with the provided Docker-Compose file.

Dockerized Services & Dependencies Using Docker-Compose File

Dockerized Services & Dependencies

Expectation

By diligently following the outlined steps, you will successfully establish a fully operational data service application setup.

Prerequisites

Setting Up curl, git, and netstat on Ubuntu

1. Install curl

`curl` is used for making HTTP requests from the command line.

🔹 Check if curl is installed:

  curl --version

🔹 If not installed, install it using:

  sudo apt update && sudo apt install -y curl

2. Install git

`git` is required for cloning repositories and managing version control.

🔹 Check if git is installed:

  git --version

🔹 If not installed, install it using:

  sudo apt update && sudo apt install -y git

3. Install netstat (via `net-tools`)

`netstat` is used to check network connections and ports.

🔹 Check if netstat is installed:

  netstat -tulnp

🔹 If not installed, install it using:

  sudo apt update && sudo apt install -y net-tools

To set up the data service application, ensure you have Docker and Docker Compose installed on your system. For Ubuntu users, detailed installation instructions for Docker found here : Install Docker engine on Ubuntu and for docker-compose follow this documentation: How To Install and Use Docker Compose on Ubuntu.

Installation

Create report Directory: Establish a directory titled reports.

Example Command: mkdir reports && cd reports/

Note: All commands are run from the reports directory.

Caution: Before proceeding, please ensure that the ports given here are available and open. It is essential to verify their availability prior to moving forward. You can run the below command in your terminal to check this:

   for port in 3000 2181 9092 8081 5432 5050 9092 8080; do
       if sudo lsof -iTCP:$port -sTCP:LISTEN &>/dev/null || sudo netstat -tulnp | grep -w ":$port" &>/dev/null; then
           echo "Port $port is in use"
       else
           echo "Port $port is available"
       fi
   done

Steps to Set Up Dockerized Services

1. Download and execute the main setup script: Execute the following command in your terminal from the reports directory.

  curl -OJL https://raw.githubusercontent.com/ELEVATE-Project/data-pipeline/main/Documentation/Docker-setup/setup.sh && chmod +x setup.sh && sudo ./setup.sh

Note: The script will download the necessary files and launch the services in Docker. Once all services are up and running, follow the provided steps and enter the required inputs when prompted by the script. For instructions on setting up PgAdmin and Metabase, please refer to the documentation.

2. General Instructions:
   • All containers which are part of the docker-compose can be gracefully stopped by pressing Ctrl + C in the same terminal where the services are running.
   • To start all services and dependencies:

     sudo docker compose --env-file ./config.env up -d

   • To stop all containers and remove volumes:

     sudo ./docker-compose down -v

Setting Up pgAdmin [optional]

Set Up pgAdmin

1. Open pgAdmin by navigating to http://localhost:5050 in your browser.
2. Log in with the default credentials:
   • Username: admin@example.com
   • Password: admin
3. Click on Servers → Create → Server.
4. Under the General tab, provide a name for your server (e.g., PostgresServer).
5. Under the Connection tab:
   • Host: postgres
   • Port: 5432
   • Maintenance Database: dev-project-analytics
   • Username: postgres
   • Password: password
6. Click Save to connect to the PostgreSQL database.

Setting Up Metabase

Configure Metabase

1. Open Metabase by navigating to http://localhost:3000 in your browser.

2. Select the Preferred Language: Choose language: English -> Next

3. Setup super admin login credentials

   • Set First Name elevate
   • Set Last Name user
   • set Company or team name shikshalokam
   • Set Email user@shikshalokam.org
   • Set Password elevate@123 -> Next \

   

4. Setup the database connection.

   • Set up your first database → Select PostgreSQL

   

   • then click on show more options

5. Enter the database credentials

   • Display name: elevateData
   • Host: postgres
   • Port: 5432
   • Database name: dev-project-analytics
   • Username: postgres
   • Password: password

   

6. Click Take me to Metabase to complete the setup and start using Metabase.

7. Setting up the state_name and district_name data type in the metabase.

   • Go to setting → Admin setting → Table Metadata → select the projects table
   • Set the data type 'state' for state_name and 'city' for district_name.
   • This settings will save automatically.

   Here is the screen shot for setting up the state_name semantic type.

   Here is the screen shot for setting up the district_name semantic type.

Onboarding user managemennt

Onboarding user management

Here is the csv for checking the other reports.
user_data.csv . Also you can interact with the api to onboard new users.

   curl --location 'http://localhost:8080/api/csv/upload' \
   --header 'Authorization: 4a2d9f8e-3b56-47c1-a9d3-e571b8f0c2d9' \
   --form 'file=@"/app/Documentation/Docker-setup/user_data.csv"' 

To list out the users

   curl --location 'http://localhost:8080/api/csv/list' \
   --header 'Authorization: 4a2d9f8e-3b56-47c1-a9d3-e571b8f0c2d9' 

Team

Open Source Dependencies

Several open source dependencies that have aided data-pipeline and dashboards development: